Researchers may be close to understanding thedeepest workings of our universe. Steve Farrar talks to a young scientist whose book about 'the final theory' has persuaded a new generation to brave the badlands of theoretical physics.
The problem with Stephen Hawking's A Brief History of Time is not that its cutting edge has been blunted by the advances of the past decade - getting left behind is an occupational hazard faced by all science books. No, the problem is that it is too brief.
So says Brian Greene, the brilliant young theoretical physicist whose own attempt to explain science's effort to discover a "theory of everything" - The Elegant Universe - has just won the Aventis Prize for Science Books, formerly the Rhone-Poulenc Prize.
Like Hawking's multimillion copy bestseller, The Elegant Universe attempts to demystify one of the most mind-bending areas of intellectual endeavour. Greene, of Columbia and Cornell universities, does so at much greater length than Cambridge University's Lucasian professor yet it does not seem to have put anyone off. Quite the opposite.
"A Brief History of Time is a great book, but its briefness makes it somewhat more challenging to read. I had the luxury of length to explain things," Greene says.
Eighteen months in the writing, The Elegant Universe has sold hundreds of thousands of copies. It was nominated for a Pulitzer Arts prize and the newly released paperback edition has been on The New York Times bestseller list for weeks.
Dare to subtitle a book Superstrings, Hidden Dimensions and the Quest for the Ultimate Theory and surely the best you can hope for are worthy reviews and perhaps a chat with Melvyn Bragg on Radio 4 -even if it is clear, uncondescending and deals with important matters. But Greene's irrepressible enthusiasm has persuaded a new generation to brave the dizzying badlands of theoretical physics.
It is hard not to be curious when he declares: "We have an opportunity to realise a goal that humanity has considered for thousands of years - to understand the deepest workings of the universe. For the first time in the history of science, I think there's a chance that we are hot on the trail of the final theory."
So what is all the fuss about?
String theory is an astonishingly ambitious attempt to bring together the two great intellectual triumphs of 20th-century physics - general relativity and quantum mechanics. It has long been the goal of physics to reconcile the differences between these two unlikely bedfellows. Many have tried and failed.
Albert Einstein's general theory of relativity is an ingenious description of gravity that tells us what the universe is like on a large scale, from a falling apple to spinning galaxies. We know it works. Predictions made by Einstein's theory, even notions about the distortion of time, have been painstakingly matched against the real world and have emerged unscathed.
At the opposite end of the scale is the baffling realm of quantum mechanics. This theory reveals that at an atomic level, everything is inherently frenetic, particles spontaneously fizz in and out of existence and it is quite within the bounds of possibility to be in two places at one time. It might defy common sense, yet if it were not true, modern electronics, among other things, would not function.
When relativity looks at space, it sees order. When quantum mechanics takes a peek, frenzied chaos greets the eye. Yet if there really is a theory that can explain everything, it has to show that the two can be made to comfortably dovetail.
String theory has been around for several decades, flitting in and out of fashion as new calculations and insights emerge. It proposes that deep down inside everything is an unimaginably small loop of one-dimensional string.
Pull apart an atom and, Russian doll-like, you will find a nucleus that, in turn, is made out of neutrons and protons. Inside each of these, are clusters of particles called quarks. According to the theory, the sub-atomic strip show does not stop there - each quark, indeed every fundamental particle, contains a loop of string.
And that is not all. There are also loops within the particles that carry forces, strings for gravity, electromagnetic, strong and weak nuclear forces. Each loop vibrates with a distinct pitch that determines what it is - one distinct vibration for an electron, for example, another for a photon of light. No one has seen a string - if they exist, they are so small as to be beyond the reach of the most advanced technology we can envisage.
"Broadly speaking, the equations that describe how strings vibrate are the same as those that describe the vibrations of a violin string - what we have within the universe is a string ensemble," Greene says.
While we have yet to pick out an individual note from this cosmic symphony, on paper the performance is, according to Greene, quite beautiful. That is, if you can stomach a universe of 11 dimensions rather than four and can get your head around the notion that the one-dimensional strings are joined by a host of multi-dimensional entities, most of which defy simple description. These are the prerequisites required to make the latest incarnation of the string theory universe work.
Edward Witten, the field's unquestioned pioneer, has called the latest breakthrough M theory, though it is not even clear what "M" stands for - matrix, magic, mother of all theories (Witten recently suggested murky). Nevertheless, it is causing great excitement as it appears to solve one of the knottiest conundrums, namely that there are not one but five different formulations of string theory, all of which seem to work equally well.
M theory suggests that the five may just be different aspects of some elusive central concept. How do we account for those extra dimensions? Apparently, they are coiled up tightly out of sight, though they might influence the universe by altering the way the loops of string vibrate.
Greene and his peers expect to be working on string theory for a long while yet. New mathematics is being created to drive the new physics forwards but even then it may well prove to be wrong when, at some point in the future, science can devise an experiment to test the theory. So far it has not.
In this field, it is a basic requirement to be brilliant, and there is no doubting Greene's intelligence and ability. At the age of five, he was setting himself massive arithmetical problems. He spent an entire day laboriously calculating a light year in miles while his friends played ball in the park. At 12, his teacher sent him with a note to find someone to tutor him at nearby Columbia University - he had exhausted all the maths and physics textbooks at his Manhattan high school.
"It is very rare to have a young prodigy in history or English literature.You need experience and maturity. Mathematics is different. Learn a few rules and you can do things that no one has done before. To many young people that's very exciting," Greene says.
From Columbia the enthusiastic youngster went to Harvard, studied at Oxford as a Rhodes scholar - where he first came into contact with string theory - and then returned to Columbia. Now 38, he has built a reputation as a daring scientist. Recent work includes insights into how the fabric of space might be torn in extreme circumstances. But being a good scientist does not necessarily make for a good communicator. The son of a one-time vaudeville performer and voice coach, Greene likes the limelight. His public lectures were the stage from which he could explain his science,but they also helped launch his literary career. He felt in his element when tackling the really awkward questions that the public can conjure up.
After a talk, people would come up to him eager to learn more about this amazing theory. "There wasn't an up-to-the-moment book I could point them towards, so I decided to try to create one that the reader could tackle with no prior knowledge," he says.
His success means he now finds himself a minor celebrity, in demand for chat shows, pursued for interviews and recently even called upon to help inject some real science into the time-twisting movie Frequency.
Sometimes, it does all get in the way of his pursuit of science, he confesses. But the nature of his work schedule is more accommodating than you might imagine. "By and large I find I cannot do more than four hours of good research in a day - your brain does begin to get a little bit muddled after intense periods of concentration," he says. Leaving plenty of time to explain string theory to a television host.
The Elegant Universe is published by Jonathan Cape, Pounds 18.99.