First right, second left...

Whose thinking underpins those manifesto pledges that make or break political parties? In the run-up to the next election, Harriet Swain kicks off a new series on the brains behind Britain's politicians by talking to politics professor John Gray, the former Thatcherite who switched his allegiance to new Labour.

The radiator in John Gray's room at the London School of Economics works only at full blast or not at all and the air is stifling. But once settled into an analysis of the modern political scene, Gray appears barely to notice, simply loosening his collar and brushing from time to time at a brow hidden beneath slightly over-long brown hair.

Attitudes towards Gray, LSE professor of European thought, blow just as fiercely hot and cold. He has his admirers. Conservative social security spokesman David Willetts, who was taught by him at Oxford, describes Gray as a great tutor and "an important political thinker". "I think he will always be intellectually on the move because he is sensitive both to the latest academic developments and to the ways in which the real world conflicts with them," he says.

But Gray also has his detractors. Alan Ryan, warden of New College, Oxford, who supervised him for his PhD, says: "I have always been completely mystified by the sudden swings from one thing to the other. I was rather appalled by the contempt for argument in his high-libertarian phase and I'm not much more persuaded by his communitarian, anti-globalisation phase than I was by the alternative."

Gray is not surprised by such radically different views of his works. He openly calls himself a "contrarian", describing his role as "to chasten, reduce and make more humble". Where academics can be most useful, he says, is in "pointing to a distance between consensus perceptions and realities".

He has certainly done that. In the early 1980s, he was seen as a passionate free marketeer, an active member of rightwing think-tanks, who helped shape the tenets of Thatcherite thinking. But just as Margaret Thatcher was at the peak of her powers, he began to change his mind.

It was a slow process, he says, but influenced by particular policies such as the poll tax, something he says he always opposed. "My main objection at the time was that it wasn't going to be recognised as legitimate because it was an attack on liberty."

He also became alarmed when people began to think Thatcher could do anything. What he could not accept was the "hubris", a concept he fundamentally dislikes.

He became uncomfortable with the new right's "indifference, bordering on contempt, for the casualties of economic progress - for the losers" and the "extremely hubristic view that difficult and intractable political problems can be solved for ever and transcended through structures of thought".

In particular, he disliked the "hardening into a global ideology of what had been a ragbag of local solutions to British problems". Privatisation, for example, which could work in Britain, where less than 20 per cent of industry was owned by the government, but not in the Soviet bloc, where 90 per cent plus of the industrial and agricultural economy had been state-owned for years.

The idea that a set of measures desirable in one country could be transferred to another was something he believed Conservatism shared with Marxism - and in his dislike of Marxism he has always been consistent.

Evidence of his changing outlook emerged towards the end of the 1980s in books such as Beyond the New Right and Enlightenment's Wake: Politics and Culture at the Close of the Common Age. In Beyond the New Right he suggested that Britain's historic inheritance of liberal institutions and practices was now being endangered by new right market fundamentalism, rather than, as previously, by left-liberal policy and ideology. Enlightenment's Wake argued that western Conservatism, particularly in the United States, was merely a variation of the Enlightenment dream of a universal civilisation, based on universal reason. This was not an idea suited to "an age in which political life is dominated by renascent particularisms, militant religions and resurgent ethnicities".

More recent work includes a couple of books further stressing the instability of global free markets - Endgames: Questions in Late Modern Political Thought and False Dawn: The Delusions of Global Capitalism, in which he predicted the collapse of the Asian tiger economies.

Since then, his thinking has come steadily closer to that of Labour under Tony Blair and his support for new Labour is difficult to shake. He praises the present government for having people at the centre who are deeply interested in political thought - both immediate policy issues and longer-term questions - for being "open-minded and eclectic" in seeking ideas and for welcoming and absorbing critical comment.

The Thatcher and Major governments upheld an ideological framework that was exclusionary, he says: "You either signed up to the underlying ideology, however frayed at the edges it was, or you didn't." This government, by contrast, is often criticised for not having an ideology and he applauds that. "It does need a framework of what a modern society is and what its needs are and what can be achieved. There is one shaping up. I do not criticise the government for not having what has in the past always led to tragic and often farcical consequences."

His role, he says, is to identify longer-term problems and trends - looking ahead not only to the next election but well beyond it - and to help shape the climate of thought in which policies are made and/or changed.

For example, he identified early a blind spot in the development of genetically modified foods, which had been led by the interests of technology and business but appeared to ignore the likely objections from the public.

Public sensitivity to risk, weakened respect for authority and demands for a higher quality of life are part of being a late modern society, he says. Concern for the environment is a sign not of anti-modernity but of ultra-modernity.

Accused of anti-Americanism and pessimism, particularly since his most recent books have predicted the world economy falling apart, social collapse and the destruction of natural resources, he responds that where he appears over-gloomy he has usually been closer to the truth than consensus opinion. Realism is not what people want. They prefer views that offer new solutions or a return to an imaginary golden past.

"I'm a pessimist in that I don't expect human future to be different from the past," Gray says. "History doesn't end. There are no paradigms. Many problems aren't solved but are only coped with, and Europe, in particular, is an old continent whose potential for disorder is not exhausted."

Environmental issues are one of Gray's primary interests - he writes for Resurgence, the journal of ecological thought. The importance of the public's views is another - hence his recognition of the part played by the media in shaping public attitudes and his own use of newspapers, in particular, to disseminate ideas. "For the kind of ideas we are talking about, the most important media I think is newspapers," he says. "A lot of debate is carried on in newspapers."

As for his political views, he describes them as "John Stuart Mill, liberal ones". "There is lots wrong with Mill, in particular his assumptions about progress, but I share the values he held." Otherwise, Gray claims no attachment to any political party or ideology. What fascinates him is the interplay between ideas and political practice "because it is richer than simple thinking about it can be".

But thinking has stood Gray in good stead for many years. He was born 51 years ago and brought up in the northeast, where his father was a carpenter. He was educated at grammar school, then Oxford.

His first degree was in philosophy, politics and economics. His doctorate was on Mill. It was then, in the early 1970s, that he became consciously interested in politics. He was aware of the student radicalism all around him in 1968, its opposition to Vietnam and the march on the US embassy. He did not take part, but went to watch.

What fascinated him was the disjunction between political ideas and political realities. By the time he became a fellow of Jesus College, Oxford, aged 28 - a post he was to hold for the next 22 years - he was also beginning to feel that the Labour party had run out of steam. He was not persuaded that setting up a new party, such as the SDP, would be the answer. It was then that he turned to the kind of free-market liberalism driving Conservative Party thinking.

Willetts describes him as being at the forefront of ideas seen as daring and radical in 1970s Oxford. Since then, he has changed his mind, Gray says, "both because I thought differently and because the world is different. What we want is a framework of thinking that isn't held to as an article of faith, but one that we are prepared to modify, alter and abandon if it doesn't serve the interests and values for which it was developed."

Certainly he prides himself on never allowing his listeners to become too comfortable.