Victims of the silent minority

Labour is so far ahead in the polls that the party will probably win the general election. But the pollsters will struggle to predict the majority because the problems that tripped them up in 1992 still exist, argues Ivor Crewe

Neil Kinnock was not the only surprise loser at the last general election. The other was the polling industry. Never before had so much egg landed on its collective face. The final polls, published on election day and conducted over the two previous days, showed the Conservative and Labour parties level. NOP and MORI put Labour ahead, ICM declared a dead heat and Gallup put the Conservatives ahead by a mere half per cent; the average of the four was a Labour lead of 0.9 per cent. Everything pointed to a hung parliament and a coalition government. In the event the Conservatives won with an overall majority of 21 seats, 7.6 per cent ahead of Labour in the popular vote. The polls were out by over 8 points, having underestimated the actual Conservative vote by 4 per cent and overestimated the actual Labour vote by 4 per cent.

Opinion polls have been wrong before, notably in 1970 when the Conservatives under Edward Heath won by 3 per cent and 30 seats; even though three of the four final polls forecast a clear Labour victory. But in 1970 the polls completed their interviewing three days before the election and thus failed to detect a late swing to the Conservatives. In 1992 the interviewing continued right up to election day and the margin of error was much greater. Despite the growing sophistication of the polling organisations, 1992 was the worst disaster in their 50-year history.

Inaccuracy in the polls would matter little if all it meant was discomfiture in the market research industry. In fact much more is at stake. A consistent message from incessant polls shapes the course of the election campaign and can affect, perhaps decisively, the outcome. Accurate polls would have shown the Conservatives slightly but consistently ahead during the campaign and probably for some months before. As a result the election might have been called earlier; Neil Kinnock might have wooed the Liberal Democrats with hints about electoral reform sooner in the campaign; and anti-Conservative tactical voting might have been boosted.

By understating the Conservative support, the polls probably helped the Conservatives in the final days of the campaign to mobilise their fainthearts and win back deserters to the Liberal Democrats - a crucial increment of support. The polls' failure in 1992 is therefore marked by irony and paradox. By mistakenly placing Labour ahead, they may have helped to create a Conservative government. Had they correctly placed the Conservatives ahead, they would have made a hung parliament - and thus a minority or coalition Labour government - a more likely outcome.

What went wrong? A plethora of postmortems pointed the finger at sampling error, undersized samples, "quickie" one-day polls, type of interview (weekday versus weekend, on-street versus at-home, face-to-face versus telephone), non-registration by poll-tax evaders, even deliberate lying by cunning Conservatives. The Market Research Society's investigating panel of academics and pollsters, chaired by the doyen of electoral studies, David Butler, showed that all these explanations were groundless. Its report, The Opinion Polls and the 1992 General Election, concluded that three factors contributed to the polls' 8.6 point error.

The first was a small swing to the Conservatives on the eve of election day. In the 1992 election the electorate was exceptionally volatile, torn between a Conservative government with a poor record and a Labour opposition it did not trust: fully 21 per cent changed their mind during the campaign, compared with 19, 15 and 13 per cent in the three previous elections. Before-and-after panel surveys showed that in the 36 hours up to election day some who declared they would abstain actually voted, and vice versa; some "don't knows" made up their minds; and some "firmly" for one party changed parties. All these last-minute shifts helped the Conservatives.

The second culprit was faulty sample design. Almost all the polls adopted a "quota" sample: interviewers were told to interview a pre-set number of people according to their age, gender, social class and employment status such that the overall sample would be socially representative of the electorate. Quota samples are frowned upon by the purists, but properly designed and executed, they are the most practical basis for conducting polls and have a good record. They suffered, however, from two flaws. First, the selection of quota variables was unsatisfactory. The variables should correlate with party preferences, in order to maximise the political representativeness of the sample, and should be easy for interviewers to apply. But gender is barely related to voting (contrary to myth, the "gender gap" is minuscule) and age only weakly. Social class correlates with voting much more, but is a poorly defined concept and notoriously difficult for busy interviewers to apply accurately to voters. Interviewers tended to misassign respondents up the social scale, including too many working class and thus Labour voters in their quotas. Second, the national social benchmarks used by the polling companies to set quotas had not kept pace with socio-demographic changes in the electorate - older, fewer council tenants, more middle class - and thus were biased in Labour's favour.

The third contributory factor to the polls' debacle was the "spiral of silence", the term coined by the German social researcher, Elizabeth Noelle-Neumann, for people's tendency to remain silent or evasive about their views when they sense a hostile climate of opinion around them. The climate of opinion in the 1992 campaign was anti-Conservative: even the Conservative media derided the Conservative campaign as lacklustre while Labour was portrayed as taking the moral high ground on taxes and spending. In polls "silence" takes the form of declining to answer the "how will you vote?" question, or responding "don't know" or refusing to be interviewed at all. These "silent" respondents are excluded from the opinion poll results; but their answers to other questions revealed that most were closet Conservatives: the majority had voted Conservative in 1987, preferred Major to Kinnock as prime minister and thought the Conservatives to be better at managing the economy.

Today's polls are again indicating a Conservative defeat, although much more emphatically than last time. Have they got it right or are they repeating the mistakes of 1992, but on a grander scale? Current polls are almost certainly continuing to underestimate the true level of Conservative support. There is ample evidence of an even more pervasive spiral of silence than in 1992. Typically 60 per cent of the "don't knows" recall voting Conservative in 1992, but the figure has risen whenever a major political event such as VAT on fuel or the BSE beef crisis has hurt the Government; moreover the proportion climbs even higher among the "don't knows" claiming to be "certain to vote". There is equally compelling evidence that those refusing to be interviewed are shy Conservatives. For example, the polls ask respondents how they voted in 1992; if they are to be believed Labour won the election by between 5 and 7 per cent. This recall bias is partly due to faulty memory: about one out of five respondents confuse the last general election with a recent local or European election or they "project" current voting intentions back onto their past vote. It owes a little to the death of elderly (and predominantly Conservative) voters since 1992. But it is also due to Conservatives refusing to be interviewed, which leads to a pro-Labour bias in the samples.

The polling organisations have undoubtedly tried to deal with the problems that led to the 1992 fiasco. Each of the big four has radically changed its methods. Gallup switched from face-to-face interviews to phoned ones in December and adjusted its raw figures to include secretive Conservative (and Labour and Liberal Democrat) voters. ICM has made similar changes and also asks respondents to cast a secret vote in a ballot box rather than declare their vote to an interviewer. NOP has drawn up new quotas based on census data, up-to-date national social surveys and weight by recalled vote to what a perfect sample would record of the last election.

Adjustments of this kind were put to the test in the 1994 European elections and worked well. In the four final polls the unadjusted results overestimated Labour's lead by 6 percentage points (on average) whereas the adjusted data was accurate to within 0.5 points. However, these promising results far from guarantee that the polls will perform as well in the general election. For one thing, the difference made to the raw results by the adjustment varies from one polling organisation to another. In the 30-month period from July 1994 to January 1997, the adjustments made by ICM and Gallup have cut Labour's lead by an average of 9-10 percentage points whereas those made by MORI and NOP have made a difference of only 4-5 points. Over the same period Labour's average lead in the adjusted polls has varied from 18 per cent in ICM to 22 per cent in MORI and NOP to 24 per cent in Gallup (before it switched to telephone interviewing). Even after adjustments one or more of the four main polls must be seriously askew.

The standing of the parties in the most recent polls, after adjustments, is Conservative 33 per cent, Labour 49 per cent, Liberal Democrats 13 per cent, which would produce a Labour majority of 161 if repeated at a general election. To overcome such a huge lead in the next ten weeks the Conservatives would need to recover at three times the rate that any government has managed so close to an election. Thus the polls are unlikely to pick the wrong winner this time. Whether they can come close to forecasting Labour's majority is a different matter. This time they are more likely than Labour to be the surprise losers.

Ivor Crewe is vice chancellor and professor of government at the University of Essex.

Top tipsters

The team headed by Gordon Reece of Bristol University has forecast the last four general elections remarkably successfully. For the 1997 election they predict:

Labour 328

Tories 284

Lib Dems 22.

If correct, this would leave Labour two seats short of an overall majority.