A closer look at the evidence 1

The sense of persecution that pervades comments from academics whose work has been "rubbished" by the Government ("Ministers vilify researchers", December 1) is misplaced. All the examples cited came directly or indirectly from the social sciences. The idea that such research evidence is infallible and therefore should dictate social and educational policy is not only naive but misguided.

The experimental paradigm of the natural sciences permits multiple replications of research findings before a new industrial product or drug is introduced. Such validation across all contexts to which a policy is to apply is rare, if not impossible, in the cross-sectional survey and ethnographically based studies coming from the social sciences - largely because of cost but also for methodological reasons.

I am glad, for example, that politicians appear to have rejected the early mantra based on the "evidence" that "class size does not matter", which was used to justify much lower investment per pupil in the public compared with the private sector and was based on research that typically failed to take account of "selection bias". In other cases, such as President Nixon's pulling the plug on compensatory education for disadvantaged children, the evidence came from faulty modelling that failed to adjust for measurement error.

Hence today, the "evidence" is rightly seen as only one of the components of the political judgment that dictates what the Government does - important though it is. The other components are the climate of opinion and accountability to political agendas presented in party manifestos. Better to see the evidence as something that politicians need to confront rather than follow slavishly. Its influence will be mediated indirectly by public opinion as much as it is directly from researchers.

This does not justify the "rubbishing" or vilification by politicians of overconfident individual researchers who have failed to recognise that the evidence they produce is always provisional. It should be properly seen as supplying entry points to dialogue to arrive at optimum conclusions in which there is always the risk of getting the political decision wrong.

John Bynner
Bedford Group for Life Course and Statistical Studies
Institute of Education