Unscientific social science is trading under false pretences

The question of whether there is, or can be, a social science has been a contentious issue throughout my 50-plus years of being a “social scientist”. I remember how, on my return from a first year of studying sociology at university, my old English teacher denied that there could be any such thing; and my response at the time did not convince him. Of course, the case against social science has long been made, not least by philosophers, from Leo Strauss to Peter Winch.

Whether social research is scientific is not a simple question, because the answer depends on what we mean by “science”, and because “social science” is such a large and diverse set of fields. Nevertheless, I suggest that there is much work by social scientists that trades falsely under the label.

There are multiple reasons. One is external pressure to produce large numbers of research publications in conditions that lack the resources necessary to do this while sustaining quality. A second is that there are practical or political commitments on the part of researchers that encourage bias – or at least exaggeration of the likely validity of what are viewed as positive findings, and the rubbishing of those regarded as unwelcome.

Unscientific work takes a variety of forms. One involves deploying standard techniques with insufficient thought about whether their use is justified given the nature of the data or the aims of the research. This is illustrated by the failure of long-running debates about the abuse of significance testing and the misuse of interviews to constrain much social science practice.

Some research uses sophisticated forms of quantitative analysis whose requirements are not met by the data employed. In the case of non-experimental research, often only relatively weak control is exercised over potential causal factors other than those being investigated. With experimental work, such as randomised controlled trials, it is often uncertain if general conclusions can be drawn about what happens “in the wild”. And both frequently suffer from the threat of major measurement error.

These problems reflect the sheer difficulties involved in studying social phenomena, arising from both their complexity and the limitations of the research strategies available. What is attempted may not be impossible – although sometimes it is – but much more caution is required about the likely validity of the results produced than is commonly exercised. “Trust in numbers” needs to be severely restrained.

A second kind of unscientific social science is qualitative in character, involving no attempt to measure and control variables. Yet often, the questions addressed are causal ones that demand some form of this if they are to be answered effectively. Instead, even while denying that they are engaged in causal analysis, many researchers proceed as if they were able to read off causal relationships straight from their data, based on theoretical assumptions that have not themselves been tested.

The result, at best, is conflicting findings and general confusion. At worst, it is a spurious consensus generated by shared bias and a lack of scientific integrity. This can be fuelled by the widespread view that producing knowledge is insufficient warrant for social research: that to be worthwhile it should have some practical or political impact – it must “make a difference”. Very often, this leads to research designed to provide evidence for a conclusion whose validity was assumed before the investigation began.

There are also social scientists who believe that the very claim to scientific knowledge is ethically or politically unacceptable because they view it as “epistemic domination” that supports the socio-political status quo. They misread it as necessarily blaming people from oppressed or marginalised communities for their own situations, and silencing those who protest.

From this point of view, the only legitimate approach is one that subverts the claimed authority of social science and amplifies “subaltern” voices coming from the disempowered. While those who take this stance may reject the label of science, they nevertheless gain access to funding, and to publication in journals and books, under its auspices. If, in these contexts, they were to announce that their sole aim was to spread their own political opinions, they would probably get little financial support.

There are also researchers who present their work as literature or art, with the concept of social science expanded to incorporate this. But is this legitimate? Sometimes what is produced amounts to little more than agit-prop. Rarely does it meet high literary or artistic standards.

In highlighting the problem of unscientific social science, I may be criticised for undermining the case for public funding of social research – at a time when fake news is rife and the need for sound knowledge is greater than ever. I certainly do not deny the importance of research: I have devoted much of my life to it. But if social scientists do not work hard to check that their conclusions are true, and do not limit themselves to what can be justified on that basis, they too are in the fake news business.

Martyn Hammersley is emeritus professor of educational and social research at the Open University.