Flawed psychology

Stephen Gibson acknowledges that there may be a “train wreck looming” for one subfield of social psychology but not for the field more generally (“This is bigger than social psychology”, Letters, 24 January). This subfield comprises research on social priming in which researchers use subtle environmental cues to try to unconsciously influence aspects of people’s social behaviour.

We can put Gibson’s claim to the test. Suppose we look at the current issue of one prestigious social psychology journal and select an article at random. Here is what I found when I did so. The article in question is not on social priming and was genuinely chosen at random (and is in fact the only one I’ve read from this issue). Its authors shall remain anonymous.

In the first of two reported experiments, neither of the key measures reaches the conventional level of statistical significance. Despite this, the authors report that the results provide support for their experimental hypothesis. In the second experiment, one of the key measures reaches significance but not the other, and the authors argue that their hypothesis is even more firmly supported. However, the significant result is present only among female participants. Male participants do not show the effect, the authors clearly did not make this analytic choice a priori, and they fail to report a statistical analysis across the entire sample. It would be hard to find a more flagrant example of the employment of an “experimenter degree of freedom” to extract a significant result from one’s data - precisely the sort of practice that has been highlighted in recent debates about psychology. Yet the article passed the journal’s peer-review process.

It would be comforting to think that low replicability is a problem only in social priming research, but it extends far into mainstream social psychology, as this example highlights. Of course, it doesn’t end there: false positives are no doubt being reported in all areas of psychology as well as, probably, science. But social psychologists must recognise their own acute vulnerability to poor research practices and consider how to get their house in order.

David Shanks, Head, Division of Psychology and Language Sciences, University College London