A devil’s advocate in every lab would drive better science

Collaboration between researchers has become more and more the norm, as witnessed by the increasing number of authors per paper. But most collaboration remains between members of single research groups. The challenge for group leaders is to maintain an environment that fosters successful collaboration.

In my more than two decades of running the Center for Adaptive Behavior and Cognition at a Max Planck Institute in Berlin (and, previously, in Munich), one of the most helpful heuristics, or rules of thumb, I have learned was: “Be sure to include a contrarian”.

Overconfidence about the correctness of our own beliefs has been claimed to be one of our most robust cognitive biases. Possibly hard-wired, it has been blamed for many human disasters; a case in point is the 2008 financial crisis, where it supposedly diverted attention from reckless practices and the excessive fragility of the financial system.

Typically, overconfidence is demonstrated in experiments that ask people general knowledge questions, such as “Which city is further south: New York or Rome?” If you aren’t sure, your intuition probably leads you to say Rome because it is warmer. If asked, you may say you are 90 per cent confident. But New York is actually further south.

One of my PhD students and I discovered that outside such counterintuitive cases, however, people’s confidence about the relative latitudes of big cities in a region is highly realistic; overconfidence vanishes. Nevertheless, after our findings were published, the student announced: “I’ll be the first to prove our theory wrong.”

He set up a study that asked Germans “easy” questions about the hundred most populated cities in Germany and “hard” questions about the hundred most populated cities in the US. To our surprise, people gave a slightly higher number of correct answers about cities in the US – despite knowing less about them.

This led to the discovery of what we called the recognition heuristic: “If you have heard of one city but not the other, infer that the recognised city is larger”. Our volunteers could not use the heuristic for German cities because they were familiar with them all.

Underlying this heuristic was a newly recognised phenomenon, the less-is-more effect: under certain conditions, people with less knowledge about something make more accurate inferences. This discovery formed the foundation of the group’s research on effective heuristics in the real world and has been applied to a wide range of areas, from medicine to business.

The experience taught me how important persistent critique is. Groupthink is the rule rather than the exception in academia, probably contributing to many irreproducible research findings. Contrarians, who dare to question what others take for granted, might annoy colleagues, but they do a great service to the entire group. By always insisting on more evidence, they increase the chances of discovering something nobody has been looking for.

Before I moved back to Germany from the University of Chicago, I worried about becoming intellectually isolated at the top of the hierarchy, with no one daring to criticise my ideas head-on. I therefore invited my best graduate students at Chicago, who had never hesitated to debate my thoughts, to join the group. These contrarians served as “bodyguards”, protecting everyone from running into harsh and justified criticism outside the safe confines of the group.  

There is one caveat. Simply hiring someone with a keen eye for error will be useless or even harmful if the group’s working culture is defined by defensiveness, rather than openness to criticism. Not being allowed to express their observations and questions openly can disgruntle contrarians, so it’s important to develop other heuristics that promote an open culture.

One of these is to hire researchers from different disciplines who work on the same topic: this ensures no one knows everything and everyone can learn from everyone else. There are also useful spatial heuristics. In my experience, if group members are on different floors, or in different buildings, they interact half as much as when located on the same floor. Open doors signal that interaction is welcome. And although a daily tea break may seem a waste of time, it is very effective. Chatting about personal things creates trust; discussing what you are working on speeds up the flow of discovery.

Before writing this piece, I emailed about 100 former lab members and associates to ask about their recollections of contrarians. To my surprise, many perceived themselves as having been the contrarian. One of them, a particularly insistent questioner, said a great strength of the group had been the possibility of speaking up without fear of any repercussions – and then took the opportunity to point out a problem with something I had written!

Contrarians shape the intellectual and social climate of a group. In turn, they shape the quality of its science.

Gerd Gigerenzer is director of the Harding Center for Risk Literacy at the University of Potsdam. He is former director of the Center for Adaptive Behavior and Cognition (ABC) at the Max Planck Institute for Human Development, Berlin. This piece is based on his recent paper “Simple heuristics to run a research group”, published in PsyCh Journal.