Confirmatory research ‘boring and stressful’, says Harvard study

Scholars who conduct studies that seek to confirm a hypothesis are more prone to stress, low motivation and boredom than those who carry out more open-ended investigations, according to a new analysis highlighting the “unintended negative consequences” caused by the rise of preregistered reports.

To combat what many saw as alarming rates of false positive results, many researchers have begun to preregister their intended research goals prior to starting experiments − with this type of “confirmatory research” often seen as more robust than “exploratory” studies that seek to discern trends or significant results after data sets have been collected.

But a new paper by Harvard University scholars has raised concerns that the recent embrace of preregistered reports may not only be stifling creativity in research but causing harm to researchers’ well-being.

In the study, published in the journal Organizational Behavior and Human Decision Processes, researchers argue that “engaging in a pre-registration task reduced researchers’ propensity to explore” and was linked to “heightened anxiety”, particularly among women and junior scholars.

The conclusions follow an experiment in which about 450 behavioural scientists were asked to assess a hypothetical data set of 1,000 responses to a survey on whether yoga improved happiness. Those who were asked to predict likely conclusions spent less time analysing data and were “less likely to discover an interesting but non-hypothesized interaction”, even after being prompted to return to the data to rerun different analyses.

A “focus on confirmation can impede exploration”, says the study by behavioural scientists Leslie John, Hanne Collins and Ashley Whillans, which urges institutions to consider whether they are striking the right balance between confirmatory and exploratory research.

Researchers were also asked to react to descriptions of confirmatory and exploratory research, with the results suggesting that they “found exploratory research more enjoyable, motivating and interesting − and less anxiety-inducing, frustrating [and] boring”.

“These studies raise the possibility that emphasising confirmation can shift researchers away from exploration, and that such a shift could degrade the subjective experience of conducting research,” the study concludes.

The study also highlights what it calls “prediction preoccupation” – a level of anxiety linked to conducting research, including choosing the right question to investigate.

“Although both exploration and confirmation are essential to rigorous scientific research, in practice, confirmation may preclude exploration, and hence, rigour might come at the expense of joy,” it suggests.

jack.grove@timeshighereducation.com