Reprimand for ‘p-hacking’ is ‘important moment’ for science

Scientific integrity experts have welcomed a landmark decision by a Dutch university to reprimand a researcher found guilty of “p-hacking”.

In what is believed to be a first for the Netherlands, and possibly for Europe, a researcher at Leiden University was found to have “acted in violation of scientific integrity by incorrectly using statistical methods (p-hacking) by continuously conducting statistical tests during the course of an experiment”.

The researcher had also acted improperly by “supplementing the research population with the aim of improving the chances of obtaining a statistically significant result,” according to a report by the Netherlands Board on Research Integrity, which agreed with Leiden that the researcher had “violated standards of academic integrity”.

The result was revealed by Daniel Lakens, an experimental psychologist at the human-technology interaction group at Eindhoven University of Technology, who noted that similar infractions had, in previous years, been described by Dutch disciplinary panels as “minor misdemeanours”.

“People have been very hesitant to call p-hacking a violation of research integrity, so it is interesting how things have evolved,” Dr Lakens told Times Higher Education. “We have treated this as a grey area for a long time but this is the first case in the Netherlands where it has been treated as a clear breach of scientific integrity.

“It feels like an important moment – I hope it will mark the next step in how we deal with this problem,” said Dr Lakens, whose personal blog, The 20 per cent Statistician, has explored issues of data handling and research ethics.

He explained, however, that the p-hacking violation had only emerged following an investigation into more serious breaches of research ethics, including taking blood from a patient without ethical consent. “If the researcher had not done this other bad stuff, we would probably have never heard about this case,” he said.

The identity of the researcher or the sanctions imposed by Leiden have not been made public, added Dr Lakens, noting that the Dutch university system is unusually “transparent” on research integrity because it publishes records of university disciplinary investigations.

The Dutch case follows a number of high-profile retractions of journal articles in the US following allegations of p-hacking. These include the case of food researcher Brian Wansink, a Cornell University professor known for his high-profile studies on “all-you-can-eat” restaurants, who retired in 2018 after his institution apologised for his “academic misconduct” related to data manipulation, which has seen 15 of his papers retracted.

David Sanders, associate professor of biological sciences at Purdue University, said it was important that p-hacking cases were taken seriously, particularly if they involved intentional deceit, which was worse than simply practising “sloppy” science.

“If one knows that what one is doing – p-hacking, for example – is unacceptable and does it anyway then it is unethical behaviour,” said Dr Sanders, who said that co-authors of p-hackers might also be culpable.

“If there is an example of unacceptable statistical manipulation in an article of which one is an author and this fact is indicated to one and yet one denies that there is a problem and refuses to correct it, then that is also unethical behaviour,” he said, adding that the “failure to correct major errors through untruthful denial might indeed be considered a form of falsification”.

jack.grove@timeshighereducation.com