How to make research impact pass the marshmallow test

You probably know about the marshmallow test. Developed by psychologists in the 1960s, it purported to show that children who defer gratification, waiting for 15 minutes after being offered a marshmallow in order to receive a second one, enjoy better cognitive and behavioural outcomes later on than children who prefer to eat the marshmallow immediately and forgo the second.

However a recent extension of the experiment shows that the mothers’ education and social background largely explain differences in the tendency to defer gratification. Moreover, among children with similar social backgrounds, deferring gratification makes little difference to future success. Other research suggests why some children do not wait: in poor families, eating the first marshmallow quickly may be sensible when daily life is uncertain and waiting on a future promise is risky.

The fact that unpredictability and lack of control over circumstances encourage situational responses different from when life is ordered and controllable is evident to researchers who have worked with poor communities. So early community engagement would have quickly alerted researchers to the limitations of the marshmallow test evidence arising from the narrowness of the sample.

Such engagement is especially critical when researchers develop interventions from research studies, because, once we move beyond controlled conditions, interventions that are not alert to people’s diverse circumstances can do significant harm.

For many of us, interventions are ultimately what social research should be aiming at. The recent release of Australia’s inaugural assessment of research impact, known as the Engagement and Impact Assessment, underlines the extent to which the socio-economic benefits of research have risen up the higher-education policy agenda.

In social science, impact often involves informing government policy. But the assumption of a linear path towards impact, in which university researchers do the work and end-users then adopt and apply what is useful, is not only potentially hazardous, it is also, in many cases, a path that does not lead very far.

Linear thinking drives arguments that the key to boosting impact is to improve communication and knowledge dissemination between researchers and potential users. But this won’t work if research findings are not relevant to practical problems. A far better way for academics to produce research that is both high quality and high impact is to partner deeply with government, business and civil society to work on significant practical problems where research evidence can help provide a solution.

This can certainly take university researchers beyond their comfort zones. We value working on the problems we choose, in ways we are familiar with, to produce publications we value and are rewarded for. Deep external partnership belies the images instilled in us during our training of disinterested scholars ploughing their own self-defined research furrows.

Even in team-based disciplines, PhD education aims to produce independent researchers, competent across a field and expert in a narrow area. Working on problems that others deem important requires us to transfer what we know to potentially unfamiliar territory. The American Historical Association’s “five skills framework” explicitly acknowledges the need to build “intellectual self-confidence” in order to collaborate with others with differing worldviews.

As experts, researchers often approach partnerships expecting to talk and be heard. We frame partners’ problems in terms of our own expertise, redefining them to be scientifically amenable or imposing an approach that reflects best practice in a narrow sense but is not commercially, practically or politically implementable. Arguing that randomised trials are the only way to properly evaluate government programmes is one such example.

Researchers who want to partner deeply need to listen to properly understand a problem as partners define it, and to understand their partners' needs and constraints. Taking these considerations seriously enables researchers to tap into real-world expertise – taking what we know, but modifying and extending it to new problems, settings and ways of working.

Stakeholder communities are “lay experts” whose understandings of their experiences, circumstances, constraints and dilemmas complement and enrich social scientific knowledge, and can safeguard against misinterpreting evidence. They offer researchers the opportunity to both advance their disciplines and enhance their societies. And some of them even bring packets of marshmallows to meetings.

Mark Western is director of the Institute for Social Science Research and the University of Queensland. He was a panel member in the Engagement and Impact Assessment.