Books editor’s blog: A vote of confidence in the scientific method

Politicians write books on many subjects, in attempts to reveal their competence, their wisdom or their human side. They rarely channel their inner nerd. But that is exactly what Andrew Leigh – a former professor of economics who is now an Australian MP and the Labor Party’s shadow assistant treasurer – has done in Randomistas: How Radical Researchers Are Changing Our World (Yale University Press).

Although the subject seems hardly likely to get voters’ pulses racing, the book is devoted to the joys of randomised trials. Indeed, it acquired its title by that very process, after 12 different versions of a similar advert were offered to web surfers (an experiment that took about an hour to set up and cost A$55). Fortunately, Leigh proves adept at bringing this seemingly dry topic to life, making the case for “the experimenting society” and citing a number of intriguing examples in areas ranging from bloodletting to philanthropy.

Take the case of the charity Freedom from Hunger and the letters it sent out asking people to donate money. All of them included the story of a poor Peruvian widow called Rita, yet half the recipients were simply told that the charity knew that “women like Rita are ready to end hunger in their own families and in their communities”, while the others were informed that Freedom from Hunger “look[ed] for more than anecdotal evidence” and had therefore “coordinated with independent researchers to conduct scientifically rigorous impact studies of our programs”.

In doing this, the charity was, as Leigh puts it, “running a randomised trial to test whether donors cared that a program was backed by randomised trials”. The results were very striking: “including information on impact raised donation rates among larger donors, while decreasing generosity among small donors”. The genuine altruists, the researchers concluded, cared about results, whereas “among those simply looking for a warm glow, mentioning evaluation raised the spectre that not all aid might be effective”.

Other chapters describe relatively simple tests that have been used to assess the value of different teaching methods, initiatives to cut crime and programmes to improve the lives of the poorest. “The best non-government organisations”, notes Leigh, “are always looking for ways to put their programs to the test, and so improve what they do. They know that a lousy outcome for a program is ultimately a great result for the community, because it means we can stop spending money in ways that don’t work.”

There are also lessons directly applicable to politics. Leigh has met plenty of campaign veterans with firm views about the relative value of communicating with voters via letters, door-knocking, phone calls and posters in electoral campaigns, but “ask them about their evidence base and it’s quickly apparent that their war stories lack a control group”. Randomised trials revealed that “knocking on a person’s door is nearly three times as effective as calling them on the telephone, and probably 700 times more effective than sending them a letter”, making it distinctly “puzzling as to why campaigns spend only about one-twentieth of their budgets on personal campaigning”.

It is very cheering that a politician should decide to write such a hymn to the scientific method. We can only hope that it will also appeal to his electorate.