Examples of little relevance

With the publication of the final report on the research excellence framework's impact pilot exercise, we now know a bit more about how impact will be assessed in the REF. But it is particularly instructive to examine the document on the findings of the expert panels, which includes case studies from English language and literature (one of five subject areas in the pilot study). Regardless of one's opinion about the impact agenda, the case studies seem to indicate that what is approved as "impact" bears little relation to what is produced in most English departments.

In all, 13 English departments provided 44 case studies. Of these, only five have been published and we must take these, presumably, as examples of good practice. So we now have examples of how to make a case for impact within a humanities discipline.

The final report considers two criteria for assessing impact: "reach" and "significance". Hitherto we were warned that evidence of dissemination was not evidence of impact, implying that "dissemination" and "reach" were distinct. However, much of the material in the case studies is actually evidence of dissemination. The report notes that when impact was claimed "through engaging the public with research", evidence "was provided about dissemination as well as a clear explanation of the significance or benefits to the audiences".

The topics of the five published studies are: topography, ecology and culture; Henry VIII at Hampton Court Palace; the impact of literacy research on informing policymaking and improving public services; public understanding of poetry; and creating educational and commercial access to English language resources.

It is immediately apparent that some of the examples represent areas of the discipline in which making the impact case is pretty straightforward. Two of the five come from specialist research units rather than English departments. The Literacy Research Centre at Lancaster is part of that university's Institute for Advanced Studies and was funded as part of the major government initiative on Skills for Life, including adult literacy. Hardly anything about its practice has transferable lessons for other English departments.

Likewise, the Survey of English Usage at University College London is a specialised advanced research institute, not an English department. "Henry VIII at Hampton Court Palace" is surely a history topic, and two of the three books for which impact is claimed are obviously history books. The benefits claimed relate to increasing income and visitor numbers to Hampton Court.

The case study on "topography, ecology and culture" is thought-provoking and influential. But of the researcher's writings, the ones on 19th- and 20th-century literature are not mentioned, and it is his role as a travel writer and environmentalist for which impact is claimed.

It is as if we are being told to give up writing about topics that most of us in English departments concentrate on most of the time.

So, only the public understanding of poetry case study offers material that is typical of what is usually researched in English departments. The work covers both scholarly and contextual work on the poetry of the past, and innovative publishing, practice and performance work in poetry today. One of the researchers is also a poet, so creative writing is covered, too. All the same, two of her three outputs for which impact is claimed are critical rather than creative, offering little help for staff in English departments whose output is entirely in creative writing.

Judy Simons, chair of the REF pilot expert panel in English language and literature, recognised in this magazine ("REF pilot: humanities impact is evident and can be measured", 11 November) that the humanities are "the poor relation of the research budget" and urged us to avoid what she called "special pleading" and instead "protect what we have, even if it's only crumbs from the rich man's table".

Sadly, it is unclear how we are to do this, as the published impact case studies largely seem to ignore the main core activities of the literature side of the discipline of English language and literature.