Watching the watchers

I read with dismay the comments made by the Arts and Humanities Research Council spokesman about my complaints regarding its peer-review standards ("Pianist wants comeback as review hits bum notes", 10 May). He apparently said that its guidelines make it clear that "impact had to be measurable in the sense of being open to evaluation".

Since the beginning of my dispute with the AHRC in October 2011, I have repeatedly told it there is no statement in any of its published documents to the effect that the impact and potential pathways to impact highlighted in a research proposal have to be "measurable" - ie, quantified and assigned a numerical value. Its documents only use the term "demonstrable" in relation to impact.

The effects of a project can be "demonstrated" through non-quantitative means (eg, video documentation, written feedback from the public, etc), which are not - in the academic jargon - ways of "measuring" phenomena. There is nothing in the guidelines indicating that impact should be "decisive, predicted, and quantifiable (measurable)" - the precise words used in the peer-review reports for my proposal. It is alarming that the AHRC spokesman seems unaware of the vital distinction between quantitative and qualitative research methodologies.

In my complaints to the AHRC, I have challenged its peer reviewers to show (with evidence) that organising interactive workshops/musical performances open to the public in prestigious London venues, the core of my proposed project, does not make demonstrable contributions to society, the economy and cultural life, and that it would have "limited impact outside of academia", as claimed in the peer-review reports and the broad feedback I received from the panel. No one from the AHRC so far has provided any such evidence.

For colleagues and members of the public interested in impact and peer-review standards, I have made the relevant documents regarding my complaint available on my web page at academia.edu. It is time to have an open debate about research funding bodies' peer-review standards.

Mine Dogantan-Dack, Research fellow, music, Middlesex University