When is a percentage not a percentage? The problem with HE marking practices

I’ll never forget the time, back in nineteen-eighty-mumble-mumble when a member of my tutorial group pulled out her freshly marked essay. “Listen to this: 58. Tick. Solid,” she read. “That’s all it says. What on earth am I meant to make of that?” I was at a loss to know how to respond. I hasten to add that I wasn’t the marker.

That incident is fact, not fiction, though it’s possible I’ve misremembered the “on earth” bit. I’m pleased to say that UK higher education has come a long way since then with regards to giving feedback. But the “58” bit remains largely intact as a central part of assessment practices. And that’s problematic.

• Can asking for students’ perception of assessment improve fairness?
• The House Cup: promoting qualitative grading in mathematics
• A unique approach to designing robust, calculation-based online assessments

The 0-100 scale is beset with problems when it comes to marking student work. Elsewhere we have documented its inherent unfairness: the ways in which it punishes students disproportionately at the bottom end and boosts those at the top, thereby advantaging already high-performing students and feeding grade inflation. Here, I want to deal with the systematic misunderstandings that I come across regularly within the sector.

Holistic and additive marking

Let’s start by distinguishing between the two different forms of summative assessment that go on in universities. The first we might call “holistic”. This is where an assessor takes an overview of a piece of work and awards an overall mark, often expressed in the form of a percentage. Essays, reports, dissertations, artwork and so forth might be evaluated in this way.

The other form is where components of an assessment task are summed to get an overall score. We can call this “additive” marking. Short answer and multiple-choice question (MCQ) exams are examples. They produce a “score out of something” mark, which can also be expressed as a percentage.

Only the latter form of assessment produces a real percentage. The former has co-opted the term – inappropriately. This wouldn’t be so much of a problem if the only bad thing about it was the lazy use of terminology. The issue is that in adopting the “percentage” nomenclature for holistic marking, the sector (long ago) caused misunderstandings about assessment processes that endure to this day.

Labels or amounts?

The number given at the end of a piece of work that has been marked holistically is a label that denotes category membership. It is not an actual percentage. The fact this is not widely recognised is of concern. I remember the answer I was given once when I was sitting on a validation panel, and I asked why the pass mark for an essay had been set at 50 “per cent”. The reply shook me: “Well, I suppose it shows that the student has got at least half of the content correct.” I haven’t got room in this piece to explore the layers of nonsense embedded in that response.

The point is that the standard causal narrative associated with marking is expressed backwards, in a way that turns out to have damaging effects. You are not awarded a first for a single piece of work because it has been measured and found to be “70-plus per cent” of something. You are awarded 70-plus per cent because the marker has made a criterion-based categorical judgement that your work is “a first”. Likewise, you don’t fail because you only got 38 per cent. You are awarded 38 per cent because a judgement has been made that you have failed.

One consequence of this way of looking at things is that proposals to tackle artefactual grade inflation by increasing the percentage requirement for a first to, say, 80, will of course not work. All that will happen is that pieces of work that used to get labelled with a 70-plus will now simply be labelled with an 80-plus. It would, admittedly, be a fraction harder to average 80-plus per cent across a degree course, but that’s a different matter.

Raw scores

There is nothing inherently “pass-worthy” about 40 per cent. Neither is there anything “excellent” about 70 per cent. These are arbitrary conventions that have become reified and, for many reasons, that is not a good thing. The problem I want to focus on here is the consequences this has for how the other sort of assessment (the “additive” type, such as MCQ tests) are sometimes used.

It is not uncommon to see the raw scores from these kinds of assessments (expressed as a percentage) put directly into students’ academic profiles alongside the marks from work that has been assessed holistically, as if they are unproblematically the same type of thing. This is like adding apples and pints of cider. Worse still, it’s like adding apples and pints of cider without even noticing they are different things. I’ve lost count of the number of people who have looked blankly at me when I have asked them how they will decide what raw score will be enough for a student to pass a given MCQ test, as if the answer is a given: “Well, 40 per cent of course…”

In order for the genuine percentages of the additive type of assessment to sit meaningfully alongside the holistically derived “percentages”, the raw scores must first be mapped onto the categories that underpin the holistic marking. Other than placing students in a rank order, what do raw scores on an MCQ test tell us about levels of achievement? If it was an easy test for this level of study, a raw score of 70 per cent could plausibly be the pass mark. If it was difficult, perhaps 70 denotes exceptional (top end of first-class) performance.

Academic judgement

Mapping raw scores onto categories can be done by pre- or post-test norming, or by a holistic academic judgement (no different in principle from the judgements that are routinely made about essays). The bottom line is that unprocessed raw scores should never be added directly into a student’s marks profile as the outcome of an assessment. The apples need to be processed if we are to add them into our “pints of cider” equation.

So, one answer to the question of when a percentage is not a percentage is: when it’s written at the bottom of an essay. A grade-based assessment system recognises that marking essays and so forth is a process of sorting them into discrete categories rather than measuring them on a continuous scale. Grade-based approaches to marking help correct some of the misunderstanding (and unfairness) that besets assessment practices structured around the 0-100 scale in the HE sector.

Andy Grayson is associate professor in psychology at Nottingham Trent University. He has worked in higher education for more than 30 years and provides leadership on learning, teaching and assessment.

If you found this interesting and want advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the THE Campus newsletter.