Calls to decolonise assessment do students a disservice

As the decolonisation agenda gathers momentum, assessment has inevitably become a target.

We can see this, for example, in the University of Leicester’s recent report Tackling Racial Inequalities in Assessment in Higher Education. The report identifies undeniable differential outcomes for non-white students. But it also makes the controversial claim that the assessment system is to blame, and so needs to be radically altered to deliver equal outcomes.

Two beliefs underpin this argument. The first is that assessment in higher education devalues diversity of experience and knowledge. The second is that assessment practices are “part of colonial systems which contribute to the marginalisation and privilege of different students”.

“Knowledge” is thus treated as a culturally constructed human variable: a somewhat outlandish position for an institution that claims to “pursue knowledge that has the power to transform”. But this belief in constructed knowledge is at the heart of standpoint epistemology, which gives a central place to perceived group identities and beliefs. I have had conversations in which I begin: “Assessment research into bias shows that…”, to which the response is: “Was the researcher white?” The bottom of this rabbit hole is a very dark and scary place.

Furthermore, as I explore in detail in my 2015 book Re-examining Language Testing: A Philosophical and Social Inquiry, modern assessment practices are not the creation of the British empire. They originated in the Sui Dynasty (AD581-618) and were designed to eliminate nepotism in the Chinese civil service. Japan and Korea also adopted such practices, and from there the Jesuits brought the competitive examination to Europe.

In the UK, the Victorian social reformer Edwin Chadwick was the first to see the value of examinations for removing aristocratic patronage in civil service appointments, thereby creating opportunities for the nascent middle classes and the indigenous inhabitants of the colonies. It was adopting the “Chinese principle” of equal opportunity at the point of assessment that laid the foundations for modern notions of inclusion and widening participation.

But debunking the ahistorical assumptions of decolonisation theory does not address the real problem of differential outcomes. To do this, we must understand validity in assessment. An assessment is valid if theory and empirical evidence show that outcomes can be used to make sound inferences about knowledge, skills or abilities (KSAs) that are relevant to subsequent decision-making. Bias occurs when an identifiable subgroup of students is awarded scores contaminated by factors that are unrelated to target KSAs.

It is then essential to provide assessment accommodations, such as variable font sizes or Braille for the partially sighted. However, it follows logically that a differential outcome is a necessary but not sufficient condition for the existence of bias. We still need to conduct research to show that a significant proportion of systematic score variation can be traced to an irrelevant factor.

A textbook example of this comes from the US, where bias is frequently tested in the courts under the 14th Amendment (equal protection under the law) and the 1964 Civil Rights Act. The case of Debra P. v Turlington (1984) pointed to differential outcomes observed in the Florida student test for functional literacy. But the ruling laid down that sufficient evidence had been provided to support the soundness of intended inferences from scores, and that scores were indeed useful in decisions to award school-leaving certificates. Ensuring equal pass rates by a protected characteristic would therefore make the assessment (and diploma) worthless.

Fairness in assessment should be about preventing those without critical KSAs gaining qualifications, practising professions or performing tasks that may harm others, while also trying to ensure that individuals are not put at a disadvantage in their careers or aspirations because of bias. The Florida test was not unfair – it revealed unfairness. It told policymakers a great deal about unequal learning opportunities and social disadvantages for a subgroup of the student population, thus making it possible to plan and implement interventions.

The approach adopted by advocates of decolonisation, as the Leicester report puts it, rejects “the harmful and counterproductive ‘deficit model’, which attributes any lack of academic attainment to issues associated with the student”. This leads to manipulating assessment practices to generate equal outcomes. It thereby subverts not only validity theory but the very role of assessment as an engine of fairness in a meritocratic society.

It is also exceptionally dangerous for the very students that decolonisation purports to serve. It can lead to paternalistic attempts to shield them from the value systems, KSAs and language that would provide access to the equality of opportunity they desire and deserve. It also takes no account of research that tells us learners need extensive induction into discipline-specific discourse communities, such as law or medicine.

Making the transition can indeed be particularly difficult for students who don’t speak English as their primary language, but acquiring such literacies is essential for success in content-based assessment. Professional and academic disciplines will not change their discourse practices or knowledge base to accommodate the uninitiated.

While ideologically motivated talk of “decolonising assessment” may resonate with some in our learning community, it ultimately does a profound disservice to the students that institutions of higher education should rightly wish to see succeed.

Glenn Fulcher is professor of education and language assessment at the University of Leicester. His 2015 book Re-examining Language Testing: A Philosophical and Social Inquiry won the SAGE/ILTA award for its contribution to assessment theory.