Outside looking in

It is "a sham", a system that is "too often abused"; it is "obsolete" and serves only as "a fig leaf": critics of the UK's external examiner system have not minced their words.

Last summer, when a series of allegations about declining standards in higher education hit the headlines, the spotlight turned on the singular role of the external examiner - a post that is ubiquitous in British universities but is found in few other countries.

Media reports, and a handful of submissions to a subsequent inquiry by the Innovation, Universities, Science and Skills (IUSS) Committee, featured claims from external examiners that universities ignored their comments when they thought standards were not up to scratch. Others came forward to complain that external examiners had bumped up the marks of their students' work.

Universities, it was argued, wanted their externals to give them an easy time.

With the process widely regarded as a keystone supporting academic standards and quality, these were accusations the sector could not ignore.

The Quality Assurance Agency launched a series of inquiries, including one focused on external examining. Last month, it concluded that higher education in England was "fundamentally sound" and that universities were "diligent" and "thorough" in their engagement with external examiners. But it also called for a set of national "minimum expectations" for external examiners, more discussion about their training and appointment, and more transparency in the process.

Despite a series of reviews on external examining over the past two decades, the QAA said, the system remains much the same as it was in the 1990s, when it was already "under strain".

A frequent lament among critics of the system as it operates today is that external examiners have suffered a loss of power.

"It was once true that the academic judgment of an external examiner was final. This is no longer the case," says Geoffrey Alderman, Michael Gross professor of politics and contemporary history at the University of Buckingham, in his submission to the inquiry.

A separate submission from his university argues for the QAA to be abolished and for the "superior" system of external examiners to be "restored to centrality and power".

Meanwhile Richard Royle, a senior lecturer in law at the University of Central Lancashire, says some universities have a policy of never altering their marks regardless of the view of the external examiner. "In other words, the external examiner is there merely to satisfy the procedural requirements. He may pass comment, but he is impotent and cannot make any difference in practice."

Walter Cairns, a tutor in law at Manchester Metropolitan University, agrees that many universities have circumscribed the duties of external examiners. "In many cases, the external examiner does not monitor the general level of the marks and is not given the opportunity to change individual grades because all he or she is called upon to do is to arbitrate between first and second markers and/or make a decision in borderline cases."

A 2004 report by Harold Silver and Andrew Hannan for the Higher Education Academy (HEA) found that at many universities, the role of external examiners was changing. They were less involved in changing individual student marks, it said, and there was increasing commitment to ensuring "due process" and internal consistency.

"If you go back over the years, external examiners were treated almost like they had God-given powers, and they still are in some institutions. But at most universities, they are now seen more as equals," says Mike Cuthbert, a lecturer in law at the University of Northampton.

Much discussion about students' marks now takes place before exam boards convene, he says. By the time the examiner attends the meeting, he or she can have little opportunity to influence the outcomes.

"The process can be very mechanistic. You do sometimes wonder, as an external examiner: why am I here - other than for my signature?"

The role of the external is not to change individual marks, but rather to take an overall view on standards and assessment procedures, says Margaret Price, professor in learning and assessment and director of ASKe, a centre of excellence in teaching and learning at Oxford Brookes University.

"The idea that external examiners are all-powerful and can change individual marks is dangerous and undermines well-structured moderation processes," she argues.

Chris Rust, from the Student Assessment and Classification Working Group at Oxford Brookes, agrees. "The fairly traditional bad model is where the course team gives up almost all responsibility, and when there is any dispute about what mark something should get they say, 'Oh well, we should ask the external examiner', as though the externals are necessarily any more objective or better placed. I think the external's is more a big-picture, wide-view role," he says.

Because of their autonomy, universities are under no obligation to take action in accordance with the comments or advice they receive from external examiners.

"The extent to which the external examiner may influence the final decision of the board is a matter for institutional policy," the QAA's code of practice says.

But the code, which sketches out good practice rather than sets out requirements, says that "full and serious consideration" should be given by the university to the comments contained in external examiners' reports and the outcomes should be recorded formally.

The QAA's inquiry found a number of cases where external examiners "are reported to feel compromised by the demands placed upon them and/or where they feel that their reports have not been given sufficiently serious consideration".

Critics also charge that the external examiner system is based on networking and cronyism. As universities are free to appoint their external examiners, it is argued, they select those who are sympathetic.

"External examiners are often friends of the module leaders and are frequently asked to scrutinise subject areas with which they are unfamiliar. They are not encouraged to pass adverse comments," Royle claims.

The QAA's inquiry found that universities' processes for identifying and appointing external examiners "appear to lack transparency to observers outside higher education (and some within it)".

During the course of the inquiry, the QAA heard more than once the suggestion that this could potentially undermine the impartiality and integrity of the system in the eyes of those unfamiliar with higher education and its quality-assurance arrangements.

Charles Penn, professor of molecular microbiology at the University of Birmingham, says that validation of the choice of external examiners by university authorities is only notional.

"I would not argue that this necessarily leads to abuses, because the great majority of examiners do an honest and searching assessment of what they find. But it is a system that is potentially open to abuse and so probably deserves some scrutiny."

There are also more practical concerns about the system.

Finding external examiners with the right expertise for the job can be tough.

"When you are a course manager, finding somebody to nominate who is suitable and willing can be difficult," Price says.

This is perhaps not surprising, given the many pressures on academics' time and the poor remuneration for external examining work. Academics without experience of external examining can be keen to add some to their CVs, but beyond this incentive, the system relies largely on goodwill.

"Put it this way - people don't usually go looking to be an external examiner once they've done it a few times," Cuthbert says. "But people do their bit because they hope others will do their bit in return for their university."

In a document on external examining published this week (see box page 35), the National Union of Students says it is concerned about the variation in the rates paid to external examiners by different institutions: it says one university pays £375 a day while another pays £200 a day.

Academics without external examining experience can also find it difficult to get on to the first rung of the ladder.

Official definitions of external examiners and their role place much emphasis on ensuring the comparability of standards.

The Government's 2003 White Paper The Future of Higher Education argues that external examiners carry out "a critical role" in advising universities on how their standards compare with those of other institutions.

Similarly, the QAA's code of practice on external examining says the role provides "one of the principal means for maintaining nationally comparable standards within autonomous higher education institutions".

It is a concept that goes back at least as far as 1832, when the University of Durham called in external examiners from the University of Oxford to approve its degrees.

However, the idea that in today's vast system, external examiners can provide benchmarking against "national" standards has been questioned from all quarters.

Giving evidence to the select committee in March, Peter Williams, the head of the QAA, said that although external examiners were a good thing, claims that they could "provide the kind of nationwide or whole cross-sector guarantee of consistency in standards cannot be sustained".

Paul Ramsden, head of the HEA, took a similar line in his report on the future of the student experience, which was delivered last year to John Denham, who was then Universities Secretary. It says that the system, "despite its successes, requires review and development to produce a truly national system that ... can guarantee comparability of standards of achievement in the future".

A report published by the QAA in 2007 questioned whether external examiners were in a position to warrant the comparability of students' achievements across more than the few institutions with which they were in close touch.

If the sector believes that a system of externals can establish a national standard, "we are totally deluding ourselves", according to Oxford Brookes' Rust.

"The individual external examiner is not equipped to do that. They don't know that many different courses, and they will come with their own agenda from their own institution."

The matter is exacerbated by a tendency in some universities to pick externals from similar institutions - although this is a practice the QAA's code says can be appropriate.

"If you are going to be an external examiner, you have your eye on the status of the place that you are going to work in," says one academic, who chooses to remain anonymous. "Russell Group academics are probably not going to choose externals from an institution that isn't as research-intensive."

In focus groups held by the QAA as part of its inquiry, some interviewees felt that external examining did ensure comparability of standards between institutions; others thought this aim was desirable but not realistic given the expansion and diversification of higher education in recent decades.

In May, Colin Riordan, vice-chancellor of the University of Essex and chair of a Higher Education Funding Council for England quality committee charged with assessing the validity of complaints about external assessment, argued in Times Higher Education that in an age of mass higher education, the system "must show more transparently that it can ensure the comparability of standards".

Roger Brown, professor of higher education policy at Liverpool Hope University, says: "In deciding what to do about the system, the central question is whether we want to persist with the notion of comparability, even in its 'broad' form. If we do, then a far more robust mechanism will be needed, with some degree of independence from institutions."

Despite the limitations of the role, many argue that there are positive aspects to it. A common view is that although the role of externals can be fairly marginal - they might visit the university once a year, write a short report and get paid very little - the sector is decidedly better off with them than without.

Stephen Bostock, head of the Learning Development Unit at Keele University, says: "In most cases, the value is in the constructive feedback and the opportunity to share practice. Having someone come in, see what you do and suggest things that perhaps you could do better is a good discipline.

"I hope the sector is witnessing a shift from quality assurance to quality enhancement," Bostock says, "and enlightened institutions are looking to their externals to make constructive comments about how things might be improved."

The role of external examiner is frequently described as being that of the "critical friend".

The QAA's code of practice says that external examiners, in addition to looking at standards, should help universities to ensure that their assessment processes are sound, fairly operated, and in line with the university's policies and regulations.

"I think there has been progressively a move to seeing externals more as auditors of systems who make sure that the systems that are in place are good and are working. I think that is good and right. You also learn a lot yourself from going in and looking at other courses," Rust says.

Brown, however, argues that it is impossible for externals to fulfil the "fairness" function because of the scale of poor assessment practice in universities. He believes the entire system should be replaced.

But Jon Renyard, chair of the executive committee of the Quality Strategy Network (QSN) and director of academic services at the Arts Institute at Bournemouth, says he is not aware of any strong evidence that the current system does not work effectively.

"What we are aware of is occasional reports. But there aren't that many if you take into account the fact that there must be some 25,000 external examiner posts. Perhaps what the sector hasn't done quickly enough is deal with things where there is prima facie evidence that something might have gone wrong," Renyard says.

In his view, there is often a misconception on the part of those outside higher education that when it comes to standards in universities, external examiners are the sole defenders.

"I think the external examiner system is absolutely critical - but it is not a stand-alone system, and it is part of the overall quality assurance framework. It couldn't possibly do the job of assuring standards and quality on its own," he says.

The QAA puts out guidance on maintaining quality and standards and issues subject benchmark statements, which set out expectations about standards of degrees in a range of subject areas. As well as being reviewed externally by the QAA, universities conduct regular internal reviews and student surveys.

"Some of the criticism certainly does imply that the external examiner, this one person, is the be-all and end-all. I don't see how someone can be," Renyard says.

"When academics are approved as external examiners, they don't suddenly become imbued with mythical qualities that turn them into super-examiner. But they are extremely important, and we do take their views extremely seriously."

Despite Renyard's reassurances, however, many are pressing for changes - among them, ASKe.

In its submission to the select committee, ASKe argues that establishing national standards in a discipline requires academics from different universities to meet and compare the quality of their students' work and their marking judgments.

It is an idea that has won Ramsden's support. In his report to Denham, he suggests the development of "colleges of peers" in different subject areas, possibly co-ordinated through the HEA's subject centres, with regular meetings to read students' work and arrive at a common understanding of standards at various levels.

As a form of peer review, this would sit well with the academic culture, he believes.

"ASKe would argue that if you want to get national standards, you have to do that nationally," Rust agrees.

The idea of disciplines having "a better conversation" about standards is a sensible one, Renyard says.

"It moved a bit towards that with subject review. Nobody wants to bring subject review back, but within institutions and across disciplines there is value to having conversations about marking standards."

Brown, however, worries about the accountability issues that might arise from putting everything in the hands of "colleges of peers".

He favours a new system under which groups of external academics would examine all aspects of a university's curriculum.

"What I think should happen is that there ought to be a proper system of curriculum review, whereby institutions themselves organise it so that every few years - maybe every seven to ten - a group of academics from outside the institution look at every aspect of the curriculum. "They would consider not just assessment, but admission of students, allocation of resources and how fresh the curriculum is, and they would report to the vice-chancellor. That could sweep up the key functions of external examiners," Brown argues.

The 2003 White Paper called for the external examining system to be "strengthened by improved training and induction", including a national development programme for external examiners by 2004-05.

No national programme has been developed, although many universities run induction schemes for external examiners to introduce them to their procedures and courses.

According to Bostock, however, such inductions can neglect basic education in good practice in assessment. One concern is that although there is a huge literature on assessment, few externals are familiar with it.

A new course for external examiners at Keele University, co-ordinated by Bostock, is the first to be accredited by the Staff and Educational Development Association. It aims to help externals develop professionally in the task of checking whether assessment practices are rigorous and fair, and in delivering cogent and constructive feedback to universities.

In his report to Denham, Ramsden proposes making the training of external examiners a condition of their appointment. This week, the National Union of Students calls for a mandatory programme of development and training.

Ramsden thinks the HEA's professional standards framework for teaching and learning could provide a model for the development of a similar framework for external examiners.

He says: "The last thing we want is a really heavy training process for external examiners, but we could help to improve the process by offering relatively short and sharp workshops that would enable people to do their job more effectively."

Another idea that has been discussed is that of a register.

In 1997, the Dearing report proposed creating a UK-wide pool of academic staff, recognised by the QAA, from which universities would select external examiners. The idea was rejected, although many HEA subject centres do now provide informal registers of academics who are available.

Ramsden resurrected the idea of a national register in his report to Denham. But there was little enthusiasm for the concept among those the QAA interviewed in its inquiry.

"It was felt that this would not in itself increase the pool of suitably qualified individuals and would change an independent peer-review process to one that was inspectorial and place further demands on institutions," the QAA report says.

In its submission to the IUSS inquiry, the QSN claims that a register would create a significant bureaucratic burden. It also thinks that a single central body could not keep the database properly updated. But its biggest worry is that a register would mean the end of universities' freedom to choose their own externals.

"Institutions take great care in selecting examiners with the specific knowledge necessary for their own programmes, which assures subject alignment ... It is hard to see how a central register would be able to match the efficacy of this process," the QSN argues.

"We are aware of no evidence that institutions select external examiners who are insufficiently critical or who are too close to the institution or course team; the need for objectivity is explicit in the QAA code of practice, and institutional criteria for appointment will make clear that this professional relationship should be supportive, but requires a degree of distance," the QSN continues. "It should also be noted that many examiners have indicated that they would not wish to join any centrally held register ... They have reservations about inclusion as part of a national database."

The NUS has proposed that external examiners' reports be published, but this idea is set to stir up controversy.

Following a recommendation in the Government's 2003 White Paper, summaries of reports were published on the Teaching Quality Information website.

But when the site was relaunched as Unistats, the summaries were dropped because of concerns that the versions online were not being read and were too bland.

The NUS says: "Students have a right to understand how and why decisions on standards across the sector are made, and information about who (external examiners) are and what they should do should be more widely available."

As Times Higher Education reports this week, others are not convinced. The QAA recently investigated a case at Kingston University in 2003-04 in which an external was pressured into changing a report partly because, at the time, the university published all assessments online.

Students and the general public, the QAA found in its inquiry, may not have a good understanding of the role of external examiners, but they value them as a perceived source of impartiality - and while that is the case, ministers will remain committed to the system.

In the words of the 2003 White Paper: "In many respects, (external examiners) act as guardians of the public purse and of the reputation of UK higher education."

What seems likely is that external examiners are here to stay, although discussion on how best to reform the system will go on.

Mike Cuthbert concludes with a question that, he admits, is hypothetical. "If we didn't already have external examiners, would we now invent them? I don't think we would."

EXAMINER'S NOTES: AN ANONYMOUS ACADEMIC ASSESSES THE SYSTEM

We all moan that work as an external examiner is time-consuming and underpaid, which it is. Nevertheless, we are pleased to be asked to do the job, and I think most academics consider it essential.

It's one area of academic life that is under the control of academic staff, for it is usually departments or faculty boards that appoint externals. Most academics at some point in their careers act as externals in their disciplines, but they also have externals appointed to their own departments. These two-way contacts are important for maintaining common standards, facilitating exchanges of experience and expertise, and keeping academics within disciplines in touch with each other.

No system designed to underwrite comparability can be perfect, but external examiners provide a straightforward and non-bureaucratic way of doing things. Externals see samples of work and make only a few visits to departments during their years as examiners, but they gain insights into the quality of teaching, the viability of the course's aims and the way in which students respond and progress.

Departments and course teams do take notice of externals' comments and suggestions. Whether they are able to act on them, however, often depends on the university and its administrators - a recommendation for a library to purchase more books for a particular module may be enthusiastically backed by the course board but not by the managers.

I think it's a good system. It reinforces the view that disciplines are seamless in that they share concepts, methods and standards and that there is a common clerisy. It's good for the externals as they learn a lot from the way other universities do things.

There are, however, problems. The system is based on the supposition that a degree from one UK university is equal to a degree from any other and that universities share identical standards. External examining underwrites this notion. Whether this was ever absolutely true is debatable, but it was once almost true. Few would, I think, give an unqualified affirmative if asked whether it was true today. The picture is made more complex by differences in course aims, the great variety of interdisciplinary degrees and the number of specialist modules within even single disciplinary degree courses.

I believe that the quality of the teaching in my discipline is broadly comparable across universities and is generally high. But although you find very able students in every university, the quality of the work done can vary considerably. The enormous expansion of higher education, together with the poor quality of the education provided by many schools, has resulted in large numbers of students ill-equipped for university courses, some with a very poor command of written English. We don't assess universities on the basis of "value added" but rather - theoretically at least - on the basis of absolute standards reached. Inevitably, however, relativism and grade inflation can creep in.

If there's a danger that an external may be too easily satisfied, there's also the danger that he may demand higher standards than he would find in his own department and in his own teaching. The virtue of the system is that externals find themselves at different times at both ends of the process.

A SEE-THROUGH SYSTEM: HOW THE NUS WOULD REFORM EXTERNAL EXAMINING

The National Union of Students says external examiners are vital. It believes, however, that the system needs to be further reformed and enhanced. In its briefing to students' unions, it offers four recommendations.

- A national convention and network

The NUS says that although external examiners are expected to ensure comparability of standards, "there is no system to ensure that all examiners interpret standards in the same way". It believes that external examiners should meet at an annual national convention on standards to share good practice and ensure parity. "A virtual network should also be created to allow discussion among external examiners in the absence of large-scale physical meetings. This should be linked to the Higher Education Academy subject centres."

Such moves would also raise the profile of the external examining system and add to the prestige of being an external examiner, which should encourage more involvement in the process, the NUS argues.

- Professionalising the system

The NUS says there is "little evidence" of any nationwide professional development of external examiners, as was recommended by the Dearing report. It says that far greater resources are needed "to allow the Quality Assurance Agency and the HEA to create a holistic programme of development and training for external examiners, which should be mandatory" for all external examiners.

The external examiner role should be recognised within the career path of senior academics to ensure that it is seen as more than just "a time-consuming add-on".

- Improving public information

The outcomes of external examiners' work should be published openly, the NUS believes.

This used to happen with the Teaching Quality Information website, but publication was dropped in the move to Unistats in 2006. The NUS says: "Students have a right to understand how and why decisions on standards across the sector are made, and information about who (external examiners) are and what they do should be more widely available." It says it accepts that such information "should be targeted primarily at informed students such as course and faculty representatives as well as students' union officers, although some students may also be interested in these summaries".

The NUS thinks institutions should be required to publish their reports (while respecting confidentiality issues) and that the HEA and the QAA should publish more detailed national outcomes documents.

Additionally, institutions "should feed back to the external examiner how they have acted on the report". Externals who feel their reports "are not being considered seriously" by the institutions should be able to invoke the QAA's "causes for concern" process. Institutions should provide a space online for students to discuss the national outcomes, as well as the internal reviews; "this would go a long way to ensuring transparency of internal quality assurance and enhancement."

- Ensuring transparency and impartiality

External examiners are currently paid a nominal stipend by institutions. The NUS believes that they should be paid in a similar way to the QAA's auditors; ie, that payment should come from outside the higher education institution "to ensure that examiners are impartial and seen to have sufficiently removed interests from the institution". The increased payment would make the role more attractive. The NUS says this would boost public confidence in the system and in the standards of UK higher education.

EXTERNAL AFFAIRS: DENMARK'S EXAMINER SYSTEM

Denmark is one of the few other countries in Europe with a system of external examiners.

One third of all assessments and exams taken by students in the country are marked by an external as well as by the students' lecturer.

Examiners are not chosen by universities; they are appointed via national boards after consultation with the institution. There are 104 boards for different disciplines, and about 10,000 external examiners in the country.

Most externals are academics, but one third come from business or other roles outside higher education. They are, however, expected to have high academic qualifications and must keep up to date with their subject.

"Students see external examiners as an important way of ensuring the fairness of examinations," says Tine Holm, director of projects in higher education at the Danish Evaluation Institute.

However, Denmark, like the UK, is in the midst of a discussion about its external examiner system.

"A weakness that has been mentioned is the cost of the system. It is estimated that 3 per cent of the university budget from government goes to external examining," Holm says.

One proposal being mooted is more use of sampling of examination papers.