Do you think this is working?

On a December day last year, Lauren Roffey breathed an enormous sigh of relief. She had passed her PhD oral examination, or viva voce, the experience she had dreaded for months rushing by in a 90-minute blur.

She was anxious about the event despite having been, she believes, “luckier than most”: Roffey benefited from extensive advice from a supervisor, three mock vivas, and she had a hand in picking an examiner suitable for her interdisciplinary thesis.

“In the build-up to the viva you hear such horror stories and get such conflicting advice from friends and colleagues,” she explains. “You go into it feeling ‘I’ve done all I can’, but given that you’ve worked for the best part of four years to finish your thesis, it is daunting that it’s such a vague examination you’re about to have.”

Her account highlights a problem faced by doctoral students across the country: too often, relatively little is known about what to expect in the final, essential hurdle in gaining a PhD, and the process can feel fraught.

In many countries, the viva is a formality, but in the UK it is designed to explore how the candidate’s research makes a contribution to knowledge, to examine an individual’s ability to defend and clarify the thesis (helping to ensure that the work is the candidate’s own) and, ultimately, to ensure that the student is worthy of a doctorate.

Generally two examiners, one internal and one external, make independent assessments of the thesis. They usually compare these before they conduct the oral exam, which sometimes takes place in the presence of a supervisor or independent chair.

After the viva, which can last anything from 90 minutes to a gruelling five hours, outcomes range from outright pass to outright fail. In between these two extremes, examiners can request that the candidate submit corrections, resubmit the thesis entirely or be awarded a lower degree.

The PhD oral examination is overseen by the student’s own institution and there is no common UK standard, says Louise Morley, professor of education at the Centre for Higher Education and Equity Research at the University of Sussex. The process may vary depending on both the institution and the individual examiners involved.

“The viva seems to have escaped some of the quality assurance mechanisms around assessment,” she says.

One candidate left the viva ‘feeling as if I had been hit on the head’ and burst into tears with a sense that her years of research had been a waste

The viva experience is often said to go on behind closed doors. Although some institutions now employ independent chairs and create a written record of events, traditionally this has not been the case. The lack of transparency can lead to fruitless appeals in which “one side says ‘you said this’, the other says ‘no I didn’t’”, Morley says.

According to Barbara Crossouard, senior lecturer in education at Sussex, another potential problem is the sometimes “macho” and “muscular” nature of the test itself.

The process is designed to challenge and engage students, but examiners’ language and behaviour could contribute towards making the experience both stressful and emotional, Crossouard discovered from interviews with PhD candidates in a range of disciplines.

Such a process can disadvantage less confident students, as well as those from outside the UK, who may not have as much experience of such an environment, says Morley.

Indeed, the worst situations described in Crossouard’s research document “bullying” behaviour from examiners. One candidate, who in fact passed with minor corrections, reported leaving the viva “feeling as if I had been hit on the head” and bursting into tears with a sense that her years of research had been a waste of time.

Writing in Times Higher Education last year, Chris Hackley, professor of marketing at Royal Holloway, University of London, highlighted the emotional burden that the current system places on individuals. “The unbridled power of the external examiner to dismiss years of work as irrelevant, misguided or simply wrong seems highly anachronistic in an era of sky-high student fees and much-trumpeted student charters,” he argued. “Do we care enough about our students to protect them from the rogue examiner?” he asked.

Apart from the possibility of tensions between the student and examiners, says Ron Barnett, emeritus professor of higher education at the Institute of Education, University of London, other matters can get in the way.

“Often there are issues around power relationships in a viva…and sometimes the relationship might be a bit awkward between the two examiners,” he says. In the worst cases, the viva can become an opportunity for academics of often different experience and rank to show off and parade their own opinions, he warns.

Nor are the standards on which to judge a PhD viva clearly set out, he says. “Regulations typically talk about producing work of publishable standard, adding to knowledge or understanding of the world, but that’s more or less it. Examiners are usually left to judge the matter for themselves, and I think some will set the bar much higher than others.”

So how widespread is the problem of the “bad viva”? Barnett estimates that issues relating to “quirky, maverick examiners” will arise in about 5 per cent of cases.

Others, such as Vernon Trafford, emeritus professor of education at Anglia Ruskin University, believe the scale of the problem is likely to be smaller. “The vast majority I’ve attended have been fair and they have fulfilled the purpose of the viva,” he says.

He can count just four instances, out of the 115 he has attended, where the viva failed to do its job. One was the fault of the student, who had managed to submit two different versions of his thesis, while two involved inappropriate, “off the wall” examiners. In the fourth, the examiner had obviously not read the thesis.

Despite occasional problems, Trafford and Morley point out that when the process is conducted well, the experience can be very valuable to the candidate. Examiners often go out of their way to guide them through the process; they might also apply their experience in suggesting conference or journal outlets for the candidate’s work. “Looked at that way, vivas are a helpful and developmental process,” says Trafford.

Equally the viva can be a chance to “save” a student who has produced a weak thesis, says Peter Hartley, professor of education development at the University of Bradford.

“As an examiner, I was involved with one viva where we had to fail the student. The viva was to see if we could get something out of it, but we couldn’t. It still sends a shiver down my spine when I talk about it,” he says.

Outright failures are rare. Based on his experience and that of colleagues, Trafford estimates that about 70 per cent of candidates pass with minor corrections. Some 10 per cent have to make major corrections, while similar proportions fail outright or pass with no alterations.

When things do go wrong, the first port of call is the university’s appeals process, the nature of which can also differ between institutions because they may grant their appeals panels varying levels of independence and power, says Barnett.

The internal nature of appeals means there is little national data on the frequency of complaints about the viva process. Candidates can only take their case to the Office of the Independent Adjudicator, the body that reviews student complaints, once this internal route is exhausted and only if a concern relates to procedure, rather than academic judgement.

Data from the OIA show that PhD students, like other postgraduates, are over-represented in the complaints it receives. They made up just 3.7 per cent of students in England and Wales in 2011-12 but accounted for 6.5 per cent of the OIA’s 1,605 complaints in 2011.

Few would wish to repeat the process, but rigour is something we demand for what is effectively a ticket to practise in higher education

But although the body does not keep records on complaints referring specifically to vivas, a spokeswoman said it would be “reasonable to say that the number that relate purely to vivas will be very small”, falling within just 70 complaints during 2011 made in total in the broad category of “academic status”.

Nonetheless, the process of the viva is beginning to receive attention from sector-wide bodies. Janet Bohrer, assistant director at the Quality Assurance Agency, says the organisation, whose remit it is to safeguard standards and quality in UK higher education, has plans to convene a group to look at the research degree examination process next year.

Although concerns such as Barnett’s are generating discussion, Bohrer says she is yet to see any “groundswell” of opinion that would point strongly to problems in the area. The job of the group convened by the QAA will be to look at whether guidance from the body, possibly in collaboration with the Higher Education Academy, would be useful, she says.

So what could be done to improve the process? More training and guidance for examiners is one way in which the viva could be refined, says Barnett. “By and large [a viva examiner] is one of the few professional roles in higher education for which one appears to not need any kind of formal training or development.”

The use of independent chairs, who attend the viva, are also thought to help minimise, although not mitigate, problems. However, says Morley, this adds to the already high labour costs of the viva process, which is a growing problem given the rise in the number of postgraduate research qualifications awarded, up 24 per cent between 2007-08 and 2011-12, according data from the Higher Education Statistics Agency.

Some suggest getting rid of vivas and assessing theses on the text alone, she says. At US institutions, the role of external examiners in vivas is usually smaller, whereas in Australia there is usually no viva at all.

A further option, which could potentially help to put the candidate at ease and reduce the impact of the power-playing examiner, would be to employ the open and public defence of the thesis in front of family and friends, as occurs in most of Europe.

But as Trafford points out, this “defence” is rarely any more than a “public event”, coming as it does once the text has already been reviewed and corrections received. “It is unlikely that the candidate could fail - the only way of doing so would be to get up and hit one of the examiners. They can’t fail because they’ve already had the viva as we know it, just purely on paper,” he says.

Stan Paliwoda, the recently retired director of the postgraduate research programme at the marketing department of the University of Strathclyde, believes that the rigour of the UK viva and its one-to-one examination process is something to be preserved.

He fears that the PhD could become devalued without such a challenging process. “To boil an egg you can set the timer for four minutes and it will usually do, but you can’t do the same with a PhD,” he says.

Paliwoda sees the external examiner’s role as a counterbalance to departmental pressures to increase the number of PhD graduates, which often stem from assessments such as the research excellence framework. “Universities have an interest to maximise their PhD graduation numbers, particularly at this time. So a concern should be voiced here that says ‘back off and leave the external examiners to do their job’,” he says. “I’ve met few people who would wish to go through the process a second time, but rigour is something we demand [for] what is effectively a…ticket to practise in higher education.”

Trafford accepts that although there will be occasions when problems arise that are the fault of examiners, similarly it may be that the candidate has not been properly prepared “to recognise that the viva is an opportunity to engage in a scholarly discussion”, he adds.

Although candidates’ fear and apprehension often stems from unfamiliarity with the viva process, university regulations are almost without exception clear and readily available, he says. Not only that, but most viva questions are also generally of a similar type and, as a consequence, predictable.

At Bradford, Hartley has used this fact to create mock viva software that he hopes will demystify the process. The program, developed with Gina Wisker from the University of Brighton, asks candidates to respond not to the detailed questions of the PhD but “the more general overarching questions that I’ve seen students struggle with”, he explains.

“They know the detail, but they can’t necessarily step back and say, ‘What makes this a PhD? What’s really significant about the research and how does it fit into previous work?’,” he adds.

Preparation and familiarisation also feature in the advice of Nathan Ryder, a recent PhD graduate who now offers freelance skills training for PhD candidates. He has produced Viva Survivors, a series of podcasts designed to get people talking about their experiences.

He says that after some four years of work, usually including annual reports and presentations and an increasing emphasis on students discussing their research, it is rare for a viva to go badly. “People always say ‘it happened to a friend of a friend’, but I think horror stories are actually very few and far between.”

Thinking back to her viva, Roffey says that although the experience felt like a “grilling” at the time, it was not nearly as horrific as she had thought it would be.

“At the time I would probably have preferred not to do it, but actually it was a great opportunity to talk to two people who’d read the whole thing from beginning to end,” she says. “If I’d just submitted it and got a pass, it might have been an anticlimax.”