Is it time to reassess student assessment?

A negotiation of power

“She mopped the floor with us in discussions.”

I was stung by this comment on an anonymous teaching evaluation during my first year of teaching – and it has stuck with me through the intervening six years. I thought I was collaborating, coaching, creating an intellectual community: inviting my students to test out new ideas even as I challenged them from my standing as an expert in my field. Yet this student saw me as an uncomfortable mix of referee and competitive peer with an unfair advantage. What I thought of as a playground was perceived as an obstacle course.

I suspect now that my failure to teach through dialogue stemmed from the fact that the class – designed in imitation of those I had experienced as a student – had traditional exams: namely, two comprehensive tests, a midterm and a final. This constructed hurdles that inhibited intellectual experimentation, that vital skill that turns humanities students into effective question-askers, data-finders and alternative-weighers. In subsequent iterations of the course, I introduced assignments that did not primarily assess retention but helped students exercise imagination, practise empathy and chase curiosity.

These included in-class exercises in which students researched characters from the past and then role-played them in a re-enactment of historical happenings. In my history classes, I have also had students compose an ancient letter in the voice of a historical person – famous, infamous or traditionally voiceless – and then annotate the letter as if they were a modern scholar who “found” it. This type of assignment invites the student to demonstrate mastery of content (both primary and secondary sources) while having the benefit of cultivating creativity.

Perhaps most importantly, feedback from me on these assignments feels to students like trusted coaching rather than refereeing – although I also required traditional research papers from them, with university-mandated word lengths; it seemed a natural complement to encouraging curiosity.

As the pandemic hit, professors who care deeply about student success have scrambled to help themselves and their students adapt not only to new online platforms, but also to new time constraints, anxieties and griefs. The least advantaged, in particular, have been facing unanticipated hurdles and pressures that are much worse than exams.

This has made it difficult for students (and teachers) to be creative, focus on big ideas or take any kind of risk. One cannot play when survival is at stake. And since assessment should ultimately serve the student, we teachers should modify our modes of assessment to meet their needs. For me, this meant flexible deadlines, zero questions about absences and a couple of cancelled classes with the direction to exercise or sleep. I came to realise that I would rather extend compassion, with the risk that a cheater scores a point, than police my students to the brink of mutual exhaustion.

Yet rather than decreasing the quality of the students’ education, I believe that the result has been greater pedagogical effectiveness. Because what is left is that which matters most.

Allow me to share the conclusion to which the pandemic’s forced paring down of pedagogical priorities has led me: it is time in the undergraduate humanities classroom to do away not only with the high-pressure exam but also with the traditional research paper.

Digging in libraries belongs to an older generation. With the steady stream of in-your-face information and misinformation available, the urgent skill that students need now is not research but analysis. They need to be able to follow an argument, find and define its explicit and implicit terms, assess its logic and scrutinise the ways in which the data underpinning its empirical claims have been gathered and are being deployed. They also need to be able to contextualise the argument within larger conversations and to interrogate their own instinctive responses to it.

Such skills can be practised through multiple models of assignments, such as low-stakes presentations, short essays, staged debates and blog posts. Instructor feedback on such assignments is sometimes hard for students to confront because it strikes right at the heart of what and how they are thinking. But when the challenges are pressing and compassion reigns, the learning environment can be magic.

Assessment in a university classroom is, indeed, always a negotiation of power. Some students, like my former one who felt mopped years ago, will be empowered by this approach. Others will continue to feel entitled to the A they’ve always thought they deserved. But I have come to believe that my job as an educator is to persuade, not to discipline.

Jill Hicks-Keeton is an associate professor of religious studies at the University of Oklahoma.

What gets measured gets done

For most salaried academics in developed countries, the pandemic-induced move online has been a godsend. It has meant no commuting, no office hours, and (best of all) no need to talk to colleagues. Just zoom into the monthly staff meeting, turn off the camera and read the newspaper. Or catch up on housework.

We also love being relieved of the need to set and mark in-person tests and exams by the rise of the three Ps: projects, presentations and papers. Projects can actually be fun to grade, even when they’re not very good. Group presentations let us give four or five grades in the 15 minutes it takes to sit through a slide deck. And while grading papers can be a pain, at least they offer some variety compared with exam answers.

Source: 

Getty

In any case, seen from a purely pedagogical standpoint, tests and exams can seem like a waste of time. We lament that they take up valuable class sessions that could have been used for learning, and our hearts sink at the thought of reducing our sophisticated subject areas to multiple choice questions. Tests are just so...unfulfilling. Multiple choice tests most of all.

The only problem with the shift away from formal testing is that it relieves our students of the burden of actually having to learn anything. That’s not a problem for us – but it is a problem for them.

Not that the students are likely to care. In a class marked mainly on the three Ps, they need only focus on deliverables. Since any good syllabus specifies up front exactly what deliverables will be expected, there are few incentives for students to focus on anything else. Only the most highly motivated will diligently study material that is not “on the test”.

Why attend an entire semester of online lectures if your integrative project will not directly draw on the material taught in them? Why spend hours mastering class material when you know that presentation marks are inordinately influenced by good graphics and speaking skills? Why dissect dozens of difficult readings if you’re allowed to pick your own topic for a final paper?

The answers are obvious. We need tests and exams for one reason and one reason only: to ensure that students study. The foundational first principle of management theory says it all: “what gets measured gets done”.

Without exams, few students learn anything other than how to recycle the same projects for multiple classes, bluff their way through group presentations, and cram selected research into narrowly focused papers. The three Ps also open multiple opportunities for cheating by hire, but that is beside the point. Even when students submit their own work in good faith, even when they are well intentioned and really do mean to attend all the lectures and read all the readings, they are unlikely to actually find the time to do so without the harsh reality check of a comprehensive exam.

Tests also hold teachers and even administrators accountable. Teachers who poorly frame test questions can have their professionalism questioned. And administrators have little wiggle room to massage marks when they are based on black-and-white test answers.

We’re all human, the students probably more so than us. If academics cut corners on Zoom meetings (and let’s be honest: we do), what makes us think that our students don’t? If we really care about our students and want them to learn, testing is the tough love that shows it. And although it is possible to examine students online, when it comes to truly tough love, there’s no substitute for the old-fashioned desk, pencil and paper exam book.

Salvatore Babones is an associate professor in sociology at the University of Sydney.

The full force of the law

I was able to teach face-to-face for most of the last semester. It was wonderful to experience again that energy and feedback from my students. But our collective experience of assessment was largely unchanged.

The main impact of Covid-19 on assessment in law has been to impede our ability to assess verbal skills, via face-to-face “moots” (legal debates) or class participation and presentations. In one subject I taught last year, for instance, we swapped the usual presentation for graded participation in an online (written) discussion board. This might seem strange when we had Zoom, but, just like teaching, giving a presentation online isn’t the same as doing so in person because you lack a sense of the room. There’s also the ever-present fear of technical issues or bad connections, particularly for overseas students.

In the compulsory law subjects I teach, though, the main method of assessment has always been an open book examination. The nature of law means there is no single “right answer” to the hypothetical legal problems it sets. Students must know the basic case law, the terms of the legislation and the legal principles, but good students apply the law to facts with an appropriate emphasis.

We also assess students’ ability to communicate arguments clearly and to understand why the law operates as it does (including whether there are ways the law can be improved). And we ask them to complete research essays and research memoranda, as they would in practice when faced with complex issues they hadn’t encountered before.

Source: 

Getty

When assessment moved online, we simply give the students slightly longer to complete the exams, in typed form (much easier to read than handwriting). Of course, concerns arose about possible collusion, but even before Covid-19 we struggled to get the right balance between assessment that fairly judges skills and allows considered thought while ensuring that students can’t cheat. Online or off, this risk is mitigated by the fact that the timing is quite tight, the answers are not searchable online, and the problems are specifically written for that exam; students who cut and paste from notes also tend to do badly as problems are specifically drafted to be unsuited to generic answers.

Of course, it is tempting to tip the assessment balance more towards group work. As law exams can be hard to assess if you don’t know the subject or haven’t taught it before, instructors at my law school often mark our own exams, rather than delegating marking to teaching assistants. Simply “spotting the issues” is not enough: students are also assessed on the quality of their discussion, with reference to appropriate case law, statute or academic opinion. This also means that marking can be time-consuming. But while setting group work can limit the marking load, it can create problems of its own when disputes within student groups arise. This is why I hated group assignments as a student  – and why I tend to avoid setting them now.

Still, marking loads in large or compulsory subjects shape the nature and number of exams. Their necessarily short, sharp nature is both their advantage and disadvantage. Students can sometimes be panicked by them, losing their ability to write considered answers. On the other hand, there is no time to plagiarise or collude; longer take-home exams can be open to this unless carefully designed.

For law students, the ramifications of being caught cheating go beyond university. Any official finding of wrongdoing must be reported to the authorities when seeking admission as a lawyer, as it goes to the question of good character. Covid-19 hasn’t changed this peril: it has just exposed it more.

Katy Barnett is a professor at Melbourne Law School. The views expressed here are her own.

Questions of authenticity

In the ignorant bliss before Covid-19, there were some unspoken and uncomfortable truths that challenged the status quo in higher education. Lecture attendance was threatened by lecture capture. Staff and student well-being was a persistent problem. And traditional exams were rarely “authentic”: the skills they test usually lack real-world application.

The pandemic became the catalyst for overdue change and a retrospective refocusing of higher education’s values. Engagement is what matters, not attendance. Learning technology combined with evidence-led pedagogy augments rather than threatens the instructor. Working and learning practices that are inclusive of those with digital accessibility issues and caring responsibilities need to be normalised.

Yet one aspect of traditional practice remains particularly resistant to change: assessment.

If conducting traditional exam-style assessments online, there is greater scope for (and reported incidence of) collusion, impersonation and contract cheating. Meanwhile, the use of proctoring software or remote invigilation is invasive and can be highly detrimental to student mental well-being. Far better, some academics have concluded, to revert to traditional in-person exams as soon as possible.

Source: 

Alamy

But this is the wrong answer to the wrong question. Cheating is arguably a problem in the exam hall, too. Moreover, the goal of assessment is surely to provide a fair and consistent platform for students to demonstrate the intended learning outcomes. Yet what are those outcomes? After all, how students demonstrate their learning in the exam hall is seldom how they would do so in employment – as is often evidenced by performance in graduate employers’ pre-recruitment assessment exercises. Memorisation and reproduction are of questionable value in an increasingly technology-enabled world.

Many academics who have reflected on these philosophical questions have moved away from assessing replication and recall, towards an authentic form of assessment that engages and empowers students not just to know their subject but to do it.

Here at the University of Stirling, I and many colleagues had already ditched the exam hall pre-pandemic, in favour of authentic pieces of assessment such as digital artefacts, policy briefs, case-study data projects, blog posts, investor pitches and stakeholder engagement tools. Giving students coursework tasks that they can become invested in is not only good assessment practice but also promotes digital accessibility and inclusive learning.

Yes, some subjects may still require on-campus practical assessments or paper-based assessments that test students’ ability to solve problems and recall knowledge in a time-sensitive, pressured environment. However, there is nothing to prevent such tests from being authentic to how students will be using their learning after they graduate.

Yet, as the adage goes, the best things in life are never easy – and in higher education, the best things in assessment are often more resource-intensive.

While digital assessments such as quizzes can be pre-programmed to mark themselves and give detailed, instant feedback en masse, their design requires a significant time investment. This is exacerbated for those designing whole test banks. Moreover, most forms of truly authentic assessment require much more time to mark than traditional exams because open-book, real-world application is much more nuanced than simple recall. It is therefore imperative that universities recognise and reward the time cost in authentic assessment practice.

Still, when our campuses reawaken, we must on no account return to speaking quietly of uncomfortable truths. We must remain energised by the sense of new possibilities, combining the best of in-person teaching with an innovative use of learning technologies, underpinned by principles of pedagogy, inclusion, accessibility and empathy.

So keep shouting your philosophical questions loud. What are we trying to assess? And why?

Paul Cowell is a lecturer in economics and deputy associate dean of learning and teaching at Stirling Management School, University of Stirling.

Computer says no phones

Covid-19 abruptly shifted higher educational instruction online, but my methods for evaluating medical and pharmacy students’ knowledge of the biological sciences have not differed much from previous approaches.

Most of the exams I set are already computer-based, and these were simple to move to electronic learning platforms if they were not already there. This format has the advantage of allowing undergraduates to receive their scores and feedback as soon as the exam closes. Many online learning platforms also have fairly robust software configurations that can be added to testing modalities to reduce – or seriously complicate – cheating. These include timed questions, scrambled questions and answer selections, and IP address-detection.

Honour codes are also useful. Having students digitally confirm before an exam opens that they will not receive or offer “help” – and confirm at the end that they stuck to this pledge – keeps my testing environments fairly honest.

Source: 

Getty

Of course, I will never be able to police or prevent all cheating, so I also rely on maturity and decency from my students. Fortunately, my limited experience is that cheating students are eventually caught or that their behaviour does not reward them for long, so the situation self-corrects – sometimes in spectacularly memorable ways.

The most important reasons that I would not consider changing my assessment strategies at this time relate to some very practical realities. First, I am charged with turning my undergraduates into future doctors, pharmacists or PhD students, with a foundation of science, logical reasoning skills, and superb reading comprehension and writing skills. Before they graduate, they take professional or graduate entrance exams, which look very much like the tests I give them. It seems to me that the assessment method on which their future success or otherwise will depend should be familiar to them.

I have also learned that the generation of students I teach often have self-described “anxiety” issues with speaking up in class (they are afraid to be wrong, they say). And I have found that student presentations are time-consuming to hear and grade. I teach six unique classes a semester, with more than 600 students across them. Marking 1,200 presentations a year would likely be my undoing.

Moreover, presentations are prepared demonstrations of knowledge, not the rapid, on-your-toes evaluations of knowledge that the future careers of my medical and pharmacy students will demand. When they enter the workplace, they will not be allowed to look up a diagnosis or first-line therapy. Rather, they will be asked for these answers, often in front of patients, and they will be expected not only to answer but also to be correct.

Graduate students may have the luxury of researching the best model for a study design, but this will come from thoroughly reading the peer-reviewed literature and being able to extrapolate what they read to what needs to be studied.

In all of these situations, a smartphone might be able to produce an answer, but I’d guess that this answer would be too simple or just wrong. The current level of technology is such that there is still no substitute for human doctors, pharmacists or researchers. In that context, breeding intellectual laziness and poor introspection into those humans while they are students seems like a prescription for disaster.

Jennifer Schnellmann is an associate professor in the Colleges of Medicine and Pharmacy and the associate director of undergraduate studies in pharmaceutical sciences in the College of Pharmacy at the University of Arizona.

Project synthesis

A paradox of our times is that while information is overwhelmingly available through the internet, there are increasing calls from beyond the academy to reduce undergraduate education to providing information – in the form of “hard skills”.

Any productive discussion of assessment must reject this. We must defend the core goals of teaching students to evaluate, analyse and think critically, empathetically and creatively about information – and to apply those skills to solving problems.

Our rethinking of assessment should prioritise three issues. The first is always to interrogate the purpose of assignments. What is the use of exams? How do we connect the classroom with life beyond the classroom? How can we fully engage students while still expecting them to apply critical thinking and other core humanities skills with rigour, and in a way that we can evaluate consistently?

One alternative to traditional assessments is “unessays”, which take a wide variety of forms, ranging from board games to visual art projects to podcasts. Their creativity has a particular appeal at this stage of Covid exhaustion – and it is far easier for students accustomed to Instagram and TikTok to imagine the applicability of creating a visual representation of an idea than to appreciate how mastering exam-taking skills will apply to life after college.

Source: 

Getty

But because unessays are a relatively new form of assignment, fair and consistent evaluation can be exceptionally challenging. Adopting this type of assessment will require productive departmental discussions about goals and standards.

The second issue we need to keep in mind is cheating. During Covid, academic dishonesty has plagued schools across the globe. But rather than insisting on harsher rules or more invasive surveillance, we need to persuade students that they hurt themselves in concrete ways when they cheat.

Most students will not value a class simply because it’s required. If they don’t think they’ll use the material after leaving the classroom, what are they losing by having someone else do their work?

In presenting course material and designing assignments, therefore, we need to be more transparent about applicability beyond the classroom. Not by watering down content but by talking regularly with students about how the thinking, questioning and investigating they learn are critical tools for professional problem-solving.

In other words, regarding any given assignment, what is in it for them? If we want to make assessment productive and discourage cheating, the answer to that question has to be something more substantial than a grade and more tangible than “an educated mind”.

Finally, we need to make sure we assess students’ overall learning. Demonstrating the importance of pulling together information, questions and methods from across students’ college careers is the key to defending the value of higher education in the 21st century.

An effective approach might be to present all seniors with the same “real world” problem regardless of their major or path of study. For example, a case study about the outbreak of a global pandemic. The assignment would be to submit all the questions they could imagine would be necessary to answer in order to fully address the problem.

Students with a broad undergraduate education could be expected to craft questions on topics ranging from the medical and scientific issues to political, economic, social, cultural and religious issues that would shape the pandemic’s spread. This would test their ability to ask a broad range of necessary questions – which, in their life after university, they would then answer by tracking down the necessary information and working with other people to construct a solution.

None of these points require us to do away entirely with traditional forms of assessment. For example, a final unessay might be combined with quizzes and more traditional essays earlier in the semester, or build on a written proposal or analysis. The point is that we must think carefully and creatively about ways to maintain our pedagogical and intellectual goals while updating them for our Covid and post-Covid times.

Karen E. Spierling is professor of history and director of the global commerce programme at Denison University in Ohio.

Sympathy for Mr Gradgrind

“Now, what I want is Facts. Teach these boys and girls nothing but Facts. Facts alone are wanted in life.”

Dickens’ Mr Gradgrind opens Hard Times with this famous address. And it sometimes feels as though fact regurgitation is all that many academics and university leaders think exams are for, too. The truth, as even Mr Gradgrind himself eventually realises, is somewhat more complex.

Source: 

Getty/Alamy montage

There is no doubt that the pandemic has had a huge effect on education. It forced us to do new things, not least in assessment. But with the pandemic seemingly in retreat, at least in places, should we now go back to the way things were?

Many say no: exams are dinosaurs and not worth the paper they are no longer written on because they only test Mr Gradgrind’s facts when knowledge retention is obsolete in the smartphone world. But why do I need a qualified forensic scientist, pharmacist or nutritionist if I can just look up knowledge myself? The answer, of course, is that having knowledge is one thing – being able to apply it correctly, in a new situation and under pressure, is quite another – and a well-written exam can test those abilities. Yes, I can look things up on my smartphone, but I must understand what information I need. I must know that what Google brings back to me is correct – and someone has to write it in the first place. Moreover, what if the machine stopped?

Gradgrindian as it might sound, there is value in knowing the basics. A physics student who doesn’t know how to solve first-year quantum mechanical problems is not going to be able to solve second- or third-year quantum mechanical problems either, no matter how many times they google it. Similarly, would you want somebody conducting a Covid test or developing a vaccine for you and your family if they had to ask Siri how to do it?

I am not against many of the innovations that the pandemic has brought. I like some of the alternatives to exams, such as bootcamps (short-term, intensive skill development). And continuous assessment, via projects and placements, does have the advantage that a student’s entire grade does not come down to three hours of writing at the end of term.

I can also understand the institutional desire to teach more students with the same resources. Physical exams need a room, which often costs money to hire, and are generally more labour-intensive to mark properly than projects or presentations.

But online exams and alternatives are not panaceas and they bring their own problems. If online assessment is to go mainstream, for example, we will need to address the increasingly large problem of contract cheating. The explosion in essay mills and online “help” sites in recent years makes it impossible for academics to be sure the person submitting work online is the person who wrote it. Invigilated exams can be gamed too, but they do make this issue easier to manage.

Perhaps, in truth, there is no one-size-fits-all approach to assessment. But, in my view, at least, the total death of the exam has been greatly exaggerated.

Oliver A. H. Jones is a professor of analytical chemistry and associate dean for biosciences and food tech at RMIT University, Melbourne.