Assessing critical thinking in critical times

Two big questions are swirling around higher education: how do institutions increase their relevance for students and society? And how will generative artificial intelligence (GenAI) impact learning? This critical time reveals the value of what universities have always been meant to do: improve students’ critical thinking. 

Improving critical thinking is neither easy to do nor easy to measure. Outside educational philosophers, it is often defined as something “you know when you see it”. Or, perhaps more clearly, you know it when you don’t see it. When people opine on the absence of critical thinking, they often use examples of illogical assumption, unsupported arguments and poor decision-making. Bolstering critical thinking skills prepares learners to listen to others’ ideas, seek evidence and draw logical conclusions. The way we assess critical thinking offers a pathway for higher education’s sustained relevance in the age of AI.

Assessing critical thinking in the age of artificial intelligence

Some 20 years ago, Grant Wiggins and Jay McTighe summarised the variety of formative and summative assessments available to instructors into five categories: informal understanding checks, observing or talking to students, multiple-choice tests, academic prompts and performance tasks. 

• GenAI as a teaching colleague in assessment: a case study
• Learning by doing in a GenAI-enabled world
• Why GenAI helps some students but not others (and what to do about it)

Instructors get real-time information about students’ critical thinking skills through informal checks for understanding such as Socratic questioning during class or exit tickets at the end of a lesson and through observations of learning or dialogues with students during learning activities. Formative assessment is a powerful first step in guiding the development of critical thinking skills and its interactive, low-stakes nature makes it relatively immune to inappropriate AI use.

Wiggins and McTighe go on to describe summative assessment strategies such as tests, academic prompts and performance tasks as increasingly formal and complex evaluation strategies. Let’s explore how we can level up these assessment approaches to evaluate critical thinking in the quest for enhanced relevance and AI-proof learning.

Multiple-choice tests

Multiple-choice tests (MCT) typically evaluate learning at the lower levels of Bloom’s Taxonomy. When learners have access to infinite information in the palm of their hands, however, it can be challenging to demonstrate the relevance of basic recall and recognition. Critical thinking requires the selection, analysis and application of facts to address complex situations. Our assessments should do the same.

Here are two ways to level up the basic exam: 

• Add “why” to traditional MCT exams. For full credit, learners must explain why the right answer is correct. The approach is not cheat-proof, but it adds another layer of complexity to the exam that increases both relevance and critical thinking.
• Decrease the stakes. With more frequent mini-exams, students’ stress and the temptation to cheat are decreased.

Academic prompts

Academic prompts are open-ended writing tasks that evaluate higher levels of Bloom’s Taxonomy. Strong academic prompts require learners to use evidence to build arguments and justify decisions, and in doing so demonstrate their critical thinking skills. While the style of the assignment is typically discipline specific – be it essays, literature reviews, analytical papers, critiques or case studies – the concern about student use of GenAI is widespread when these assignments are completed outside class.

Here are ways to level up the academic prompt to mitigate AI use: 

• Paper-and-pencil activities. In-class handwritten responses slow down thinking and ramp up engagement.
• Oral exams. It might be possible for wearable technology to assist learners, but oral exams make it much harder for them to lean on AI for responses.

Performance tasks

Performance tasks are complex, real-world assignments that require students to define and solve discipline-relevant problems, often while collaborating with a team. This approach to assessment may be the key to increasing relevance while decreasing learner dependence on AI. After all, along with critical thinking, skills that employers have had on their wish list for decades include communication, leadership, professionalism and teamwork, according to the National Association of Colleges and Employers (NACE). Employers like KPMG US are refocusing their intern training on critical thinking and teamwork, recognising that these valuable, transferable skills will last their employees longer than training with the latest technology.

Decades of research on high-impact practices (HIPs) further demonstrate the educational value of real-world performance tasks such as collaborative assignments and projects, community-based learning, and undergraduate research.

You can level up performance tasks with:

• Community-based learning: Partner with a community organisation to co-create projects that are mutually beneficial to the course and the community agency, such as an industrial design course that paired students with residents of a senior-living community to improve the design of gardening equipment. 
• Course-based undergraduate research experiences: Infuse real research questions into an undergraduate course. This approach prioritises the development of high-fidelity assignments as contrasted with “cookbook” assignments with one right answer. Students are prompted to discover new ideas relevant to the discipline.
• Explore AI: Incorporate AI into project assignments. This could use, for example, student-generated statements about ethical use of AI, student reflections about the quality of learning with AI or tasks for which they want – and don’t want – to use AI. In this way, educators can help students test boundaries of human and machine learning. 
• Scaffolding: Break an assignment into smaller deliverables and provide timely feedback from peers and the instructor. This will reduce the stakes and build students’ critical thinking skills over time.
• HIPs’ quality dimensions: George D. Kuh and Ken O’Donnell identify key elements that faculty can use to create high-impact, real-world assignments. For example, inviting local employers to attend class presentations or changing a paper assignment to a class poster session requires students to display their critical thinking skills publicly, decreasing the temptation to submit an AI-created product.

Demands from students and the public for greater relevance in higher education, along with the advent of widely accessible GenAI, create a pivotal moment for the teaching and assessment of critical thinking skills. Real-world performance tasks help address both concerns, ramping up the practical application of student learning while preparing students to engage professionally in an AI world.  

Kate Williams is associate director for transformative teaching and learning in the Center for Teaching and Learning at Georgia Institute of Technology.

If you would like advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the Campus newsletter.