Effective assessment practices for a ChatGPT-enabled world

ChatGPT can provide step-by-step solutions to mathematical problems, write code and generate lesson plans, draft emails and produce stories, music lyrics and social media posts. Therefore, it is not surprising that many in academia are excited about the potential positive influence of ChatGPT on productivity and efficiency.

On the other hand, ChatGPT has triggered a lot of debate around integrity issues in higher education, including concerns about plagiarism and cheating in and outside the classroom. AI models designed for plagiarism detection are being developed. However, this problem naturally leads to an increase in faculty workload to deal with challenges in detecting AI usage in their students’ work. Also, beyond modifying their course syllabus, instructors will need to be trained on new assessment practices in response to students’ exposure to and usage of ChatGPT.

• Collection: AI transformers like ChatGPT are here, so what next?
• THE podcast: is AI in higher education worth the hype?
• Prompt engineering as academic skill: a model for effective ChatGPT interactions

There is a third path: generating ideas for meaningful and innovative assessment methods for instructors, while also embracing the advancement of large language models. Based on a synthesis of recently published papers and other web sources, this article offers practical suggestions on assessment methods that instructors can consider using in their courses.

Assessment practices to enhance learning

First, given that ChatGPT tends to generate high-level and generic text due to its training mechanism, if we want to work with it (or at least accept its presence) and maintain academic integrity, we need to consider using authentic and personalised assessment methods in courses. These methods include using real-life examples and contextually specific situations that are meaningful to individual students.

Set personalised, complex or topical tasks

Instructors may ask students to include their personal experience or perspectives in their writing. Students can also be asked to conduct analysis that draws on specific class discussions. Another way to promote this type of assessment method is to give students complex instructions that involve long texts that do not fit a typical ChatGPT prompt or to ask students to write about the most recent events that are not necessarily reflected in the data used to train the language model.

Consider options that are not text-based

Alternatives to essay-based assessment also need to be further explored. These methods can include using (impromptu) video presentations for assessments or using other digital forms such as animations.

Tap into students’ thoughts, reflections and process

Through self-assessment or reflective writing, students could discuss their writing or thinking process. Additionally, peer evaluations or interactive assessment activities could be integrated into grading by engaging students in group discussions or other activities such as research and analysis in which students are expected to co-construct knowledge and apply certain skills. Instructors may consider placing an emphasis on assessing the process of learning rather than the outcome. Instead of grading a final writing product alone, instructors could grade multiple drafts and assess how a student’s writing improved.

Assess critical thinking and avoid formulas

In addition to its ability to write college-level essays, ChatGPT allows students to quite easily find answers (albeit not necessarily accurate ones) to coding assignments and physics problems, for example. Thus, assessments that are formulaic need to be avoided. Instead, instructors could use assessments that are open-ended or encourage students to demonstrate originality and creativity in their performance. Creating original ideas or research questions can be one such example. Considering these aspects, it is crucial to find ways to assess students’ deep, sophisticated and critical-thinking skills. Some example assessment methods include asking students to critique papers (or even text output generated by ChatGPT) or build logical arguments with appropriate reasoning skills and create a concept map or diagram to demonstrate a deep understanding of a topic and its knowledge structure.

Take advantage of AI’s shortcomings

One of the limitations of the current version of ChatGPT is that it does not provide appropriate sources and quotations in its text output. Instructors may use this as an opportunity to engage students in writing practices focused on correcting factual errors and locating accurate data sources. Students can also be asked to cite and reference the work of others accurately and properly by using in-text citations or including bibliography at the end of their writing product.

Beyond learning the mechanics of writing, instructors could ask students to critique a piece of writing generated by ChatGPT through analysing and interpreting how it conveys an idea and assessing its strengths and weaknesses in terms of readability, credibility, comprehensiveness, accuracy and so on.

Ask ChatGPT to design assessment

Lastly, instructors can use ChatGPT as a supplemental tool for designing assessment activities. Instructors may ask detailed and specific prompts of ChatGPT to generate a rubric for grading students’ knowledge and skills, create multiple-choice quiz questions or come up with group discussion questions about course topics. However, when using these sources, it would be important to ensure they are accurate and to check how the information presented can be interpreted or if it contains any biased views.

The takeaway? Focus on meaningful, student-focused tasks

Ongoing discussions about assessment methods in light of the growing influence of generative AI such as ChatGPT remind us to adopt more student-centric assessments that can create meaningful learning opportunities for individual students. Design assessments that can provide students with ample scope to apply their skills to analyse ideas and integrate concepts in order to deepen their knowledge.

Beyond assessing these cognitive skills, how to incorporate assessments of non-cognitive skills such as collaboration, communication and leadership into classroom learning needs further discussion. That is, instead of emphasising skills that are traditionally measured by standardised or psychometric tests on which AI can perform very well, we should emphasise skills that are useful and meaningful for human learning.

Tools such as ChatGPT do not necessarily generate factual information. Rather, they produce pieces based on the probability of potential relationships between words. Depending on the questions asked, the outputs generated by ChatGPT may be incorrect, outdated or illogical. It has a limited understanding of context, can potentially generate offensive or harmful content, has limited ability to fact-check or verify information, lacks empathy and has limited knowledge on certain topics (especially if the subject matter is complex). In such case, instructors must caution students about the limitations of such tools and encourage them to compare the outputs with other reliable sources to prevent the spread of misinformation and misconceptions.

Jonna Lee is director for research in education innovation in the Center for 21st Century Universities at the Georgia Institute of Technology.

If you found this interesting and want advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the Campus newsletter.