How to assess and enhance students’ AI literacy

The infiltration of AI into various aspects of modern daily life raises questions about whether we are prepared to understand, use and interact with it. This is where AI literacy, a combination of theoretical knowledge and practical and critical thinking skills related to artificial intelligence, comes in. This article explores the capabilities and knowledge we must empower our students to develop. 

How do we measure AI literacy?

Students must be able to:

• Identify whether the computer system they interact with is AI-powered and how to communicate effectively with its interface
• Understand how AI features work and critically assess their strengths, weaknesses and limitations, how the technology collects data and how our data is used to train their models
• Think critically about the AI-generated content, judge the implications of its use, question assumptions, assess reliability and identify potential bias, risks and benefits
• Demonstrate awareness of the ethical issues and privacy and security concerns.

Using ChatGPT as an example, students must know how large language models work and how the tool was trained for tasks like question answering. They need to know how to write prompts to get relevant answers to their questions. They must understand that the chatbot is trained on data from diverse internet sources such as books, articles and encyclopedias and incorporates human feedback. They must be able to evaluate the responses it gives to decide whether to accept them. Encourage them to ask themselves: 

• “Is the answer correct?” 
• “What is the source of the information?” 
• “Is the answer a result of AI hallucination?”
• “Is there any bias in the AI-generated response?” 

Finally, students need to understand the impact of AI-generated content on academic integrity. Does the course they have enrolled on permit its use for assignments? If the use of AI-generated content is allowed, is there a limit to the percentage of the work that can be produced by AI? Will you as their teacher use tools to detect AI-generated content? Writing your policy down on paper will help students understand your boundaries.

• Resource collection: AI and assessment in higher education
• ChatGPT and generative AI: 25 applications in teaching and assessment
• AI and assessment redesign: a four-step process

Filling in knowledge gaps with literacy programmes

Institutions can educate their students about AI and assess their readiness to use it by running AI literacy programmes that feature general and specialised modules tailored to their areas of study and career aspirations. For instance, a business student may need to learn how to use AI-powered tools for data analysis while medical students will need to be well versed in using them to assist with medical diagnoses. 

A generic AI literacy programme could contain modules on the technical understanding of AI, especially machine learning and generative language models. Students must understand how the technology perceives the environment and collects and processes data. It can incorporate practical elements by requiring students to learn how to communicate effectively with AI models. This can involve teaching best practice prompt writing, followed by a prompt-writing task. 

We can also teach students to use AI tools to enhance productivity. For example, we can train them to use virtual assistants to set up calendar events and meetings, organise emails and generate to-do lists and transcripts of online meetings, put together effective presentations, create images from textual documents, analyse data, spot patterns and predict trends. 

It is also useful to show students how to use writing assistant tools such as QuillBot or Grammarly to check their grammar and language use. As teachers, we can use these to assess writing skills, give feedback and detect plagiarism. 

Finally, we can design exercises aimed at helping students understand data privacy laws and other ethical concerns as often the student may have to input data into the tools.

AI technologies are permeating many aspects of daily life, impacting society and transforming industries. This makes it all the more important to incorporate AI literacy into the curriculum so that we can help our students enhance their productivity, make informed decisions, participate meaningfully and shape AI’s responsible and ethical development.

Rohini Rao is an associate professor in the department of data science and computer applications at Manipal Academy of Higher Education, India.

If you would like advice and insight from academics and university staff delivered directly to your inbox each week, sign up for the Campus newsletter.