When to use an AI professor (and why you might not need one)

“Aiyoh, you ask one very solid question lah!” *

When students hear this distinctly Singaporean response from our AI teaching assistant, they know they’re engaging with something unique. But as universities worldwide rush to implement artificial intelligence in their classrooms, our experience offers crucial insights into what makes an effective AI teaching assistant – and, more importantly, why it isn’t suitable for every course. 

Our journey began with a fundamental question: in an era where AI tools are becoming as ubiquitous as search engines, how do we create meaningful educational experiences while preserving the essential aspects of traditional learning? Rather than fighting the tide of AI adoption, we developed our own pseudo-instructor: Professor Leodar, a Singlish-speaking AI teaching assistant with a carefully crafted personality and deep integration with our curriculum.

Professor Leodar has handled more than 12,334 queries from 154 students since January 2024, but the numbers tell only part of the story. What makes our selective embrace of this technology distinctive isn’t just the creation of a virtual instructor with local cultural awareness – it’s our careful consideration of where and when such tools are appropriate.

Image

Let me be absolutely clear: there is no substitute for the fundamental academic experiences that shape deep understanding. The midnight oil must still burn as students work through complex derivations by hand, synthesise information from multiple sources and develop the critical thinking skills that come from wrestling with difficult concepts. Our AI teaching assistant is a supplement to – never a replacement for – these essential learning processes.

Our development process focused on three key elements often overlooked in the rush to implement AI solutions. First, we carefully selected a technical subject – data science and artificial intelligence – where answers have clear right/wrong outcomes as a pilot course. We deliberately avoided subjects requiring deep conceptual synthesis, complex derivations or extensive critical analysis – areas where traditional study methods remain irreplaceable. Second, we ensured strong educator leadership throughout development, recognising that pedagogical expertise is crucial in determining appropriate use cases. Third, we built a comprehensive knowledge base from course materials before implementing any AI functionality.

• How AI teaching assistants are transforming student support
• Spotlight guide: Bringing GenAI into the university classroom
• DeepSeek and shallow moats: what does it mean for higher education?

The technical architecture relies on retrieval augmented generation (RAG) technology deployed via Amazon Web Services, allowing real-time updates for course materials and announcements. However, we are careful to limit its scope to appropriate applications – primarily in the particular subject where solutions follow clear patterns and methodologies.

The user analytics tell a fascinating story about student engagement. During our first continuous assessment week, we recorded more than 1,200 queries – but the pattern of usage told us more than the volume. Most revealingly, our students used Professor Leodar when instructors were not available, either in the wee hours of the night or during the weekends. Also, our highest-performing students don’t use Professor Leodar as a crutch; they use it as one tool in a broader learning strategy. While struggling students often seek quick answers, top performers use the AI assistant to supplement their understanding after putting in the necessary groundwork through traditional study methods.

In our data science and AI course, successful students consistently demonstrate an ability to grasp fundamental concepts through traditional learning before using AI to explore applications. They spend long hours in front of their computer not because they rely on AI but because they’re doing the essential work of understanding core principles, working through problems step-by-step and developing deep conceptual understanding.

The impact has been significant: 97.1 per cent of students report positive experiences with Professor Leodar. However, this success comes with an important caveat: it’s limited to specific types of courses and specific types of questions. We actively discourage its use for tasks requiring deep analytical thinking, theoretical understanding or creative problem-solving. In these areas traditional study methods remain essential.

For institutions considering their own AI teaching assistants, our experience offers three crucial lessons. First, resist the temptation to deploy chatbots across all courses; many subjects require traditional learning methods that cannot and should not be replaced. Second, carefully evaluate where AI assistance is truly beneficial versus where it might hinder deep learning. Finally, maintain rigorous academic standards that require students to demonstrate understanding through traditional means.

As we prepare students for a workforce where AI proficiency isn’t optional, the question is not whether to embrace AI tools but how to integrate them meaningfully – and selectively – into education. Our experience suggests that success lies in finding the right balance: leveraging AI where appropriate while preserving the irreplaceable aspects of traditional academic rigour. The midnight oil will still burn in university libraries and laboratories, not because students are asking chatbots for answers but because genuine understanding requires time, effort and dedicated study.

* “You have asked an excellent question.”

Leonard Ng Wei Tat is assistant professor in the School of Materials Science and Engineering at Nanyang Technological University, Singapore.

This is an edited version of the blog post “Eh, don’t play play! What we learned building a successful AI teaching assistant (and why you might not need one)”, which was first published by NTU’s Institute for Pedagogical Innovation, Research & Excellence.

If you would like advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the Campus newsletter.