Where to start with generative AI chatbot customisation

Generative artificial intelligence (AI) has made significant inroads into the higher education sector, with huge potential to transform how educators teach and students learn. From AI-generated lecturers at Hong Kong universities to AI-powered apps that transcribe lecture notes, AI-driven tools are revolutionising educational practices by offering personalised learning experiences and improving engagement. One of the most compelling applications of this technology is in the deployment of AI chatbots to support teaching and learning. But it sparks a crucial question: should institutions rely on generic chatbots or invest in customised solutions?

The distinction between generic and customised chatbots

Generic chatbots such as ChatGPT and Gemini are pre-trained on vast amounts of data and generate human-like responses when answering general questions and fulfilling common requests. While they are good at providing broad information, their responses can be generic, predictable and sometimes mediocre, because chatbots lack the contextual information necessary to tailor their responses to specific educational needs. 

• Apply the principles of critical pedagogy to GenAI
• Can AI offer everyone a personal tutor 24/7?
• Resource collection: AI and the university

Prompt engineering plays a crucial role in bridging this gap as part of AI literacy training. A well-crafted prompt can significantly improve the chatbot’s response quality. For instance, by introducing a role statement such as “You are an experienced chemistry teacher” and providing contextual information, such as detailed assignment instructions, the generic chatbot can deliver a more customised experience for students struggling with their chemistry homework. Giving the chatbots some examples of the expected output, also known as few-shot prompting, enables AI to produce more desirable output than asking for help without an example (in other words, zero-shot prompting).

Customised chatbots take this a step further by using system prompts written by designers, such as teachers, to create an interactive and personalised learning experience. These are different from the prompts written by the end users during a conversation with AI. Instead, these system prompts were written by the designers of the chatbot from the back end to guide the chatbot’s interactions with end users. 

There are two basic approaches to writing system prompts: branching and sequencing. Through branching, the chatbot would begin the conversation by offering a menu of options. For example, it could offer to illustrate a technique, practice a technique or test a technique, and provide different interactive experiences according to the user selection. Alternatively, through sequencing, the chatbot can offer the end user a structured learning experience, taking a series of pre-defined steps.

System prompts also enable the educators to address the concerns of using generic chatbots to support teaching and learning, including hallucination and plagiarism. With information and examples vetted by the teachers in the system prompts, customised chatbots are less likely to produce hallucinated responses than generic chatbots relying on pre-trained textual data. The teachers may also add explicit constraints, asking the chatbot to refuse requests to write full-text drafts, and introduce step-by-step scaffolding to prevent plagiarism.

Two pathways to customise a chatbot

When it comes to customising a chatbot for educational purposes, institutions have two options. Using a well-established platform such as Poe.com minimises development costs because the service provider handles the technical aspects. However, a Poe.com subscription can be up to $20 (£15.68) per month. As such platforms often allow a limited space for system prompts – the maximum system prompt length for poe.com is about 2,200 words – it would be challenging for teachers to write elaborate system prompts to design sophisticated learning experiences. In addition, students’ conversations with chatbots customised in such platforms are private and not convenient to share, making learning analytics challenging.

An alternative pathway is to develop an in-house chatbot customisation platform, where teachers could build customised chatbots with a longer system prompt and could access students’ chat history after obtaining their consent. Of course, the up-front costs of developing a platform such as this are significant. Programming expertise and technical know-how are required to send users’ prompts together with system prompts to AI service providers such as OpenAI. This would be undertaken through an application programming interface (API) and with responses returned from the APIs. 

Yet the long-term costs can be lowered. The AI services could be purchased through API tokens based on the actual usage amount of the university community, rather than per-user monthly fees. 

Collaboration between teachers and programmers

To build a successful platform for chatbot customisation, it is essential for front-line teachers to collaborate with colleagues with programming skills. At Hong Kong Baptist University (HKBU), such collaborative efforts have already borne fruit. As a lecturer in English from the language centre, I have been working alongside a programmer, a recent MSc graduate in AI, to develop a chatbot customisation platform over the course of 11 months. 

In a recent seminar on gen AI that attracted nearly 300 participants from more than 70 institutions, we introduced the rationale and features of this platform and received positive feedback and numerous enquiries about trying the platform in the new academic year. The platform has also been used in a summer boot camp on AI that allows secondary school students to develop customised chatbots for promoting STEM education. This collaboration exemplifies how interdisciplinary teamwork can lead to innovative educational tools that better serve students’ needs.

Practical tips for customising chatbots in education

Embarking on the journey of customising chatbots for educational purposes can seem daunting, but with the right approach and resources it can be a rewarding endeavour. Here are a few practical tips to get you started. 

Document existing practices: Review and document your current teaching practices, especially those involving personalised interactions such as one-to-one tutoring. This documentation will serve as valuable data to customise chatbots that can emulate these interactions effectively, making the AI more relevant and useful for your specific educational needs.

Engage with the latest research: Stay updated with the newest studies and findings in the realm of AI and education, especially generative AI customisation. Following and contributing to scholarly discussions, such as the special issue on this topic at STiLE (Scholarship of Teaching in Language Education), can provide insights into the latest trends and successful practices in chatbot customisation. Such engagement also helps in networking with peers who are working on similar projects, providing opportunities for collaboration and innovation.

Leverage internal expertise: Reach out to colleagues within your institution who have programming skills and experience with generative AI. Collaborating with them can help you to understand the technical requirements and possibilities of integrating AI through APIs for a more seamless and powerful use.

By following these tips, you can embark on the path of creating customised chatbots that enhance educational experiences and outcomes, tailored specifically to the needs of your students and your teaching objectives.

Simon Wang is a lecturer in English at the Hong Kong Baptist University.

If you would like advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the Campus newsletter.