
When AI is revolutionizing quizzes and assessment in training
AI is a great tool, it opens the door to numerous opportunities in terms of professional training but also in terms of knowledge assessment and skills certification.
Assessment, an essential tool
Assessment can take many forms, from diagnostic evaluation to formative and summative evaluation. It is a powerful tool to enrich any vocational training path. Assessment makes it possible to enrich and boost learning while providing precise indicators on learners' performance.
But assessment can often be perceived as a tedious or painful step, for trainers and instructional designers, as well as for learners:
Trainers should design their training with assessments as an integral part of the learning journey. They must create questions and quizzes, course explanations and reminders, and analyze and report the results. All this can represent an additional workload that can discourage some trainers, especially if they do not have the opportunity to rely on adapted and efficient tools.
Learners, on the other hand, may perceive the assessment stages as a stressful test that can have consequences on the rest of their training, or even on their professional career. Hence the importance of trivializing evaluations, thanks to clear explanations and training quizzes.
The digitalization of assessments and, more generally, of vocational training has opened the way to new practices and new tools have greatly facilitated the work of trainers and the experience of learners.
Generative artificial intelligence has become widespread and accessible. It was quickly integrated into traditional digital tools such as LMS, LAS, LXP and author tools. The results and benefits are sometimes questionable because of both the limitations of AI and the very design of functionalities based on such technology, sometimes more flashy than genuinely useful and relevant.
But artificial intelligence offers real opportunities and has real added value.
AI to create quizzes, quizzes, and evaluations
Artificial intelligence now makes it possible to quickly create a large number of questions on any subject. It can also generate different types of questions in different languages. The creation of questionnaires is therefore much faster and no longer represents an additional workload for trainers.
Assessment with all its benefits is becoming more usable and can be integrated into training courses with greater simplicity, whether in the form of training quizzes, summative evaluations or even microlearning.
Having a large number of questions makes it possible to ask a learner about the same concept in different ways in order to help him truly assimilate knowledge or to check if he really understood and that his first answer was not simply a fluke.
AI also helps to correct open-ended questions that are often too little used because they are too tedious to correct: artificial intelligence can pre-analyze the answers to transform the text into a “score” or “grade”, which will allow the trainer to focus on the lowest grades for example, thus avoiding the need to manually correct all the answers.
The limits of artificial intelligence
Generative artificial intelligence can contribute enormously to evaluation and more generally to professional training, but it also has its limits and it cannot replace the trainer or the instructional designer.
Fundamental limits
Artificial intelligence, by its nature, can hallucinate, offer content that is inconsistent, off-topic, and sometimes even false. Even though this trend was more visible in the early days of ChatGPT, these flaws persist. With each new model, always more powerful and efficient than the previous one, the improvements are palpable, but the flaws have not disappeared.
Thus, in the context of generating bibliographical references for systematic reviews, A study observed that GPT-4 has a level of hallucination of 28.6% (i.e. fabricated or incorrect references), against 39.6% for GPT-3.5 and 91.4% for Bard.
In the medical field, GPT-4 responds correctly to 86.7% USMLE-like questions in some benchmarks, which means that there are about 13-14% errors or inaccurate answers (see the study here).
These figures are a reminder that, despite rapid progress, it is still necessary to keep a critical eye on the results provided by AI, and to check the quality and veracity of the content offered.
Limits in integration
It is very easy to use AI via personal prompts directly on the OpenAI, Gemini or Mistral AI chat interfaces and to copy and paste the elements by hand into a professional training tool. But this approach remains tedious and does not allow the full potential of AI to be exploited.
Artificial intelligence must be integrated directly into the functionalities of the LMS, which must use the AI APIs to generate and retrieve content and automatically feed the media: for example, the platform must, according to the theme indicated by the user, ask the AI, via its API, to create a certain number of questions. The questions created will automatically feed into the question base so that each question generated can be directly used in the LMS, without additional manipulation.
Limits linked to lack of accountability
The ease of use of generative artificial intelligence and its accessibility can have consequences on the involvement and accountability of some people who will see it as an opportunity to produce more, more quickly, without worrying about the quality of the content generated. This practice will inevitably have consequences on the overall quality of the training, since the trainer will not have brought his expertise, knowledge and critical perspective.
All the content created must be re-read, corrected and enriched by the trainer so that it is useful for the learners and that it enriches the training. AI, by itself, cannot replace the trainer.