Visualising candidate behaviour in computer-based testing: Using ClickMaps for exploring ClickStreams in undergraduate and postgraduate medical examinations

BackgroundThe rapid introduction of computer-based testing (CBT) in UK (United Kingdom) undergraduate and postgraduate medical education, mainly as a result of the COVID-19 pandemic, has generated large amounts of examination data, which we call the ClickStream. As candidates navigate through exams, read questions, view images, choose answers and sometimes change answers and return, re-read, and make further changes, the multiple actions are recorded as a series of time-stamped clicks or keystrokes. Analysing that mass of data is far from simple, and here we describe the creation of ClickMaps, which allow examiners, educationalists and candidates to visualise behaviour in examinations. MethodsAs an example of ClickMaps, we describe data from a single examination lasting three hours, with 100 best-of-five questions, which was one of two papers sat in 2021 by 508 candidates as a part of the MRCP(UK) Part 2 exam. Two ClickMaps were generated for each candidate. The Full ClickMap allows the complete three-hours of the examination to be visualised, while the Early ClickMap, shows in more detail how candidates responded during the first six minutes of presentation of each of the 100 questions in the exam. ResultsSince the primary purpose of this paper is expository, detailed descriptions and examples of ClickMaps from eleven candidates were chosen to illustrate different patterns of responding, both common and rare, and to show how straightforward are ClickMaps to read and interpret. ConclusionsThe richness of the data in ClickStreams allows a wide range of practical and theoretical questions to be asked about how candidates behave in CBTs, which are considered in detail. ClickMaps may also provide a useful method for providing detailed feedback to candidates who have taken CBTs, not only of their own behaviour but also for comparison with different strategies used by other candidates, and the possible benefits and problems of different approaches. In research terms, educationalists urgently need to understand how differences in ClickMaps relate to differences in student characteristics and overall educational performance.