Zero-Shot Evaluation of Kimi K2 on Pediatric Clinical Cases
Mondillo, G.; Masino, M.; Colosimo, S.; Perrotta, A.; Frattolillo, V.; Abbate, F. G.
Show abstract
BackgroundThe application of large language models (LLMs) in pediatric medicine requires rigorous performance evaluation prior to clinical implementation. ObjectiveTo evaluate the accuracy of the Kimi K2 model in analyzing pediatric clinical cases using a zero-shot approach. Methods: 2,249 multiple-choice questions from pediatric clinical cases, ranging in age from 1 day to 16 years, extracted from the MedQA dataset were analyzed. The model was tested via API with standardized parameters, temperature set to zero, and zero-shot prompts. Accuracy was calculated by comparing the responses with the datasets ground truth. ResultsKimi K2 achieved an overall accuracy of 78.39%, corresponding to 1,763 correct answers out of 2,249 total, with 100% of responses in the required format. Conclusions: The model demonstrates competitive performance for medical education and diagnostic support, while still having limitations that require human clinical supervision.
Matching journals
The top 3 journals account for 50% of the predicted probability mass.