Reproducible Generative AI Evaluation for Healthcare: A Clinician-in-the-Loop Approach

ObjectiveTo develop and apply a reproducible methodology for evaluating generative artificial intelligence powered systems in healthcare, addressing the gap between theoretical evaluation frameworks and practical implementation guidance. Materials and MethodsA five dimension evaluation framework was developed to assess query comprehension and response helpfulness, correctness, completeness, and potential clinical harm. The framework was applied to evaluate ClinicalKey AI using queries drawn from user logs, a benchmark dataset, and subject matter expert curated queries. Forty one board certified physicians and pharmacists were recruited to independently evaluate query-response pairs. An agreement protocol using the mode and modified Delphi method resolved disagreements in evaluation scores. ResultsOf 633 queries, 614 (96.99%) produced evaluable responses, with subject matter experts completing evaluations of 426 query-response pairs. Results demonstrated high rates of response correctness (95.5%) and query comprehension (98.6%), with 94.4% of responses rated as helpful. Two responses (0.47%) received scores indicating potential clinical harm. Pairwise consensus occurred in 60.6% of evaluations, with remaining cases requiring third tie-breaker review. DiscussionThe framework demonstrated effectiveness in quantifying performance through comprehensive evaluation dimensions and structured scoring resolution methods. Key strengths included representative query sampling, standardized rating scales, and robust subject matter expert agreement protocols. Challenges emerged in managing subjective assessments of open-ended responses and achieving consensus on potential harm classification. ConclusionThis framework offers a reproducible methodology for evaluating healthcare generative artificial intelligence systems, establishing foundational processes that can inform future efforts while supporting the implementation of generative AI applications in clinical settings.