Patient-centered Evaluation of AI Answers to Genetic Counseling Questions
Huang, Y.; Kambhamettu, H.; Wood, E.; Cacioppo, C.; Ofidis, D.; Mim, R.; Bradbury, A. R.; Johnson, K. B.
Show abstract
The growing use of large language models for health communication raises important questions about patient preferences, trust, and satisfaction with AI-generated content. We conducted a qualitative survey comparing patient perceptions of clinician versus AI answers to questions about Alzheimers disease and genetics. Twenty-six participants scored responses on relevance, trustworthiness, and coherence, and additionally selected a preferred response and provided free-text comments. We found that participants generally preferred AI responses over human-written ones and rated them higher on all evaluation axes. Qualitative analysis of free-text comments identified key factors influencing patient preferences, including clarity, verbosity, certainty, empathy, and numeracy, with patients showing diverse and sometimes contradictory preferences along these dimensions. These findings suggest the AI-generated responses were well-received by patients and may have a role in triaging patients with information-seeking queries. Future work should focus on dynamically identifying patient communication preferences and tailoring communication styles accordingly.
Matching journals
The top 5 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Medical Clinical Minds Meet Artificial Intelligence: Italian Physicians' Knowledge, Attitudes, and Concordance between Italian Physicians and AI-Generated Diagnoses. A National Cross-Sectional Study 92%
- Listening to mental health crisis needs at scale: using Natural Language Processing to understand and evaluate a mental health crisis text messaging service 91%
- User Perceptions of Individually-Tailored Health Information in 1 Digital Apps: Development of a Scale 91%
Similar papers in this journal
- User Testing of a Diagnostic Decision Support System with Machine-assisted Chart Review to Facilitate Clinical Genomic Diagnosis 95%
- Connecting Artificial Intelligence and Primary Care Challenges: Findings from a Multi-Stakeholder Collaborative Consultation 93%
- Measures of socioeconomic advantage are not independent predictors of support for healthcare AI: subgroup analysis of a national Australian survey 92%
Similar papers in this journal
- How suitable are clinical vignettes for the evaluation of symptom checker apps? A test theoretical perspective 93%
- Validating a Clinical Decision Support System for Palliative Care using healthcare professionals’ insights 93%
- User Experience Evaluation of Cogscreen for Screening Mild Cognitive Impairment: Formative and Summative Evaluation 92%
Similar papers in this journal
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.