Evaluating a Medical-Grade Voice AI for Patient and Caregiver Guidance: A Multi-Scenario Nurse Panel Study
Sridhar, S.; Vasantha, R.; Deshpande, S.; Pathak, P.
Show abstract
Medical-grade conversational AI offers the potential to extend patient support programs (PSPs) in regulated therapeutic areas, but its safety and reliability must be rigorously evaluated. We conducted a large-scale, nurse-led assessment of a voice-based AI system across 30 patient and caregiver scenarios spanning diabetes, oncology, neurology, cardiometabolic disease, and rare disorders. Nearly 1,000 U.S.-licensed nurses role-played patients or caregivers in more than 20,000 interactions, scoring the AI across five domains: clinical accuracy, empathy, communication clarity, appropriateness of advice, and compliance with evidence. The system achieved over 97% top ratings across all domains, with empathy noted most strongly in oncology and rare disease caregiving contexts, and clarity reflecting minor opportunities in pacing and call-closing behavior. Qualitative feedback emphasized tone, personalization, and regulatory compliance as consistent strengths. These findings demonstrate that voice-based AI can safely and effectively support patient and caregiver interactions, suggesting readiness for scaled deployment in pharma-led PSPs to expand after-hours coverage, provide consistent patient engagement, and generate feedback loops to inform both AI and human nurse training.
Matching journals
The top 5 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- A typology of physician input approaches to using AI chatbots for clinical decision-making: a mixed methods study 93%
- Utilization of Generative AI-drafted Responses for Managing Patient-Provider Communication 93%
- A Framework to Assess Clinical Safety and Hallucination Rates of LLMs for Medical Text Summarisation 91%
Similar papers in this journal
- Adopting an American framework to optimize nursing admission documentation in an Australian health organization 93%
- Enhancing Research Data Infrastructure to Address the Opioid Epidemic: The Opioid Overdose Network (02-Net) 90%
- Natural Language Processing for Automated Annotation of Medication Mentions in Primary Care Visit Conversations 88%
Similar papers in this journal
- Improving Patient Engagement in Phase 2 Clinical Trials with a Trial-specific Patient Decision Aid (tPDA): A Development and Usability Study 92%
- Understanding how the design and implementation of Online Consultations influence primary care outcomes: Systematic review of evidence with recommendations for designers, providers, and researchers 92%
- Using a Multilingual AI Care Agent to Reduce Disparities in Colorectal Cancer Screening: Higher FIT Test Adoption Among Spanish-Speaking Patients 91%
Similar papers in this journal
- Protocol For Human Evaluation of Artificial Intelligence Chatbots in Clinical Consultations 91%
- Development of the Tool for Advancing Practice Performance, a practice-level survey to assess primary care structures and processes 91%
- Essential Indicators of Quality in Primary Care Settings: An Evidence-Based, Structured, Expert Approach 91%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.