AI chatbots abandon sleep apnoea referral when simulated patients minimise symptoms
Ratneswaran, D.; Poirier, S.; Hamman, D.; Rykov, M.; Steier, J.
Show abstract
Background: Patients increasingly consult free artificial intelligence (AI) chatbots about their symptoms, but these models may favour agreement with the patient over accuracy. In obstructive sleep apnoea (OSA), where most cases in the general population remain undiagnosed and diagnosis depends on referral, this could validate symptom denial. We tested whether consumer chatbots maintain guideline-concordant referral when a simulated patient minimises symptoms. Methods: Seven guideline-concordant and standardised OSA vignettes (apnoea-hypopnoea-index, AHI 8-52 events/h; body mass index 28-42 kg/m2) were presented to five free chatbots (GPT-5, Gemini 2.5 Flash, Claude Sonnet 4.6, DeepSeek V4 Flash and Grok-4.3) under neutral and symptom-minimising conditions (10 simulated conversations per model-vignette-condition cell; N=700). Referral maintenance was scored by a structured rubric and every capitulation was human-confirmed. Results: All chatbots maintained referral recommendations in neutral conversations (350/350; 100%). When simulated patients minimised symptoms and resisted referral, maintenance fell to 64.3% (225/350), a reduction of 35.7 percentage points (95% CI 5.5-66.0; p=0.028). Rates ranged from 55.7% to 85.7% across chatbots, but evidence of between-chatbot differences was inconsistent across analyses. Rates differed across clinical vignettes (p=0.001). A variance analysis estimated greater variation between vignettes than between chatbots. In the three driving-risk vignettes, referral recommendations were abandoned in 46-68% of symptom-minimising conversations, pooling all five chatbots within each vignette. Conclusions: Consumer AI chatbots abandoned correct OSA referral recommendations in about one-third of symptom-minimising conversations, with higher observed abandonment in simulated cases involving driving risk. This may delay appropriate assessment and raises patient- and public-safety concerns.
Matching journals
The top 8 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Identification of acute exacerbations of chronic obstructive pulmonary disease using simple patient-reported symptoms and cough feature analysis: A diagnostic agreement study 91%
- Passive Detection of COVID-19 with Wearable Sensors and Explainable Machine Learning Algorithms 89%
- Quantifying Sleep-Wake Rhythms in the Hospital Environment with Digital Technologies 89%
Similar papers in this journal
- Identification of physiological adverse events using continuous vital signs monitoring during paediatric critical care transport: a novel data-driven approach 89%
- Feasibility characteristics of wrist-worn fitness trackers in health status monitoring for post-COVID patients in remote and rural areas 89%
- Deep sleep homeostatic response to naturalistic sleep loss 89%
Similar papers in this journal
- A clinical trial to evaluate the dayzz smartphone app on employee sleep, health, and productivity at a large US employer 93%
- Are adversities and worries during the COVID-19 pandemic related to sleep quality? Longitudinal analyses of 48,000 UK adults 90%
- Alarm tones, music and their elements: A mixed methods analysis of reported waking sounds for the prevention of sleep inertia. 90%
Similar papers in this journal
- Scalable Incident Detection via Natural Language Processing and Probabilistic Language Models 91%
- Novel Digital Markers of Sleep Dynamics: A Causal Inference Approach Revealing Age and Gender Phenotypes in Obstructive Sleep Apnea 90%
- Satisfaction with Life in relation to Sleep Health among a Nationally Representative Sample of U.S. Adults 90%
Similar papers in this journal
- Sleep in Frontline Healthcare Workers on Social Media During the COVID-19 Pandemic 91%
- Longitudinal Physiological Data from a Wearable Device Identifies SARS-CoV-2 Infection and Symptoms and Predicts COVID-19 Diagnosis 87%
- Circadian Rhythm Analysis Using Wearable Device Data: A Novel Penalized Machine Learning Approach 87%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.