Back

AI chatbots abandon sleep apnoea referral when simulated patients minimise symptoms

Ratneswaran, D.; Poirier, S.; Hamman, D.; Rykov, M.; Steier, J.

2026-10-12 health informatics
10.64898/2026.10.09.26365165 medRxiv
Show abstract

Background: Patients increasingly consult free artificial intelligence (AI) chatbots about their symptoms, but these models may favour agreement with the patient over accuracy. In obstructive sleep apnoea (OSA), where most cases in the general population remain undiagnosed and diagnosis depends on referral, this could validate symptom denial. We tested whether consumer chatbots maintain guideline-concordant referral when a simulated patient minimises symptoms. Methods: Seven guideline-concordant and standardised OSA vignettes (apnoea-hypopnoea-index, AHI 8-52 events/h; body mass index 28-42 kg/m2) were presented to five free chatbots (GPT-5, Gemini 2.5 Flash, Claude Sonnet 4.6, DeepSeek V4 Flash and Grok-4.3) under neutral and symptom-minimising conditions (10 simulated conversations per model-vignette-condition cell; N=700). Referral maintenance was scored by a structured rubric and every capitulation was human-confirmed. Results: All chatbots maintained referral recommendations in neutral conversations (350/350; 100%). When simulated patients minimised symptoms and resisted referral, maintenance fell to 64.3% (225/350), a reduction of 35.7 percentage points (95% CI 5.5-66.0; p=0.028). Rates ranged from 55.7% to 85.7% across chatbots, but evidence of between-chatbot differences was inconsistent across analyses. Rates differed across clinical vignettes (p=0.001). A variance analysis estimated greater variation between vignettes than between chatbots. In the three driving-risk vignettes, referral recommendations were abandoned in 46-68% of symptom-minimising conversations, pooling all five chatbots within each vignette. Conclusions: Consumer AI chatbots abandoned correct OSA referral recommendations in about one-third of symptom-minimising conversations, with higher observed abandonment in simulated cases involving driving risk. This may delay appropriate assessment and raises patient- and public-safety concerns.

Matching journals

The top 8 journals account for 50% of the predicted probability mass.

50% of probability mass above

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.