Version-aware question answering over living oncology guidelines
Ravi Kumar, T.; Kaniyar Narayana Iyengar, A. I.; Khan, M. A.; Khakwani, K. Z. R.; Kathuria, K.; Sonbol, M. B.; Riaz, I.; Gupta, V.
Show abstract
Living clinical guidelines revise recommendations as evidence emerges, so an AI answer can be faithful to a real guideline and still be out of date. Here we examine this failure using ASCOBench, 288 unique questions in 96 three-turn conversations grounded in versioned American Society of Clinical Oncology (ASCO) breast and prostate cancer guidelines, with oncologist-reviewed reference answers. Oncologists corrected one in four version-sensitive answers drafted by a frontier model, against fewer than one in ten factual ones. Across the guideline corpus, 19 recommendations changed between versions, seven of them reversals, yet only 1 of 14 superseded documents states that it has been replaced. With a strong model, retrieval over several guideline versions made stale answers four times more frequent than answering without retrieval. SentryLine, a verification-first system, lowered incorrect answers to 4.2-5.2% from 9.7-22.9% for baselines across three models. Recognizing change before a guideline declares it remains an open problem.
Matching journals
The top 9 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- DeepPhe-CR: Natural Language Processing Software Services for Cancer Registrar Case Abstraction 91%
- Automated Identification of Patients with Immune-related Adverse Events from Clinical Notes using Word embedding and Machine Learning 90%
- Large language models to help appeal denied radiotherapy services 90%
Similar papers in this journal
- Prognostic pan-cancer and single-cancer models: A large-scale analysis using a real-world clinico-genomic database 93%
- Systematic identification of pan-cancer single-gene expression biomarkers in drug high-throughput screens 91%
- Using distant supervision to augment manually annotated data for relation extraction 90%
Similar papers in this journal
- Interpretable Fine-tuned Large Language Models Facilitate Making Genetic Test Decisions for Rare Diseases 93%
- High-Sensitivity Pan-Cancer AI Assessment of Lymph Node Metastasis via Uncertainty Quantification 93%
- DeePaN: A deep patient graph convolutional network integrating clinico-genomic evidence to stratify lung cancers benefiting from immunotherapy 92%
Similar papers in this journal
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.