Answering clinicians' questions over trial evidence tables with verifiable, feedback-driven language models
Roy Choudhury, M.; Roy Chowdhury, S.; Sahoo, S.; Khan, M. A.; Khakwani, K. Z. R.; Sonbol, M. B.; Riaz, I.; Gupta, V.
Show abstract
Systematic reviews condense clinical trials into evidence tables, yet clinicians can interrogate these tables only through database queries, and many questions concern attributes that the table does not record, such as a drug's target class or a harmonised endpoint. Here we introduce FD-SCoPE, a language-model framework that answers both kinds of question, exposes the query, the selected trials and the derivation rule behind every answer, and learns from expert corrections. On an oncology evidence table of 159 immune checkpoint inhibitor trial records, FD-SCoPE completed all 140 clinician-style tasks (alternatives, 90.7--97.9%). For questions needing derived attributes it retrieved 99.3% of relevant trial records at a positive predictive value of 89.8% and outperformed four alternative approaches (derived-value F1 77.7% versus 64.8--73.4%). Corrections on 299 questions, simulated from reference answers, raised F1 on 1,201 unseen questions from 77.9% to 84.9%. Language models coupled with executable queries, verified programs and expert feedback can give clinicians auditable access to trial evidence.
Matching journals
The top 4 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Interpretable Fine-tuned Large Language Models Facilitate Making Genetic Test Decisions for Rare Diseases 94%
- Large language models improve transferability of electronic health record-based predictions across countries and coding systems 93%
- Clinically Informed Semi-Supervised Learning Improves Disease Annotation and Equity from Electronic Health Records: A Glaucoma Case Study 92%
Similar papers in this journal
Similar papers in this journal
- Trialstreamer: a living, automatically updated database of clinical trial reports 95%
- Analysis of Eligibility Criteria Clusters Based on Large Language Models for Clinical Trial Design 94%
- Is One Run Enough? Reproducibility of Flagship Large Language Models Across Temperature and Reasoning Settings in Biomedical Text Processing 94%
Similar papers in this journal
- Prognostic pan-cancer and single-cancer models: A large-scale analysis using a real-world clinico-genomic database 92%
- Exploring scalable assessment methods for terminated trials in ClinicalTrials.gov: A cohort analysis of German and Californian trials 92%
- Analysis of clinical trial registry entry histories using the novel R package cthist 92%
Similar papers in this journal
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.