Two-phase Framework Clinical Question-Answering; A case-study of Autocorrection for Guideline-concordance
Tariq, A.; Yu, N.; Patel, B.; Banerjee, I.
Show abstract
Use of large language models for generative tasks in critical domains like medicine is fraught with challenges like hallucination. In the domain of medicine, hallucination may take a unique shape where the LLM-generated language is not inaccurate but the suggested treatment or medication has now been discontinued in a specific context. Reinforcement learning based solutions for building reliable LLM-based frameworks are limited by the fact that the reinforcement is typically focused on only identifying the mistake; correcting the mistake is left up to the primary LLM. We propose an innovative solution where a two-phase question answering framework composed of two LLMs is designed such that one LLM learns to generate answers while the other learns to correct any mistakes in the answer generated by the first model. We experimented with the particular domain of prostate cancer and LLMs designed for various domains and showed that domain-specific LLMs outperform generic or wide-domain LLMs.
Matching journals
The top 6 journals account for 50% of the predicted probability mass.
Similar papers in this journal
Similar papers in this journal
- A Study of Calibration as a Measurement of Trustworthiness of Large Language Models in Biomedical Research 96%
- Natural Language Processing for Automated Annotation of Medication Mentions in Primary Care Visit Conversations 94%
- Automating Evaluation of LLM-generated Responses to Patient Questions about Rare Diseases 94%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.