Extracting clinical chief complaints from patient-physician conversations with a non-AI detection model
Kanpa, V. N.; D'Onofrio, D.; Samanta, N.; Tada, M.; Patel, S.
Show abstract
Generative artificial intelligence (GenAI) applications have been at the forefront of clinical documentation assistants, aiming to reduce physician notetaking burden. However, GenAI systems are resource-intensive, and deployment in low-resource healthcare settings can be challenging and cost prohibitive. We present a symbolic reasoning model (SRM) for detecting chief complaints from clinical conversations and evaluate it against two large language models (LLMs), Gemma2-9b and Llama3.3-70B-Versatile. We use a benchmarking dataset of simulated doctor-patient conversations to develop an SRM leveraging proximity-based keywords and Zipfs Law to extract complaints. Three physicians from different hospital systems evaluate the accuracy and clinical utility of the LLMs and SRM. The SRM achieves comparable or superior performance to LLMs across three conversational modalities of varying complexity (cross-modality mean utility score: 5.77 {+/-} 0.78 out of 10). Runtime analyses demonstrated that the SRM executes substantially faster than LLMs, though with higher local RAM usage, contrasted with the cloud-based execution of LLMs. Conversational adjustments by physicians to acknowledge a third-party scribe system substantially improved utility of SRM-detected complaints. These findings support SRM integration as a plausible solution for automating documentation in under-resourced healthcare environments.
Matching journals
The top 5 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Evaluating Anti-LGBTQIA+ Medical Bias in Large Language Models 95%
- From theoretical models to practical deployment: A perspective and case study of opportunities and challenges in AI-driven healthcare research for low-income settings 94%
- Performance of Generative Pretrained Transformer on the National Medical Licensing Examination in Japan 94%
Similar papers in this journal
- Large Language Models in Real-World Clinical Workflows: A Systematic Review of Applications and Implementation 95%
- Development and Validation of a Machine Learning Model Integrated with the Clinical Workflow for Inpatient Discharge Date Prediction 94%
- Listening to mental health crisis needs at scale: using Natural Language Processing to understand and evaluate a mental health crisis text messaging service 94%
Similar papers in this journal
Similar papers in this journal
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.