Does Recording Hardware Matter for Clinical Speech Recognition Evaluating ASR Performance Across Consumer Devices
Tran, B. D.; Hu, D.; Kim, S.; Guo, Y.; Mangu, R.; Reynolds, T. L.; Lafata, J. E.; Tai-Seale, M.; Zheng, K.
Show abstract
Ambient clinical intelligence (ACI) systems use automatic speech recognition (ASR) to capture patient-provider conversations for downstream clinical documentation. However, many ASR evaluations are conducted under controlled conditions using specialized hardware. We evaluated how recording devices influence transcription performance of contemporary ASR engines applied to clinical dialogue. Thirty-five primary care encounters were re-enacted from transcribed conversations and recorded using five devices simultaneously: smartphone, laptop microphone, portable recorder, clip-on microphone, and a desktop microphone. Six ASR engines were evaluated using word error rate (WER), clinical concept extraction precision and recall, and sentence-level semantic similarity. Median WER ranged from 16.7% to 20.7% across engines. Engine choice produced larger variation in transcription performance than recording device, although device-related differences were statistically significant. Overall, contemporary ASR engines demonstrated relative robustness to consumer-grade recording hardware, suggesting that model selection may have greater impact on transcription performance than recording device configuration in real-world ACI deployments.
Matching journals
The top 5 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Natural language processing to evaluate texting conversations between patients and healthcare providers during COVID-19 Home-Based Care in Rwanda at scale 93%
- Evaluating Anti-LGBTQIA+ Medical Bias in Large Language Models 93%
- Ethical review of clinical research with generative AI: Evaluating ChatGPT’s accuracy and reproducibility 92%
Similar papers in this journal
- A Framework to Assess Clinical Safety and Hallucination Rates of LLMs for Medical Text Summarisation 95%
- From Tool to Teammate: A Randomized Controlled Trial of Clinician-AI Collaborative Workflows for Diagnosis 92%
- Evaluating large language model workflows in clinical decision support: referral, triage, and diagnosis 91%
Similar papers in this journal
Similar papers in this journal
- Listening to mental health crisis needs at scale: using Natural Language Processing to understand and evaluate a mental health crisis text messaging service 92%
- Large Language Models in Real-World Clinical Workflows: A Systematic Review of Applications and Implementation 92%
- AI chatbots not yet ready for clinical use 89%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.