Development and comparison of natural language processing models for abdominal aortic aneurysm repair identification and classification using unstructured electronic health records
Thompson, D. C.; Mofidi, R.
Show abstract
BackgroundPatient identification for national registries often relies upon clinician recognition of cases or retrospective searches using potentially inaccurate clinical codes, potentially leading to incomplete data capture and inefficiencies. Natural Language Processing (NLP) offers a promising solution by automating analysis of electronic health records (EHRs). This study aimed to develop NLP models for identifying and classifying abdominal aortic aneurysm (AAA) repairs from unstructured EHRs, demonstrating proof-of-concept for automated patient identification in registries like the National Vascular Registry. MethodUsing the MIMIC-IV-Note dataset, a multi-tiered approach was developed to identify vascular patients (Task 1), AAA repairs (Task 2), and classify repairs as primary or revision (Task 3). Four NLP models were trained and evaluated using 4,870 annotated records: scispaCy, BERT-base, Bio-clinicalBERT, and a scispaCy/Bio-clinicalBERT ensemble. Models were compared using accuracy, precision, recall, F1-score, and area under the receiver operating characteristic curve. ResultsThe scispaCy model demonstrated the fastest training (2 mins/epoch) and inference times (2.87 samples/sec). For Task 1, scispaCy and ensemble models achieved the highest accuracy (0.97). In Task 2, all models performed exceptionally well, with ensemble, scispaCy, and Bio-clinicalBERT models achieving 0.99 accuracy and 1.00 AUC. For Task 3, Bio-clinicalBERT and the ensemble model achieved an AUC of 1.00, with Bio-clinicalBERT displaying the best overall accuracy (0.98). ConclusionThis study demonstrates that NLP models can accurately identify and classify AAA repair cases from unstructured EHRs, suggesting significant potential for automating patient identification in vascular surgery and other medical registries, reducing administrative burden and improving data capture for audit and research.
Matching journals
The top 5 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- From theoretical models to practical deployment: A perspective and case study of opportunities and challenges in AI-driven healthcare research for low-income settings 94%
- Cardiology Knowledge Assessment of Retrieval-Augmented Open versus Proprietary Large Language Models 93%
- Designing a computer-assisted diagnosis system for cardiomegaly detection and radiology report generation 93%
Similar papers in this journal
- Development and Validation of ‘Patient Optimizer’ (POP) Algorithms for Predicting Surgical Risk with Machine Learning 95%
- Optimized Feature Selection and Advanced Machine Learning for Stroke Risk Prediction in Revascularized Coronary Artery Disease Patients 94%
- Evaluating Semantic Similarity Methods for Comparison of Text-derived Phenotype Profiles 93%
Similar papers in this journal
- Developing A Deep Learning Natural Language Processing Algorithm For Automated Reporting Of Adverse Drug Reactions 94%
- EHR-QC: A streamlined pipeline for automated electronic health records standardisation and preprocessing to predict clinical outcomes 93%
- A Deep Learning Approach for Transgender and Gender Diverse Patient Identification in Electronic Health Records 93%
Similar papers in this journal
- Automated stratification of trauma injury severity across multiple body regions using multi-modal, multi-class machine learning models 95%
- A Comparative Analysis of Privacy-Preserving Large Language Models For Automated Echocardiography Report Analysis 94%
- Natural language inference for clinical registry curation 94%
Similar papers in this journal
- A standardized analytics pipeline for reliable and rapid development and validation of prediction models using observational health data 93%
- SPELL-LLMs: A Scalable and Privacy-Compliant NLP Pipeline Using Locally Hosted Large Language Models for Clinical Information Extraction 92%
- Predicting gene and protein expression levels from DNA and protein sequences with Perceiver 91%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.