Clinical evaluation of a natural language processing system for assisting structured diagnosis recording at the point of care: MiADE (Medical Information AI Data Extractor)

BackgroundStructured recording of key information such as diagnoses is essential for safe, efficient patient care, but is currently done incompletely because it is time consuming for clinicians. We developed a natural language processing system called MiADE integrated with the Epic electronic health record to provide suggestions for structured diagnosis entries at the point of care. ObjectivesTo evaluate the usability, usefulness and impact of MiADE, and identify recommendations for systems to improve point of care structured data recording. MethodsMixed methods evaluation of the implementation of MiADE, with surveys, interviews and observed outpatient consultations. The number of structured diagnoses recorded per outpatient encounter was compared before and after MiADE, and completeness of inpatient problem lists was evaluated using the billing diagnoses as a gold standard. Results85 clinicians consented to the study and were provided access to MiADE, and 24 used MiADE to receive structured data suggestions during the study period. Baseline survey data and observations showed wide variation in structured data recording despite clinicians considering it to be important. Half of post-implementation survey respondents considered MiADE to be very or moderately useful. Among outpatient users there was a 36% increase in the number of diagnoses recorded per encounter, but no improvement was seen in the inpatient setting. ConclusionsNatural language processing using MiADE has the potential to improve structured data recording, but further development and better clinician engagement are needed in order to maximise its impact. SUMMARY BOXO_ST_ABSWhat is already known on this topicC_ST_ABSO_LIDespite the benefits of structured recording of healthcare data for individual care and research, much of the key information in electronic health records (such as diagnoses) is only recorded in free text C_LIO_LIStructured data entry in electronic health record systems can be time-consuming and cumbersome for clinicians C_LI What this study addsO_LIEmbedding natural language processing within the electronic health record to suggest structured data entries was reported by clinicians to be useful, and increased the recording of outpatient diagnoses C_LIO_LIClinician engagement was challenging, and overall usage of structured data remained suboptimal C_LI How this study might affect research, practice or policyO_LIUsability of electronic health record systems needs to improve to enable clinicians to record high quality data without impeding their workflow C_LIO_LINatural language processing embedded within electronic health records may improve ease of use for data entry, but high-level clinician buy-in is also needed to influence professional documentation practice C_LI