Usability and workflow implications of an electronic health record-integrated tool for automated phenotyping: Formative evaluation of Pheno+

ObjectiveTo evaluate usability and workflow implications of Pheno+, an electronic health record- (EHR) embedded natural language processing tool that extracts patient phenotypes and maps them to Human Phenotype Ontology (HPO) terms. Materials and MethodsThis study performed a mixed-methods formative evaluation of Pheno+ in the context of rapid genome sequencing in the neonatal intensive care unit. Two rounds of think-aloud interviews and surveys were conducted with neonatologists, advanced practice providers, medical geneticists, genetic counselors, non-genetics pediatric subspecialists and laboratory molecular geneticists. Usability was assessed using the System Usability Scale (SUS). Structured data were analyzed descriptively. Qualitative data underwent hybrid deductive-inductive thematic analysis. ResultsTwenty-eight interviewees participated in two rounds of data collection. Users retained an average of 24.2 HPO terms from an average of 45.6 terms per patient. Mean time curating terms was 8:59 per patient. Final SUS score was 76 (above scale mean). Qualitative themes included: burden of curating terms; interpretation of phenotypes; adding information to phenotypes; role of genetics expertise; and tool uses beyond the original core use case for Pheno+. DiscussionPheno+ demonstrated positive usability but required curation effort to reduce erroneous terms. Participant responses indicate that effective use may depend on knowledge of HPO structure and laboratory workflows. Automated phenotyping seems likely to reduce net clinician workload, while also reshaping clinical responsibilities, expertise requirements, and communication in genetic testing. ConclusionTools like Pheno+ can support scalable genomic medicine. Design and implementation may benefit from addressing curation burden, phenotype contextualization, expertise requirements, and other workflow needs.