Performance of an Ambient Generative AI Documentation Tool in a Linguistically Diverse Clinical Setting

Ambient artificial intelligence scribes are being increasingly used in healthcare to improve efficiency and reduce provider clinical documentation burden, yet their performance across linguistically diverse patient populations is not well characterized. We conducted a retrospective analysis of 54,160 outpatient encounters within a U.S. safety net health system to evaluate the performance of an artificial intelligence documentation tool in English and non-English clinical encounters, and in encounters where an interpreter or bilingual provider was present. Documentation performance was measured by the percentage of words in the final note that were generated by the ambient AI documentation tool and not edited by the provider. Associations between language factors and documentation performance were measured using Generalized Estimating Equations with exchangeable correlation structures to account for clustering of multiple encounters within unique patients. Univariable models were fitted to estimate the odds of adequate performance by language and interpreter modality, and a multivariable interaction model was used to evaluate within-language differences between bilingual providers and interpreter-mediated encounters. Non-English encounters were 21% to 25% less likely than English encounters to achieve the same performance threshold. There was no significant difference in generative documentation performance between interpreter-mediated and bilingual provider encounters. These findings underscore the importance of equity-focused evaluation and multilingual model refinement to ensure that artificial intelligence documentation benefits are distributed fairly across diverse patient populations.