Beyond the hype: large language models propagate race-based medicine
Omiye, J. A.; Lester, J.; Spichak, S.; Rotemberg, V.; Daneshjou, R.
Show abstract
ImportanceLarge language models (LLMs) are being integrated into healthcare systems; but these models recapitulate harmful, race-based medicine. ObjectiveThe objective of this study is to assess whether four commercially available large language models (LLMs) propagate harmful, inaccurate, race-based content when responding to eight different scenarios that historically included race-based medicine or widespread misconceptions around race. Evidence ReviewQuestions were derived from discussion among 4 physician experts and prior work on race-based medical misconceptions of medical trainees. FindingsWe assessed four large language models with eight different questions that were interrogated five times each with a total of forty responses per a model. All models had examples of perpetuating race-based medicine in their responses. Models were not always consistent in their responses when asked the same question repeatedly. Conclusions and RelevanceLLMs are being proposed for use in the healthcare setting, with some models already connecting to electronic health record systems. However, this study shows that based on our findings, these LLMs could potentially cause harm by perpetuating debunked, racist concepts.
Matching journals
The top 5 journals account for 50% of the predicted probability mass.
Similar papers in this journal
Similar papers in this journal
- Cracking the Code: A Scoping Review to Unite Disciplines in Tackling Legal Issues in Health Artificial Intelligence 91%
- Measures of socioeconomic advantage are not independent predictors of support for healthcare AI: subgroup analysis of a national Australian survey 89%
- Natural Language Word-Embeddings as a glimpse into healthcare at the End Of Life 89%
Similar papers in this journal
- Performance of Advanced Large Language Models (GPT-4o, GPT-4, Gemini 1.5 Pro, Claude 3 Opus) on Japanese Medical Licensing Examination: A Comparative Study 90%
- A Deep Learning Method to Detect Opioid Prescription and Opioid Use Disorder from Electronic Health Records 89%
- Synthetic Data Generation in Healthcare: A Scoping Review of reviews on domains, motivations, and future applications 89%
Similar papers in this journal
- Determining prescriptions in electronic health care (EHR) data: methods for development of standardised, reproducible drug codelists 90%
- Development and Application of Pharmacological Statin-Associated Muscle Symptoms Phenotyping Algorithms Using Structured and Unstructured Electronic Health Records Data 90%
- Algorithmic Individual Fairness and Healthcare: A Scoping Review 89%
Similar papers in this journal
- COVID-19 outcomes, risk factors and associations by race: a comprehensive analysis using electronic health records data in Michigan Medicine 93%
- Low adherence to existing model reporting guidelines by commonly used clinical prediction models 92%
- Characterizing Potential Conflicts of Interest Among UpToDate and DynaMed Content Contributors 89%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.