Back

Utilizing Large Language Models to Simplify Radiology Reports: a comparative analysis of ChatGPT-3.5, ChatGPT-4.0, Google Bard, and Microsoft Bing

Doshi, R. H.; Amin, K.; Khosla, P.; Bajaj, S.; Chheang, S.; Forman, H.

2023-06-05 radiology and imaging
10.1101/2023.06.04.23290786 medRxiv
Show abstract

This paper investigates the application of Large Language Models (LLMs), specifically OpenAIs ChatGPT3.5, ChatGPT4.0, Google Bard, and Microsoft Bing, in simplifying radiology reports, thus potentially enhancing patient understanding. We examined 254 anonymized radiology reports from diverse examination types and used three different prompts to guide the LLMs simplification processes. The resulting simplified reports were evaluated using four established readability indices. All LLMs significantly simplified the reports, but performance varied based on the prompt used and the specific model. The ChatGPT models performed best when additional context was provided (i.e., specifying user as a patient or requesting simplification at the 7th grade level). Our findings suggest that LLMs can effectively simplify radiology reports, although improvements are needed to ensure accurate clinical representation and optimal readability. These models have the potential to improve patient health literacy, patient-provider communication, and ultimately, health outcomes.

Matching journals

The top 6 journals account for 50% of the predicted probability mass.

1
Journal of the American Medical Informatics Association
◐ 71 papers in training set
Top 0.3%
12.3%
Similar papers in this journal
2
PLOS Digital Health
● 106 papers in training set
Top 0.6%
11.0%
Similar papers in this journal
3
European Radiology
○ 15 papers in training set
Top 0.1%
10.1%
Similar papers in this journal
4
npj Digital Medicine
○ 118 papers in training set
Top 0.7%
8.1%
Similar papers in this journal
5
Scientific Reports
○ 3612 papers in training set
Top 11%
6.5%
Similar papers in this journal
6
PLOS ONE
● 5266 papers in training set
Top 26%
5.7%
50% of probability mass above
7
JCO Clinical Cancer Informatics
● 22 papers in training set
Top 0.2%
4.5%
8
The Lancet Digital Health
○ 25 papers in training set
Top 0.1%
3.4%
9
Neurosurgery
○ 11 papers in training set
Top 0.1%
3.3%
10
Cureus
○ 68 papers in training set
Top 2%
2.7%
11
JAMA Network Open
● 130 papers in training set
Top 1%
2.5%
12
Medical Physics
○ 14 papers in training set
Top 0.3%
2.2%
13
JMIR Medical Informatics
◐ 18 papers in training set
Top 0.4%
1.5%
14
International Journal of Medical Informatics
○ 26 papers in training set
Top 0.8%
1.5%
15
BMJ Open
● 601 papers in training set
Top 10%
1.4%
16
BMJ Health & Care Informatics
● 15 papers in training set
Top 0.7%
1.4%
17
Diagnostics
○ 50 papers in training set
Top 1%
1.2%
18
GigaScience
◐ 212 papers in training set
Top 3%
1.1%
19
Journal of Medical Imaging
● 11 papers in training set
Top 0.4%
1.0%
20
Computers in Biology and Medicine
○ 128 papers in training set
Top 4%
0.9%
21
Frontiers in Digital Health
○ 24 papers in training set
Top 1%
0.9%
22
JMIRx Med
◐ 32 papers in training set
Top 2%
0.9%
23
Medicine
○ 31 papers in training set
Top 2%
0.9%
24
PLOS Global Public Health
● 344 papers in training set
Top 8%
0.9%
25
Rheumatology
◐ 24 papers in training set
Top 0.3%
0.6%
26
BMC Cancer
○ 67 papers in training set
Top 2%
0.6%
27
Biomedical Physics & Engineering Express
○ 11 papers in training set
Top 0.4%
0.5%
28
Children
○ 10 papers in training set
Top 0.7%
0.5%

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.