Back

Closing the Pediatric Divide: A Performance Analysis of the GPT-5 Family in Medical Diagnostics

Mondillo, G.; Abbate, F. G.; Masino, M.; Colosimo, S.; Perrotta, A.; Frattolillo, V.

2025-08-29 pediatrics
10.1101/2025.08.28.25334657 medRxiv
Show abstract

AO_SCPLOWBSTRACTC_SCPLOWO_ST_ABSBackgroundC_ST_ABSLarge Language Models (LLMs) have demonstrated significant potential in clinical medicine, but a persistent performance gap exists in the pediatric domain due to its unique complexities. This study provides the first comparative evaluation of the new GPT-5 family (Nano, Mini, and full) to assess the impact of model scale on diagnostic accuracy and this specific adult-pediatric disparity. MethodsA benchmarking study was conducted using 2,000 multiple-choice questions from the MedQA dataset, equally divided between adult (n=1,000) and pediatric (n=1,000) domains. GPT-5, GPT-5 Mini, and GPT-5 Nano were tested via API with standardized parameters (temperature=0, reasoning effort=minimal, verbosity=low, maxtoken=170). Accuracy was calculated and statistically compared across domains for each model. ResultsA clear dose-response relationship was observed between model size and accuracy. GPT-5 Nano exhibited a significant performance gap, with an accuracy of 71.0% in adult medicine versus 55.4% in pediatrics (a 15.6 percentage point difference, p<0.001). GPT-5 Mini substantially narrowed this gap to 5.7 points (81.5% vs. 75.8%, p=0.001). Critically, the full GPT-5 model eliminated the disparity, achieving comparable accuracy in adult medicine (86.3%) and slightly higher accuracy in pediatrics (88.5%) (p=0.138). Performance gains from scaling up were disproportionately larger for the pediatric domain. ConclusionThe GPT-5 family marks a substantial advancement in medical AI. The full-size model not only achieves high diagnostic accuracy but, crucially, overcomes the previously documented performance limitations in pediatrics. This demonstrates that sufficient model scale is vital for mastering the nuances of specialized clinical domains. These findings support a tiered implementation strategy based on task criticality and underscore the need for continued validation in real-world clinical settings.

Matching journals

The top 3 journals account for 50% of the predicted probability mass.

1
PLOS Digital Health
106 papers in training set
Top 0.1%
23.2%
2
GigaScience
212 papers in training set
Top 0.1%
15.6%
3
npj Digital Medicine
118 papers in training set
Top 0.4%
13.1%
50% of probability mass above
4
BioData Mining
22 papers in training set
Top 0.1%
9.2%
5
PLOS ONE
5266 papers in training set
Top 27%
5.7%
6
Communications Medicine
113 papers in training set
Top 1%
2.5%
7
Journal of Clinical Epidemiology
31 papers in training set
Top 0.3%
2.5%
8
Healthcare
17 papers in training set
Top 0.2%
2.2%
9
Scientific Reports
3612 papers in training set
Top 50%
2.0%
10
Medicine
31 papers in training set
Top 0.9%
1.8%
11
Journal of the American Medical Informatics Association
71 papers in training set
Top 1%
1.8%
12
Journal of Medical Internet Research
87 papers in training set
Top 2%
1.6%
13
Cortex
119 papers in training set
Top 1%
1.2%
14
Annals of Internal Medicine
28 papers in training set
Top 0.5%
1.0%
15
Computers in Biology and Medicine
128 papers in training set
Top 4%
0.9%
16
PLOS Global Public Health
344 papers in training set
Top 8%
0.6%
17
JMIRx Med
32 papers in training set
Top 2%
0.6%
18
Children
10 papers in training set
Top 0.7%
0.6%
19
JAMIA Open
42 papers in training set
Top 2%
0.6%
20
JAMA Network Open
130 papers in training set
Top 4%
0.6%
21
IEEE Journal of Biomedical and Health Informatics
37 papers in training set
Top 2%
0.5%
22
BMC Bioinformatics
457 papers in training set
Top 6%
0.5%
23
Computational and Structural Biotechnology Journal
242 papers in training set
Top 9%
0.5%
24
BMC Medical Education
21 papers in training set
Top 0.8%
0.5%