Back

Probing the Surgical Competence of LLMs: A global health study leveraging AfriMedQA benchmarks

2025-10-07 surgery Title + abstract only
View on medRxiv
Show abstract

Global surgical care faces a severe workforce shortage, with more than 1.2 million additional specialists needed by 2030, particularly in low- and middle-income countries (LMICs). Large language models (LLMs) have demonstrated impressive medical reasoning on standardized exams, but their safety, reliability, and specialty-specific performance--especially in procedural fields such as surgery--remain uncertain. Here we evaluate over 40 state-of-the-art LLMs on 3,900 expert-authored multiple-choice...

Predicted journal destinations

1
npj Digital Medicine
85 training papers
#1 24.5%
2
PLOS ONE
1737 training papers
Top 34% 19.5%
3
Scientific Reports
701 training papers
Top 12% 12.1%
4
PLOS Digital Health
88 training papers
Top 4% 6.1%
5
BMC Medical Informatics and Decision Making
36 training papers
Top 5% 2.1%
6
BMJ Open
553 training papers
Top 46% 2.1%
7
Journal of Medical Internet Research
81 training papers
Top 8% 2.1%
8
Computers in Biology and Medicine
39 training papers
Top 4% 1.7%
9
Frontiers in Public Health
135 training papers
Top 18% 1.7%
10
Nature Medicine
88 training papers
Top 6% 1.5%
11
PLOS Computational Biology
141 training papers
Top 9% 1.5%
12
eLife
262 training papers
Top 36% 1.3%
13
PLOS Global Public Health
287 training papers
Top 26% 1.3%
14
JAMA Network Open
125 training papers
Top 22% 1.1%
15
Nature Communications
483 training papers
Top 52% 1.0%
16
Journal of the American Medical Informatics Association
53 training papers
Top 6% 0.8%
17
Human Brain Mapping
53 training papers
Top 10% 0.8%
18
Scientific Data
30 training papers
Top 4% 0.8%
19
Journal of Clinical Medicine
77 training papers
Top 21% 0.8%
20
Heliyon
57 training papers
Top 20% 0.5%
21
Journal of Biomedical Informatics
37 training papers
Top 6% 0.5%
22
Trials
24 training papers
Top 5% 0.5%
23
Frontiers in Medicine
99 training papers
Top 30% 0.5%
24
BMC Medicine
155 training papers
Top 42% 0.5%
25
Biology Methods and Protocols
19 training papers
Top 2% 0.5%
26
eBioMedicine
82 training papers
Top 17% 0.5%