Back

Comparing AI and Human Coding of NIH Grant Abstracts to Identify Innovations in Opioid Addiction Treatment

2026-02-17 health informatics Title + abstract only
View on medRxiv
Show abstract

Large language models (LLMs) are increasingly used for qualitative analysis in substance use research, yet their performance relative to human coders remains underexplored. This study compares ChatGPT-4.0 with human coders in identifying and describing the core innovation of NIH grants focused on reducing opioid overdose. A total of 118 NIH HEAL Initiative grant abstracts were independently coded by ChatGPT and humans to generate innovation descriptions, which were then evaluated by both human r...

Predicted journal destinations

1
PLOS ONE
1737 training papers
Top 48% 14.1%
2
Journal of the American Medical Informatics Association
53 training papers
Top 2% 9.0%
3
Journal of Medical Internet Research
81 training papers
Top 2% 7.5%
4
JAMIA Open
35 training papers
Top 2% 6.2%
5
BMJ Open
553 training papers
Top 27% 6.2%
6
Drug and Alcohol Dependence
21 training papers
Top 0.5% 5.7%
7
PLOS Digital Health
88 training papers
Top 5% 5.7%
8
BMC Medical Informatics and Decision Making
36 training papers
Top 4% 4.0%
9
Frontiers in Psychiatry
56 training papers
Top 4% 3.7%
10
Journal of Biomedical Informatics
37 training papers
Top 4% 3.0%
11
International Journal of Medical Informatics
25 training papers
Top 2% 2.9%
12
BMC Medical Research Methodology
41 training papers
Top 0.9% 2.7%
13
JAMA Network Open
125 training papers
Top 6% 2.7%
14
JMIR Formative Research
31 training papers
Top 1% 1.9%
15
Scientific Reports
701 training papers
Top 82% 1.9%
16
JMIR Medical Informatics
16 training papers
Top 3% 1.9%
17
Addiction
24 training papers
Top 1% 1.9%
18
npj Digital Medicine
85 training papers
Top 14% 1.3%
19
Frontiers in Digital Health
18 training papers
Top 1% 1.3%
20
International Journal of Environmental Research and Public Health
116 training papers
Top 23% 1.1%
21
Journal of Clinical Epidemiology
29 training papers
Top 2% 1.0%
22
Frontiers in Public Health
135 training papers
Top 29% 0.9%
23
JMIR Public Health and Surveillance
45 training papers
Top 6% 0.9%
24
Patterns
15 training papers
Top 4% 0.7%
25
Psychiatry Research
33 training papers
Top 6% 0.7%
26
BMC Health Services Research
43 training papers
Top 6% 0.7%