Back

AutoReporter: Development and validation of an LLM-based tool for automated research reporting guideline assessment

2025-04-20 health informatics Title + abstract only
View on medRxiv
Show abstract

ObjectiveTo develop AutoReporter, a large-language-model system that automates evaluation of adherence to research reporting guidelines. Materials and MethodsEight prompt-engineering and retrieval strategies coupled with reasoning and general-purpose LLMs were benchmarked on the SPIRIT-CONSORT-TM corpus. The top-performing approach, AutoReporter, was validated on BenchReport, a novel benchmark dataset of expert-rated reporting guideline assessments from 10 systematic reviews. ResultsAutoReport...

Predicted journal destinations

1
Journal of the American Medical Informatics Association
53 training papers
#1 28.8%
2
JAMIA Open
35 training papers
Top 1% 6.5%
3
Journal of Medical Internet Research
81 training papers
Top 3% 6.0%
4
Journal of Biomedical Informatics
37 training papers
Top 2% 6.0%
5
PLOS Digital Health
88 training papers
Top 5% 5.1%
6
Journal of Clinical Epidemiology
29 training papers
Top 0.3% 5.1%
7
BMJ Open
553 training papers
Top 37% 4.2%
8
BMC Medical Informatics and Decision Making
36 training papers
Top 4% 4.2%
9
BMC Medical Research Methodology
41 training papers
Top 0.4% 4.2%
10
npj Digital Medicine
85 training papers
Top 7% 3.9%
11
PLOS ONE
1737 training papers
Top 89% 3.2%
12
Scientific Reports
701 training papers
Top 80% 2.0%
13
JMIR Medical Informatics
16 training papers
Top 2% 2.0%
14
International Journal of Medical Informatics
25 training papers
Top 3% 2.0%
15
Research Synthesis Methods
17 training papers
Top 0.4% 2.0%
16
Nature Communications
483 training papers
Top 42% 1.6%
17
BMC Medicine
155 training papers
Top 23% 1.2%
18
JMIR Formative Research
31 training papers
Top 7% 0.7%
19
The Lancet Digital Health
25 training papers
Top 2% 0.7%
20
Computers in Biology and Medicine
39 training papers
Top 10% 0.7%
21
JAMA Network Open
125 training papers
Top 30% 0.7%
22
Scientific Data
30 training papers
Top 4% 0.7%
23
Frontiers in Digital Health
18 training papers
Top 4% 0.7%
24
Patterns
15 training papers
Top 5% 0.5%