Back

Performance of Google NotebookLM for AI-assisted data extraction and consensus statement generation in a heterogenous systematic review on inflammatory bowel disease, obesity, and cardiometabolic comorbidities: A Methodological Report

Samaan, S.; Devi, J.; Vincent, M.; Coombs, S.; Sehgal, P.; Mouhamed, M.; Rai, V.; Johnson, A. M.; Yarur, A. J.; Barnes, E. L.; Deepak, P.

2026-06-26 gastroenterology
10.64898/2026.06.16.26355773 medRxiv
Show abstract

Background: Large language models (LLMs) offer promise for systematic review data extraction, but performance in complex multidisciplinary domains and utility for clinical statement generation remain insufficiently described. Objectives: To evaluate Google NotebookLM for AI-assisted data extraction and RAND/UCLA consensus statement generation in a systematic review of IBD, obesity, and cardiometabolic comorbidities. Methods: Studies were organized into domain-specific notebooks; structured prompts generated standardized evidence tables. Two independent reviewers validated outputs against full-text articles using a four-category error classification. Cell-level accuracy and critical accuracy (cells free of major factual errors) were the primary metrics; workflow time was compared against a published conventional extraction benchmark. Concordance between AI-generated and expert-finalized statements was assessed. Results: Across 57 articles, 1,710 data cells were extracted; 151 (8.83%) were flagged, yielding 91.17% cell-level accuracy. Major factual errors occurred in only 4 cells (0.23%), for a critical accuracy of 99.77%. Most errors were minor omissions (59.6%) or incomplete extractions (30.5%); domain error rates ranged from 7.08% to 11.33%. The pipeline required 17.7 versus a projected 165.1 person-hours (89.3% reduction). PICO-structured prompting generated 70 candidate statements; 58 of 112 finalized panel statements (51.8%) were AI-derived, and 85.7% were retained in the finalized set. Conclusion: Google NotebookLM demonstrates feasibility as a primary extraction and synthesis tool in a multidisciplinary systematic review, with extractive incompleteness as the principal limitation and substantial time savings over conventional approaches. Its novel application to RAND/UCLA consensus statement generation extends AI-assisted evidence synthesis to clinical consensus generation workflow.

Matching journals

The top 6 journals account for 50% of the predicted probability mass.

1
Contemporary Clinical Trials Communications
11 papers in training set
Top 0.1%
15.8%
2
Research Synthesis Methods
20 papers in training set
Top 0.1%
10.2%
3
BMJ Open
601 papers in training set
Top 2%
8.2%
4
PLOS ONE
5266 papers in training set
Top 23%
7.6%
5
PLOS Digital Health
106 papers in training set
Top 1%
5.4%
6
Journal of Clinical Epidemiology
31 papers in training set
Top 0.2%
3.7%
50% of probability mass above
7
American Journal of Gastroenterology
17 papers in training set
Top 0.2%
2.9%
8
BMC Medicine
176 papers in training set
Top 1%
2.8%
9
Systematic Reviews
15 papers in training set
Top 0.1%
2.8%
10
Journal of the American Medical Informatics Association
71 papers in training set
Top 1%
2.6%
11
Scientific Data
209 papers in training set
Top 1%
1.8%
12
PLOS Medicine
110 papers in training set
Top 2%
1.8%
13
Journal of Medical Internet Research
87 papers in training set
Top 1%
1.8%
14
Scientific Reports
3612 papers in training set
Top 58%
1.6%
15
F1000Research
88 papers in training set
Top 2%
1.6%
16
Nature Communications
5641 papers in training set
Top 47%
1.5%
17
BMC Medical Research Methodology
47 papers in training set
Top 0.8%
1.4%
18
Frontiers in Pharmacology
111 papers in training set
Top 2%
1.2%
19
PeerJ
308 papers in training set
Top 7%
1.2%
20
JAMIA Open
42 papers in training set
Top 1%
1.2%
21
Value in Health
11 papers in training set
Top 0.2%
1.2%
22
Clinical and Translational Science
22 papers in training set
Top 0.4%
1.2%
23
Clinical Trials
11 papers in training set
Top 0.3%
0.9%
24
Pharmacoepidemiology and Drug Safety
18 papers in training set
Top 0.5%
0.9%
25
iScience
1154 papers in training set
Top 38%
0.6%
26
Medicine
31 papers in training set
Top 2%
0.6%
27
PLOS Biology
486 papers in training set
Top 13%
0.6%
28
Frontiers in Psychology
56 papers in training set
Top 2%
0.6%
29
Communications Medicine
113 papers in training set
Top 5%
0.6%
30
Frontiers in Medicine
120 papers in training set
Top 5%
0.6%