Back

Advancing Question-Answering in Ophthalmology with Retrieval Augmented Generations (RAG): Benchmarking Open-source and Proprietary Large Language Models

Nguyen, Q.; Nguyen, D.-A.; Dang, K.; Liu, S.; Wang, S. Y.; Nguyen, K.; Woof, W. A.; Thomas, P.; Patel, P. J.; Balaskas, K.; Thygesen, J. H.; Wu, H.; Pontikos, N.

2024-11-19 ophthalmology
10.1101/2024.11.18.24317510 medRxiv
Show abstract

Withdrawal statementThe authors have withdrawn this manuscript due to the inadvertent use of copyrighted material without necessary permissions from the rights holder. Therefore, the authors do not wish this work to be cited as reference for the project. If you have any questions, please contact the corresponding author.

Published in Translational Vision Science & Technology · not in our set (fewer than 10 published preprints to learn from) · training set

Matching journals

The top 6 journals account for 50% of the predicted probability mass.

1
PLOS Digital Health
● 106 papers in training set
Top 0.4%
14.9%
Similar papers in this journal
2
Journal of Medical Internet Research
◐ 87 papers in training set
Top 0.1%
12.0%
Similar papers in this journal
3
Computers in Biology and Medicine
○ 128 papers in training set
Top 0.3%
7.3%
Similar papers in this journal
4
PLOS ONE
● 5266 papers in training set
Top 27%
5.5%
Similar papers in this journal
5
Scientific Reports
○ 3612 papers in training set
Top 15%
5.5%
Similar papers in this journal
6
Bioinformatics
◐ 1204 papers in training set
Top 4%
4.9%
50% of probability mass above
7
npj Digital Medicine
○ 118 papers in training set
Top 1%
3.2%
8
Eye
○ 11 papers in training set
Top 0.2%
2.4%
9
DIGITAL HEALTH
○ 17 papers in training set
Top 0.2%
2.4%
10
Journal of Vision
● 110 papers in training set
Top 0.4%
2.1%
11
Journal of the American Medical Informatics Association
◐ 71 papers in training set
Top 1%
2.1%
12
Bioengineering
○ 29 papers in training set
Top 0.2%
2.1%
13
Brain Sciences
○ 55 papers in training set
Top 0.6%
1.9%
14
F1000Research
○ 88 papers in training set
Top 1%
1.7%
15
Bioinformatics Advances
◐ 203 papers in training set
Top 3%
1.7%
16
GigaScience
◐ 212 papers in training set
Top 2%
1.7%
17
Journal of Neural Engineering
○ 221 papers in training set
Top 2%
1.5%
18
Medical Image Analysis
○ 35 papers in training set
Top 0.5%
1.3%
19
British Journal of Ophthalmology
● 14 papers in training set
Top 0.2%
1.1%
20
Translational Vision Science & Technology
● 39 papers in training set
Top 0.4%
1.1%
21
Ophthalmology Science
○ 22 papers in training set
Top 0.3%
1.1%
22
BMC Bioinformatics
○ 457 papers in training set
Top 5%
1.1%
23
BMC Medical Informatics and Decision Making
○ 43 papers in training set
Top 2%
1.0%
24
Frontiers in Bioinformatics
○ 49 papers in training set
Top 1%
1.0%
25
Patterns
○ 78 papers in training set
Top 2%
0.9%
26
BioData Mining
○ 22 papers in training set
Top 0.6%
0.8%
27
Frontiers in Neuroscience
○ 256 papers in training set
Top 6%
0.8%
28
JAMIA Open
◐ 42 papers in training set
Top 2%
0.8%
29
BMC Genomics
○ 406 papers in training set
Top 8%
0.8%
30
Journal of Biomedical Informatics
○ 47 papers in training set
Top 1%
0.6%

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.