Back

Automating Academic Document Analysis with ChatGPT: A Mendeley Case

Abuella, M.

2024-03-21 scientific communication and education
10.1101/2024.03.18.585620 bioRxiv
Show abstract

The management and organization of a large collection of academic documents is an important part of scientific research. This study explores the use of ChatGPT, a large language model from OpenAI, to extract insights from a large collection of academic documents stored in Mendeley. The study found that ChatGPT can be used to generate insightful graphs, concise summaries, and other tasks tailored to user needs. The study also demonstrated that ChatGPT can be used to analyze a large number of publications in PDF format. This suggests that ChatGPT could be a valuable tool for researchers who want to save time and effort by automating the analysis of their data. The GitHub repository for the source code and the output of this study is available at: https://github.com/MohamedAbuella/Analysis_Mendeley.

Matching journals

The top 5 journals account for 50% of the predicted probability mass.

1
PLOS ONE
● 5266 papers in training set
Top 10%
18.5%
Similar papers in this journal
2
Gigabyte
● 62 papers in training set
Top 0.1%
11.9%
Similar papers in this journal
3
Heliyon
○ 152 papers in training set
Top 0.1%
7.9%
Similar papers in this journal
4
GigaScience
◐ 212 papers in training set
Top 0.3%
6.7%
Similar papers in this journal
5
BMC Research Notes
○ 33 papers in training set
Top 0.1%
6.2%
Similar papers in this journal
50% of probability mass above
6
Bioinformatics Advances
◐ 203 papers in training set
Top 1%
4.0%
7
Ecological Informatics
○ 33 papers in training set
Top 0.3%
2.4%
8
Bioinformatics
◐ 1204 papers in training set
Top 6%
2.4%
9
PLOS Digital Health
● 106 papers in training set
Top 2%
2.4%
10
Frontiers in Artificial Intelligence
○ 20 papers in training set
Top 0.2%
2.4%
11
Sustainability
○ 10 papers in training set
Top 0.1%
2.1%
12
F1000Research
○ 88 papers in training set
Top 1%
2.1%
13
Scientific Reports
○ 3612 papers in training set
Top 52%
1.7%
14
Ecology and Evolution
○ 267 papers in training set
Top 4%
1.7%
15
Frontiers in Psychology
○ 56 papers in training set
Top 0.7%
1.7%
16
BioData Mining
○ 22 papers in training set
Top 0.3%
1.4%
17
PLOS Computational Biology
● 1863 papers in training set
Top 17%
1.1%
18
Journal of Biomedical Informatics
○ 47 papers in training set
Top 1%
1.1%
19
BMC Bioinformatics
○ 457 papers in training set
Top 5%
1.1%
20
IEEE Journal of Biomedical and Health Informatics
● 37 papers in training set
Top 0.9%
1.1%
21
Database
◐ 61 papers in training set
Top 0.6%
1.1%
22
BMJ Health & Care Informatics
● 15 papers in training set
Top 0.8%
1.1%
23
Wellcome Open Research
○ 67 papers in training set
Top 0.9%
1.1%
24
Frontiers in Bioinformatics
○ 49 papers in training set
Top 1%
1.0%
25
Frontiers in Microbiology
○ 427 papers in training set
Top 8%
0.8%
26
Brain Sciences
○ 55 papers in training set
Top 2%
0.6%
27
eneuro
● 439 papers in training set
Top 8%
0.6%

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.