Back

RefDeduR: A text-normalization and decision-tree aided R package enabling accurate and high-throughput reference deduplication for large datasets

Shen, J.; Ling, F.; Hartmann, E. M.

2022-10-03 scientific communication and education
10.1101/2022.09.29.510210 bioRxiv
Show abstract

As the scientific literature grows exponentially and research becomes increasingly interdisciplinary, accurate and high-throughput reference deduplication is vital in evidence synthesis studies (e.g., systematic reviews, meta-analyses) to ensure the completeness of datasets while reducing the manual screening burden. Existing tools fail to fulfill these emerging needs, as they are often labor-intensive, insufficient in accuracy, and limited to clinical fields. Here, we present RefDeduR, a text-normalization and decision-tree aided R package that enables accurate and high-throughput reference deduplication. We modularize the pipeline into text normalization, three-step exact matching, and two-step fuzzy matching processes. We also introduce a decision-tree algorithm, consider preprints when they co-exist with a peer-reviewed version, and provide actionable recommendations. Therefore, the tool is customizable, accurate, high-throughput, and practical. RefDeduR provides an effective solution to perform reference deduplication and represents a valuable advance in expanding the open-source toolkit to support evidence synthesis research.

Matching journals

The top 5 journals account for 50% of the predicted probability mass.

1
Bioinformatics
◐ 1204 papers in training set
Top 2%
14.7%
Similar papers in this journal
2
eLife
● 5828 papers in training set
Top 7%
12.4%
Similar papers in this journal
3
GigaScience
◐ 212 papers in training set
Top 0.1%
10.4%
Similar papers in this journal
4
PLOS Biology
● 486 papers in training set
Top 0.2%
7.7%
Similar papers in this journal
5
PLOS Computational Biology
● 1863 papers in training set
Top 7%
5.3%
Similar papers in this journal
50% of probability mass above
6
Wellcome Open Research
○ 67 papers in training set
Top 0.1%
5.0%
7
F1000Research
○ 88 papers in training set
Top 0.2%
4.7%
8
PLOS ONE
● 5266 papers in training set
Top 30%
4.7%
9
Bioinformatics Advances
◐ 203 papers in training set
Top 2%
3.2%
10
BMC Medical Research Methodology
○ 47 papers in training set
Top 0.4%
3.2%
11
Scientific Data
○ 209 papers in training set
Top 0.9%
2.6%
12
PeerJ
◐ 308 papers in training set
Top 4%
2.1%
13
BioData Mining
○ 22 papers in training set
Top 0.2%
1.9%
14
Patterns
○ 78 papers in training set
Top 2%
1.5%
15
Scientific Reports
○ 3612 papers in training set
Top 58%
1.5%
16
Nature Biotechnology
○ 172 papers in training set
Top 3%
1.1%
17
Research Synthesis Methods
○ 20 papers in training set
Top 0.2%
1.1%
18
Journal of Cheminformatics
○ 29 papers in training set
Top 0.6%
1.1%
19
Nature Human Behaviour
○ 95 papers in training set
Top 2%
1.0%
20
Frontiers in Bioinformatics
○ 49 papers in training set
Top 1%
1.0%
21
Behavior Research Methods
○ 30 papers in training set
Top 0.6%
0.8%
22
Journal of the American Medical Informatics Association
◐ 71 papers in training set
Top 2%
0.8%
23
Lab Animal
○ 11 papers in training set
Top 0.2%
0.8%
24
npj Digital Medicine
○ 118 papers in training set
Top 3%
0.8%
25
Journal of Clinical Epidemiology
○ 31 papers in training set
Top 1.0%
0.6%
26
Database
◐ 61 papers in training set
Top 1%
0.6%

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.