Back

In-text citation error rate as a scientometric tool for evaluating accuracy and weighing evidence

Lee, D. E.

2022-06-02 scientific communication and education
10.1101/2022.06.01.494325 bioRxiv
Show abstract

Scientists are fallable and biased, but accuracy can be assessed through empirical analysis of published work that quantifies in-text citation (or quotation) errors. In scientific conflicts, it can be difficult for outsiders to know whose evidence or interpretation to trust. In-text citation error rate can assist decision- and policy-making bodies, as well as the courts when conflicts reach the judicial branch of government, by quantifying absolute and relative accuracy of scientists presenting scientific evidence. I propose the use of in-text citation error rates as a scientometric tool to quantify the accuracy of an authors work. In-text citation error rates in excess of an established overall mean (e.g., 11% for minor errors and 7% for major errors in ecology), or differences in in-text citation error rates between opposing groups of scientists could be used to reveal excessive inaccuracies in an author or group. The spotted owl (Strix occidentalis) has been at the center of a multi-decadal conflict caused by competition among people over forest resources, with scientific experts representing opposing stakeholders often presenting conflicting evidence. I applied the in-text citation error rate tool to important papers in the spotted owl and forest fire debate and found evidence of greater error rates in works on one side of this debate. In-text citation error rate can be an effective tool for quantifying accuracy among scientists.

Matching journals

The top 4 journals account for 50% of the predicted probability mass.

1
PLOS ONE
4510 papers in training set
Top 5%
23.6%
2
FACETS
11 papers in training set
Top 0.1%
10.6%
3
PeerJ
261 papers in training set
Top 0.2%
9.6%
4
Ecological Informatics
29 papers in training set
Top 0.1%
7.1%
50% of probability mass above
5
PLANTS, PEOPLE, PLANET
21 papers in training set
Top 0.1%
5.1%
6
Royal Society Open Science
193 papers in training set
Top 0.2%
5.1%
7
Philosophical Transactions of the Royal Society B
51 papers in training set
Top 0.9%
4.4%
8
PLOS Biology
408 papers in training set
Top 4%
3.4%
9
Conservation Biology
14 papers in training set
Top 0.1%
2.5%
10
eLife
5422 papers in training set
Top 34%
2.2%
11
Ecology and Evolution
232 papers in training set
Top 2%
2.0%
12
Heliyon
146 papers in training set
Top 2%
1.8%
13
Scientific Reports
3102 papers in training set
Top 56%
1.8%
14
Biological Conservation
43 papers in training set
Top 0.4%
1.8%
15
eneuro
389 papers in training set
Top 7%
1.3%
16
Frontiers in Marine Science
55 papers in training set
Top 0.8%
1.2%
17
BMC Medical Research Methodology
43 papers in training set
Top 0.9%
1.2%
18
FEBS Letters
42 papers in training set
Top 0.2%
0.9%
19
Ecosphere
53 papers in training set
Top 0.6%
0.8%
20
Proceedings of the Royal Society B: Biological Sciences
341 papers in training set
Top 6%
0.8%
21
PLOS Computational Biology
1633 papers in training set
Top 23%
0.8%
22
Entropy
20 papers in training set
Top 0.4%
0.8%
23
Research Synthesis Methods
20 papers in training set
Top 0.2%
0.8%
24
Wellcome Open Research
57 papers in training set
Top 3%
0.5%