Back

Guidelines for reliable and reproducible functional enrichment analysis

Wijesooriya, K.; Jadaan, S. A.; Perera, K. L.; Kaur, T.; Ziemann, M.

2021-09-07 bioinformatics
10.1101/2021.09.06.459114 bioRxiv
Show abstract

Gene set enrichment tests (a.k.a. functional enrichment analysis) are among the most frequently used methods in computational biology. Despite this popularity, there are concerns that these methods are being applied incorrectly and the results of some peer-reviewed publications are unreliable. These problems include the use of inappropriate background gene lists, lack of false discovery rate correction and lack of methodological detail. An example analysis of public RNA-seq reveals that these methodological errors alter enrichment results dramatically. To ascertain the frequency of these errors in the literature, we performed a screen of 186 open access research articles describing functional enrichment results. We find that 95% of analyses using over-representation tests did not implement an appropriate background gene list or did not describe this in the methods. Failure to perform p-value correction for multiple tests was identified in 43% of analyses. Many studies lacked detail in the methods section about the tools and gene sets used. Only 15% of studies avoided major flaws, which highlights the poor state of functional enrichment rigour and reporting in the contemporary literature. We provide a set of minimum standards that should act as a checklist for researchers and peer-reviewers.

Matching journals

The top 8 journals account for 50% of the predicted probability mass.

1
PLOS ONE
● 5266 papers in training set
Top 16%
11.8%
Similar papers in this journal
2
BMC Bioinformatics
○ 457 papers in training set
Top 0.6%
10.9%
Similar papers in this journal
3
Briefings in Bioinformatics
◐ 354 papers in training set
Top 1%
6.7%
Similar papers in this journal
4
Genome Biology
○ 637 papers in training set
Top 2%
6.2%
Similar papers in this journal
5
NAR Genomics and Bioinformatics
◐ 242 papers in training set
Top 0.6%
5.5%
Similar papers in this journal
6
Nucleic Acids Research
◐ 1281 papers in training set
Top 4%
4.3%
7
BMC Genomics
○ 406 papers in training set
Top 2%
3.5%
8
GigaScience
◐ 212 papers in training set
Top 1%
3.4%
50% of probability mass above
9
Scientific Reports
○ 3612 papers in training set
Top 33%
3.2%
10
Bioinformatics
◐ 1204 papers in training set
Top 5%
3.2%
11
BMC Biology
○ 265 papers in training set
Top 0.8%
3.2%
12
Computational and Structural Biotechnology Journal
● 242 papers in training set
Top 2%
3.1%
13
Genes
○ 144 papers in training set
Top 0.8%
3.1%
14
Bioinformatics Advances
◐ 203 papers in training set
Top 3%
1.9%
15
PeerJ
◐ 308 papers in training set
Top 5%
1.7%
16
Database
◐ 61 papers in training set
Top 0.5%
1.7%
17
PLOS Biology
● 486 papers in training set
Top 5%
1.7%
18
Life Science Alliance
● 285 papers in training set
Top 5%
1.3%
19
BMC Research Notes
○ 33 papers in training set
Top 0.5%
1.3%
20
PLOS Computational Biology
● 1863 papers in training set
Top 17%
1.1%
21
Genomics, Proteomics & Bioinformatics
188 papers in training set
Top 2%
1.1%
22
Metabolites
○ 53 papers in training set
Top 0.7%
1.1%
23
eLife
● 5828 papers in training set
Top 62%
1.0%
24
BioData Mining
○ 22 papers in training set
Top 0.5%
1.0%
25
PROTEOMICS
○ 43 papers in training set
Top 0.6%
1.0%
26
Cell Systems
○ 201 papers in training set
Top 5%
0.8%
27
Genomics
○ 64 papers in training set
Top 2%
0.8%
28
F1000Research
○ 88 papers in training set
Top 4%
0.8%
29
iScience
○ 1154 papers in training set
Top 36%
0.8%
30
PLOS Digital Health
● 106 papers in training set
Top 5%
0.6%

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.