Back

Hidden multiplicity in the analysis of variance (ANOVA): multiple contrast tests as an alternative

Hothorn, L. A.

2022-01-18 bioinformatics
10.1101/2022.01.15.476452 bioRxiv
Show abstract

In bio-medical studies, the p-values of the F-tests in ANOVA are usually interpreted independently as measures of the significance of the associated factors. This hidden multiplicity effect increases the false positive rate. Therefore, Cramer et al. (2016) proposed the Bonferroni adjustment of the p-values to control for familywise error rate for the experiment. Here, instead of using F-tests, it is alternatively suggested to use multiple contrast tests vs. total mean and to perform multiplicity adjustment by object merging in the interplay between the R-packages emmeans and multcomp. This new approach, denotes as multipleANOM, allows not only to interpret global factor effects but also local effects between factor levels as adjusted p-values or simultaneous confidence intervals for selected effect measures in generalized linear models. R-code is provided by means of selected data examples.

Matching journals

The top 3 journals account for 50% of the predicted probability mass.

1
PLOS ONE
● 5266 papers in training set
Top 2%
40.9%
Similar papers in this journal
2
Bioinformatics
◐ 1204 papers in training set
Top 3%
8.2%
Similar papers in this journal
3
BMC Bioinformatics
○ 457 papers in training set
Top 1%
6.5%
Similar papers in this journal
50% of probability mass above
4
Peer Community Journal
● 281 papers in training set
Top 0.6%
6.5%
Similar papers in this journal
5
PLOS Computational Biology
● 1863 papers in training set
Top 8%
4.5%
Similar papers in this journal
6
PeerJ
◐ 308 papers in training set
Top 2%
3.6%
7
Royal Society Open Science
● 214 papers in training set
Top 2%
2.8%
8
Scientific Reports
○ 3612 papers in training set
Top 40%
2.5%
9
Briefings in Bioinformatics
◐ 354 papers in training set
Top 6%
1.1%
10
BMC Research Notes
○ 33 papers in training set
Top 1%
0.9%
11
Biology
○ 45 papers in training set
Top 1%
0.6%
12
Heliyon
○ 152 papers in training set
Top 9%
0.6%
13
Mathematical Biosciences
○ 49 papers in training set
Top 1%
0.6%
14
Statistics in Medicine
○ 40 papers in training set
Top 0.6%
0.6%
15
GigaScience
◐ 212 papers in training set
Top 5%
0.6%
16
Frontiers in Genetics
○ 230 papers in training set
Top 6%
0.6%
17
Biosystems
○ 31 papers in training set
Top 0.6%
0.6%
18
Frontiers in Pharmacology
○ 111 papers in training set
Top 4%
0.5%
19
Computers in Biology and Medicine
○ 128 papers in training set
Top 6%
0.5%
20
International Journal of Environmental Research and Public Health
○ 128 papers in training set
Top 6%
0.5%
21
Applied Sciences
○ 25 papers in training set
Top 1%
0.5%
22
Entropy
○ 21 papers in training set
Top 0.4%
0.5%
23
Molecules
○ 39 papers in training set
Top 2%
0.5%
24
Ecology and Evolution
○ 267 papers in training set
Top 6%
0.5%
25
BMC Medical Research Methodology
○ 47 papers in training set
Top 2%
0.5%

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.