Back

The MTIST platform: a microbiome time series inference standardized test simulation, dataset, and scoring systems

Hussey, G. A.; Zhang, C.; Sullivan, A. P.; Fenyo, D.; Schluter, J.

2022-10-18 microbiology
10.1101/2022.10.18.512783 bioRxiv
Show abstract

The human gut microbiome is promising therapeutic target, but development of interventions is hampered by limited understanding of the microbial ecosystem. Therefore, recent years have seen a surge in the engineering of inference algorithms seeking to unravel rules of ecological interactions from metagenomic data. Research groups score algorithmic performance in a variety of different ways, however, there exists no unified framework to score and rank each inference approach. The machine learning field presents a useful solution to this issue: a unified set of validation data and accompanying scoring metric. Here, we present MTIST: a platform for benchmarking microbial ecosystem inference tools. We use a generalized Lotka-Volterra framework to simulate microbial abundances over time, akin to what would be obtained by quantitative metagenomic sequencing studies or lab experiments, to generate a massive in silico training dataset (MTIST) for algorithmic validation, as well as an "ecological sign" score (ES score) to rate them. MTIST comprises 24,570 time series of microbial abundance data packaged into 648 datasets. Together, the MTIST dataset and the ES score serve as a platform to develop and compare microbiome ecosystem inference approaches.

Matching journals

The top 7 journals account for 50% of the predicted probability mass.

1
Nature Communications
○ 5641 papers in training set
Top 16%
11.6%
Similar papers in this journal
2
PLOS Computational Biology
● 1863 papers in training set
Top 4%
9.5%
Similar papers in this journal
3
mSystems
● 394 papers in training set
Top 0.7%
9.4%
Similar papers in this journal
4
Cell Systems
○ 201 papers in training set
Top 0.3%
8.7%
Similar papers in this journal
5
eLife
● 5828 papers in training set
Top 24%
5.3%
6
Microbiome
○ 154 papers in training set
Top 0.6%
5.3%
Similar papers in this journal
7
Bioinformatics
◐ 1204 papers in training set
Top 4%
4.7%
50% of probability mass above
8
Methods in Ecology and Evolution
○ 176 papers in training set
Top 0.6%
3.9%
9
Genome Biology
○ 637 papers in training set
Top 4%
3.2%
10
Proceedings of the National Academy of Sciences
● 2444 papers in training set
Top 17%
3.1%
11
PLOS Biology
● 486 papers in training set
Top 2%
3.1%
12
Cell Reports Methods
○ 165 papers in training set
Top 2%
1.6%
13
ISME Communications
◐ 120 papers in training set
Top 1%
1.5%
14
Cell
○ 431 papers in training set
Top 7%
1.4%
15
Briefings in Bioinformatics
◐ 354 papers in training set
Top 5%
1.4%
16
Nature Microbiology
○ 155 papers in training set
Top 2%
1.3%
17
Molecular Ecology Resources
○ 171 papers in training set
Top 1%
1.3%
18
Cell Host & Microbe
126 papers in training set
Top 2%
1.2%
19
iScience
○ 1154 papers in training set
Top 28%
1.1%
20
Molecular Biology and Evolution
◐ 542 papers in training set
Top 4%
1.0%
21
Microbiology Spectrum
● 469 papers in training set
Top 9%
1.0%
22
Cell Reports
○ 1498 papers in training set
Top 25%
1.0%
23
The ISME Journal
◐ 228 papers in training set
Top 3%
1.0%
24
Ecology and Evolution
○ 267 papers in training set
Top 5%
0.9%
25
Genome Research
● 468 papers in training set
Top 6%
0.8%
26
PeerJ
◐ 308 papers in training set
Top 11%
0.8%
27
Nature Biotechnology
○ 172 papers in training set
Top 4%
0.8%
28
Ecology Letters
○ 135 papers in training set
Top 2%
0.8%
29
Nature Methods
○ 385 papers in training set
Top 7%
0.6%
30
Nucleic Acids Research
◐ 1281 papers in training set
Top 15%
0.6%

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.