Back

Pipeline Olympics: continuable benchmarking of computational workflows for DNA methylation sequencing data against an experimental gold-standard

Lin, Y.-Y.; Breuer, K.; Weichenhan, D.; Lafrenz, P.; Wilk, A.; Chepeleva, M.; Muecke, O.; Schoenung, M.; Petermann, F.; Kensche, P.; Weiser, L.; Thommen, F.; Giacomelli, G.; Nordstroem, K.; Gonzales-Avalos, E.; Merkel, A.; Kretzmer, H.; Fischer, J.; Kraemer, S.; Iskar, M.; Wolf, S.; Buchhalter, I.; Esteller, M.; Lawerenz, C.; Twardziok, S.; Zapatka, M.; Hovestadt, V.; Schlesner, M.; Schulz, M.; Hoffman, S.; Gerhauser, C.; Walter, J.; Hartmann, M.; Lipka, D.; Assenov, Y.; Bock, C.; Plass, C.; Toth, R.; Lutsik, P.

2024-09-19 bioinformatics
10.1101/2024.09.16.609142 bioRxiv
Show abstract

DNA methylation is a widely studied epigenetic mark and a powerful biomarker of cell type, age, environmental exposures, and disease. Whole-genome sequencing following selective conversion of unmethylated cytosines into thymines via bisulfite treatment or enzymatic methods remains the reference method for DNA methylation profiling genome-wide. While numerous software tools facilitate processing of DNA methylation sequencing reads, a comprehensive benchmarking study has been lacking thus far. In this study, we systematically compared complete computational workflows for processing DNA methylation sequencing data using a dedicated benchmarking dataset generated with five genome-wide profiling protocols. As an evaluation reference, we employed highly quantitative locus-specific measurements from our preceding benchmark of targeted DNA methylation assays. Based on this experimental gold-standard assessment and several comprehensive metrics, we identified workflows that consistently demonstrated superior performance and revealed major workflow development trends. To facilitate the sustainability of our benchmark, we implemented an interactive workflow execution and data presentation platform, adaptable to user-defined criteria and seamlessly expandable to future software.

Matching journals

The top 3 journals account for 50% of the predicted probability mass.

1
Genome Biology
○ 637 papers in training set
Top 0.1%
33.6%
Similar papers in this journal
2
GigaScience
◐ 212 papers in training set
Top 0.2%
9.6%
Similar papers in this journal
3
Briefings in Bioinformatics
◐ 354 papers in training set
Top 1%
7.1%
Similar papers in this journal
50% of probability mass above
4
Nature Communications
○ 5641 papers in training set
Top 24%
6.6%
Similar papers in this journal
5
Bioinformatics
◐ 1204 papers in training set
Top 4%
6.1%
Similar papers in this journal
6
Genome Research
● 468 papers in training set
Top 2%
4.0%
7
BMC Bioinformatics
○ 457 papers in training set
Top 3%
3.2%
8
Nucleic Acids Research
◐ 1281 papers in training set
Top 6%
2.7%
9
PLOS Computational Biology
● 1863 papers in training set
Top 11%
2.6%
10
NAR Genomics and Bioinformatics
◐ 242 papers in training set
Top 2%
2.3%
11
BMC Genomics
○ 406 papers in training set
Top 3%
2.3%
12
Cell Reports Methods
○ 165 papers in training set
Top 1%
2.1%
13
Nature Methods
○ 385 papers in training set
Top 4%
1.9%
14
Genomics, Proteomics & Bioinformatics
188 papers in training set
Top 1%
1.7%
15
Computational and Structural Biotechnology Journal
● 242 papers in training set
Top 4%
1.3%
16
Bioinformatics Advances
◐ 203 papers in training set
Top 5%
0.8%
17
Scientific Reports
○ 3612 papers in training set
Top 74%
0.8%
18
iScience
○ 1154 papers in training set
Top 37%
0.8%
19
Nature Biotechnology
○ 172 papers in training set
Top 4%
0.8%
20
Genome Medicine
○ 183 papers in training set
Top 5%
0.8%
21
eLife
● 5828 papers in training set
Top 71%
0.6%

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.