Back

Deep FASTQ and BAM co-compression in Genozip 15

Lan, D. M.; Hughes, D. S. T.; Llamas, B.

2023-07-07 bioinformatics
10.1101/2023.07.07.548069 bioRxiv
Show abstract

We introduce Genozip Deep, a method for losslessly co-compressing FASTQ and BAM files. Benchmarking demonstrates improvements of 75% to 96% versus the already-compressed source files, translating to 2.3X to 6.8X better compression than current state-of-the-art algorithms that compress FASTQ and BAM separately. The Deep method is independent of the underlying FASTQ and BAM compressors, and here we present its implementation in Genozip, an established genomic data compression software.

Matching journals

The top 4 journals account for 50% of the predicted probability mass.

1
Bioinformatics
◐ 1204 papers in training set
Top 0.8%
26.3%
Similar papers in this journal
2
BMC Bioinformatics
○ 457 papers in training set
Top 0.5%
11.8%
Similar papers in this journal
3
NAR Genomics and Bioinformatics
◐ 242 papers in training set
Top 0.3%
7.8%
Similar papers in this journal
4
Genome Biology
○ 637 papers in training set
Top 1%
7.8%
Similar papers in this journal
50% of probability mass above
5
Bioinformatics Advances
◐ 203 papers in training set
Top 1%
5.1%
Similar papers in this journal
6
Nature Communications
○ 5641 papers in training set
Top 28%
5.1%
7
GigaScience
◐ 212 papers in training set
Top 0.7%
4.8%
8
Nature Computational Science
○ 55 papers in training set
Top 0.1%
3.4%
9
Nucleic Acids Research
◐ 1281 papers in training set
Top 6%
2.8%
10
Genome Research
● 468 papers in training set
Top 3%
2.6%
11
Nature Methods
○ 385 papers in training set
Top 3%
2.4%
12
PLOS Computational Biology
● 1863 papers in training set
Top 12%
2.4%
13
Scientific Reports
○ 3612 papers in training set
Top 46%
2.1%
14
Nature Biotechnology
○ 172 papers in training set
Top 2%
1.7%
15
F1000Research
○ 88 papers in training set
Top 2%
1.3%
16
Molecular Biology and Evolution
◐ 542 papers in training set
Top 4%
1.0%
17
PLOS ONE
● 5266 papers in training set
Top 59%
0.8%
18
iScience
○ 1154 papers in training set
Top 36%
0.8%
19
Communications Biology
○ 993 papers in training set
Top 35%
0.6%
20
Algorithms for Molecular Biology
○ 17 papers in training set
Top 0.2%
0.6%
21
BMC Genomics
○ 406 papers in training set
Top 9%
0.6%

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.