Back

CRAM compression: practical across-technologies considerations for large-scale sequencing projects

Al Ali, A.; Kandavel, P. K.; Al Mabrazi, H.; Carvalho, G.; Kusuma, V.; Katagi, G.; Elavalli, S.; Yousif, A.; Akhter, M. R.; Mafofo, J.; Magalhaes, T.; Quilez, J.

2022-12-22 bioinformatics
10.1101/2022.12.21.521516 bioRxiv
Show abstract

CRAM is an efficient format to store high-throughput sequencing data and it has been widely adopted. We thus plan to use CRAM for the Emirati Genome Program, which aims to sequence the genomes of ~1 million nationals in the United Arab Emirates using short- and long-read sequencing technologies (Illumina, MGI and Oxford Nanopore Sequencing). We conducted a pilot study on the three technologies before start using CRAM at scale. We found CRAM achieved 40-70% compression depending on the sequencing platform. As expected, CRAM compression was data lossless and did not alter variant calls. In our cloud, we observed compression speeds 0.7-1.4 GB per minute, varying on the sequencing platform too. This translates into ~1-2 hours using a single CPU to compress a ~30X human whole-genome sequencing sample. Despite its wide use, we found little publicly available information about CRAM compression rate, speed, losslessness and parallelization, especially across many sequencing platforms. This work will have direct application for Emirati Genome Program and provide practical considerations for other large-scale sequencing efforts.

Matching journals

The top 4 journals account for 50% of the predicted probability mass.

1
Bioinformatics
◐ 1204 papers in training set
Top 1%
21.7%
Similar papers in this journal
2
GigaScience
◐ 212 papers in training set
Top 0.1%
18.3%
Similar papers in this journal
3
Bioinformatics Advances
◐ 203 papers in training set
Top 0.4%
7.8%
Similar papers in this journal
4
NAR Genomics and Bioinformatics
◐ 242 papers in training set
Top 0.4%
6.7%
Similar papers in this journal
50% of probability mass above
5
BMC Bioinformatics
○ 457 papers in training set
Top 1%
6.7%
Similar papers in this journal
6
Molecular Ecology Resources
○ 171 papers in training set
Top 0.6%
4.0%
7
PeerJ
◐ 308 papers in training set
Top 2%
3.2%
8
F1000Research
○ 88 papers in training set
Top 0.5%
3.2%
9
Scientific Reports
○ 3612 papers in training set
Top 46%
2.1%
10
Genome Research
● 468 papers in training set
Top 3%
1.9%
11
G3: Genes, Genomes, Genetics
◐ 252 papers in training set
Top 2%
1.9%
12
iScience
○ 1154 papers in training set
Top 21%
1.5%
13
Genome Biology
○ 637 papers in training set
Top 6%
1.5%
14
BMC Genomics
○ 406 papers in training set
Top 5%
1.4%
15
Computational and Structural Biotechnology Journal
● 242 papers in training set
Top 4%
1.3%
16
Molecular Biology and Evolution
◐ 542 papers in training set
Top 4%
1.1%
17
Frontiers in Genetics
○ 230 papers in training set
Top 4%
1.1%
18
G3 Genes|Genomes|Genetics
◐ 351 papers in training set
Top 3%
1.1%
19
Nature Communications
○ 5641 papers in training set
Top 51%
1.1%
20
PLOS ONE
● 5266 papers in training set
Top 57%
0.9%
21
PLOS Computational Biology
● 1863 papers in training set
Top 20%
0.8%
22
Briefings in Bioinformatics
◐ 354 papers in training set
Top 7%
0.8%
23
Communications Biology
○ 993 papers in training set
Top 31%
0.8%
24
Journal of Bioinformatics and Systems Biology
○ 15 papers in training set
Top 0.4%
0.6%

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.