Back

Sourmash Branchwater Enables Lightweight Petabyte-Scale Sequence Search

Irber, L. C.; Pierce-Ward, N. T.; Brown, C. T.

2022-11-03 bioinformatics
10.1101/2022.11.02.514947 bioRxiv
Show abstract

We introduce branchwater, a flexible and fast petabase-scale search for the 767,000 public metagenomes presently in the NCBI Sequence Read Archive. Our search is based on the FracMinHash k-mer sketching technique and can search all public metagenomes with 1000 query genomes in approximately 36 hours using 50 GB of RAM and 32 threads. Branchwater is a Rust-based multithreading front-end built on top of the sourmash library. We provide biological use cases, examine performance, and discuss design and performance considerations.

Matching journals

The top 5 journals account for 50% of the predicted probability mass.

1
Bioinformatics
◐ 1204 papers in training set
Top 1%
18.0%
Similar papers in this journal
2
BMC Bioinformatics
○ 457 papers in training set
Top 0.6%
11.6%
Similar papers in this journal
3
Genome Biology
○ 637 papers in training set
Top 0.9%
10.7%
Similar papers in this journal
4
GigaScience
◐ 212 papers in training set
Top 0.2%
9.5%
Similar papers in this journal
5
Bioinformatics Advances
◐ 203 papers in training set
Top 0.7%
6.5%
Similar papers in this journal
50% of probability mass above
6
PLOS Computational Biology
● 1863 papers in training set
Top 7%
5.3%
7
PLOS ONE
● 5266 papers in training set
Top 37%
3.1%
8
NAR Genomics and Bioinformatics
◐ 242 papers in training set
Top 2%
2.3%
9
BMC Genomics
○ 406 papers in training set
Top 3%
2.3%
10
Journal of Open Source Software
● 25 papers in training set
Top 0.2%
2.3%
11
Genome Research
● 468 papers in training set
Top 3%
2.1%
12
PeerJ
◐ 308 papers in training set
Top 6%
1.7%
13
Briefings in Bioinformatics
◐ 354 papers in training set
Top 5%
1.7%
14
mSystems
● 394 papers in training set
Top 5%
1.4%
15
Nature Biotechnology
○ 172 papers in training set
Top 3%
1.3%
16
F1000Research
○ 88 papers in training set
Top 3%
1.1%
17
Nucleic Acids Research
◐ 1281 papers in training set
Top 11%
1.1%
18
Nature Methods
○ 385 papers in training set
Top 6%
1.1%
19
Journal of Computational Biology
○ 48 papers in training set
Top 0.9%
1.0%
20
Gigabyte
● 62 papers in training set
Top 1%
1.0%
21
Molecular Biology and Evolution
◐ 542 papers in training set
Top 5%
1.0%
22
G3: Genes, Genomes, Genetics
◐ 252 papers in training set
Top 5%
0.8%
23
iScience
○ 1154 papers in training set
Top 38%
0.8%
24
G3 Genes|Genomes|Genetics
◐ 351 papers in training set
Top 4%
0.8%
25
Molecular Ecology Resources
○ 171 papers in training set
Top 2%
0.6%
26
Journal of Bioinformatics and Systems Biology
○ 15 papers in training set
Top 0.4%
0.6%
27
SoftwareX
○ 15 papers in training set
Top 0.4%
0.6%
28
Nature Communications
○ 5641 papers in training set
Top 60%
0.6%

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.