Back

CryoViT: Efficient Segmentation of Cryogenic Electron Tomograms with Vision Foundation Models

Gupte, S. R.; Hou, C.; Wu, G.-H.; Galaz-Montoya, J. G.; Chiu, W.; Yeung-Levy, S.

2024-06-30 bioinformatics
10.1101/2024.06.26.600701 bioRxiv
Show abstract

Cryogenic electron tomography (cryoET) directly visualizes subcellular structures in 3D at the nanometer scale. Quantitative analyses of cryoET data can reveal structural biomarkers of diseases, provide novel mechanistic insights, and inform the effects of treatments on phenotype. However, existing automated annotation approaches primarily focus on localizing molecular features with few methods accurately quantifying complex structures such as organelles. We address this challenge with CryoViT, a paradigm shift from traditional convolutional neural networks that leverages vision transformers to enhance the segmentation of large pleomorphic structures that can occupy almost the entire field of view in high-magnification images, such as mitochondria. CryoViT is powered by a large-scale vision foundation model and overcomes limitations of popular U-Net based methods, particularly when training data are scarce. We demonstrate the efficacy of CryoViT on a large cryoET dataset of neurons differentiated from iPSCs derived from Huntington disease (HD) patients and cultured HD mouse model neurons.

Matching journals

The top 3 journals account for 50% of the predicted probability mass.

1
Nature Methods
○ 385 papers in training set
Top 0.2%
23.5%
Similar papers in this journal
2
Nature Communications
○ 5641 papers in training set
Top 7%
19.4%
Similar papers in this journal
3
Bioinformatics
◐ 1204 papers in training set
Top 3%
8.3%
Similar papers in this journal
50% of probability mass above
4
Communications Biology
○ 993 papers in training set
Top 2%
5.1%
Similar papers in this journal
5
Nature Machine Intelligence
○ 70 papers in training set
Top 0.6%
4.2%
Similar papers in this journal
6
Scientific Reports
○ 3612 papers in training set
Top 27%
3.7%
7
Journal of Structural Biology
○ 64 papers in training set
Top 0.2%
2.9%
8
Briefings in Bioinformatics
◐ 354 papers in training set
Top 3%
2.8%
9
Nature Biotechnology
○ 172 papers in training set
Top 2%
2.5%
10
eLife
● 5828 papers in training set
Top 47%
2.0%
11
PLOS Computational Biology
● 1863 papers in training set
Top 14%
1.8%
12
Cell Reports Methods
○ 165 papers in training set
Top 1%
1.8%
13
Scientific Data
○ 209 papers in training set
Top 2%
1.6%
14
Proceedings of the National Academy of Sciences
● 2444 papers in training set
Top 31%
1.4%
15
Cell Systems
○ 201 papers in training set
Top 4%
0.9%
16
Genome Biology
○ 637 papers in training set
Top 8%
0.9%
17
Structure
○ 193 papers in training set
Top 2%
0.9%
18
Advanced Science
○ 286 papers in training set
Top 8%
0.9%
19
Biological Imaging
◐ 15 papers in training set
Top 0.3%
0.6%
20
Nature
○ 645 papers in training set
Top 11%
0.6%
21
Science Advances
● 1243 papers in training set
Top 32%
0.6%
22
Communications Chemistry
○ 48 papers in training set
Top 2%
0.5%
23
GigaScience
◐ 212 papers in training set
Top 5%
0.5%
24
PLOS ONE
● 5266 papers in training set
Top 64%
0.5%
25
iScience
○ 1154 papers in training set
Top 44%
0.5%
26
Nature Biomedical Engineering
○ 47 papers in training set
Top 2%
0.5%

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.