SciCore-Omics: a tri-modal foundation model unifying histology, spatial transcriptomics and language for spatial biology
Xiao, X.; Li, Y.; Zeng, Z.; Yan, Y.; Liu, Z.; Liu, Z.; Xiang, Y.; Ye, Z.; Ying, J.; Li, Y.; Xie, L.; He, F.
Show abstract
Histomorphology and spatial transcriptomics capture complementary aspects of tissue biology, but their relationships remain difficult to extract, align, and interpret at scale. Existing foundation models typically connect histology, omics, or language only pairwise, which limits their capacity to jointly infer molecular states, decode spatial tissue organization, and generate biologically grounded explanations. Here, we show SciCore-Omics, the first tri-modal foundation model linking histology images, spatial transcriptomics, and biological language. We constructed a spatially paired image-gene-text dataset comprising 151,182 spots across multiple tissues and performed a three-stage progressive training of SciCore-Omics on this dataset. Across gene expression prediction and spatial domain recognition, SciCore-Omics achieved 23.6-80.9% relative gains in task-specific metrics over the strongest external baselines. It further showed robust zero-shot generalization in histopathology classification, outperforming GPT-5 by 6.16 percentage points in mean accuracy across four benchmarks. Expert evaluation in 10 breast cancer cases confirmed its H&E-only case-level molecular reasoning capability. Together, our method demonstrates that a tri-modal framework can effectively bridge histomorphology and molecular state, providing a more general and interpretable foundation model for computational pathology and omics analysis.
Matching journals
The top 4 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Learning interpretable cellular and gene signature embeddings from single-cell transcriptomic data 96%
- uniPort: a unified computational framework for single-cell data integration with optimal transport 96%
- scMODAL: A general deep learning framework for comprehensive single-cell multi-omics data alignment with feature links 96%
Similar papers in this journal
- scCross: A Deep Generative Model for Unifying Single-cell Multi-omics with Seamless Integration, Cross-modal Generation, and In-silico Exploration 96%
- stGCL: A versatile cross-modality fusion method based on multi-modal graph contrastive learning for spatial transcriptomics 96%
- Cross-species imputation and comparison of single-cell transcriptomic profiles 95%
Similar papers in this journal
- Inferring spatial single-cell-level interactions through interpreting cell state and niche correlations learned by self-supervised graph transformer 95%
- Construction of a 3D whole organism spatial atlas by joint modeling of multiple slices 95%
- Simultaneous dimensionality reduction and integration for single-cell ATAC-seq data using deep learning 95%
Similar papers in this journal
- Cell type identification in spatial transcriptomics data can be improved by leveraging cell-type-informative paired tissue images using a Bayesian probabilistic model. 96%
- Developing a general AI model for integrating diverse genomic modalities and comprehensive genomic knowledge 95%
- SPOTlight:Seeded NMF regression to Deconvolute Spatial Transcriptomics Spots with Single-Cell Transcriptomes 95%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.