Integrating Histology with Spatial Molecular Programs Using a Multimodal Foundation Model
Zhang, Z.; Qin, B.; Zhao, Y.; Qi, Z.; Xu, H.; Wang, Y.; Zheng, W.; Dai, J.; Chen, A.; Wang, N.; Nie, L.; Zhang, P.; Zhang, H.; Zhao, Y.; Xu, T.; Lin, S.; Ren, P.; Zhang, Z.; Xue, L.; Xue, X.; Yang, Z.; Xu, J.; Pan, D.; Wang, C.; Liu, Z.; Meng, Y.; Zeng, Z.
Show abstract
Histopathological assessment remains central to cancer diagnosis and stratification, yet its mechanistic interpretation remains limited without molecular context. To address this, we developed SQUALL, a multimodal foundation model integrating histology with spatial molecular programs. For pretraining, we assembled histMol, a large-scale corpus of 1.76 billion paired histology-spatial transcriptomics spots/bins across 33 tissues and 12 platforms from 3,446 tissue sections. Following pretraining, SQUALL enables transcriptome-wide virtual biomarker profiling, prognostically relevant spatial niches discovery, and integrative disease progression modeling. Leveraging its multimodal embeddings, SQUALL identifies niches associated with tertiary lymphoid structure (TLS) maturation and ovarian cancer relapse, reconstructs molecular trajectories of breast cancer invasion across 325,112 spots, and uncovers underlying transcriptional programs. Applied to whole-slide images from 898 patients, SQUALL outperforms existing pathology foundation models in outcome prediction while enabling interpretable risk stratification. Together, these results establish spatially aligned multimodal pretraining as a new paradigm for extending molecular insights into pathology images.
Matching journals
The top 6 journals account for 50% of the predicted probability mass.
Similar papers in this journal
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.