H&Enium, Applying Foundation Models to Computational Pathology and Spatial Transcriptomics to Learn an Aligned Latent Space
Glettig, M.; Ehrensperger, T.; Yates, J.; Boeva, V.
Show abstract
Bridging the gap from transcriptomic to imaging data at single-cell resolution is essential for understanding tumor biology and improving cancer diagnostics. Spatial transcriptomics enables mapping gene expression onto H&E images of segmented single cells, but remains limited by cost and throughput. We introduce H&Enium, a contrastive alignment framework that projects image and gene expression embeddings from foundation models into an aligned latent space using projection heads and a novel soft alignment target. This alignment enriches image-derived embeddings with transcriptomic context improving downstream tasks such as cell type classification and gene expression prediction. Additional evaluations on independent pathology datasets demonstrate superior generalization of our aligned representations over unaligned baselines. Our method offers a scalable path to enhance the utility of standard H&E imaging in both research and clinical settings.
Matching journals
The top 5 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Spatial Transcriptomics Expression Prediction from Histopathology Based on Cross-Modal Mask Reconstruction and Contrastive Learning 95%
- A Framework for Falsifiable Explanations of Machine Learning Models with an Application in Computational Pathology 95%
- Clinical Validation of Saliency Maps for Understanding Deep Neural Networks in Ophthalmology 92%
Similar papers in this journal
- Hypergraph factorisation for multi-tissue gene expression imputation 94%
- Generalized Radiograph Representation Learning via Cross-supervision between Images and Free-text Radiology Reports 93%
- Inferring spatial single-cell-level interactions through interpreting cell state and niche correlations learned by self-supervised graph transformer 93%
Similar papers in this journal
- Interpretable multimodal deep learning for real-time pan-tissue pan-disease pathology search on social media 95%
- Built to last? Reproducibility and Reusability of Deep Learning Algorithms in Computational Pathology 95%
- Attention-based whole-slide image compression achieves pathologist-level pre-screening of multi-organ routine histopathology biopsies 92%
Similar papers in this journal
- ROSIE: AI generation of multiplex immunofluorescence staining from histopathology images 97%
- Integrating T-cell receptor and transcriptome for large-scale single-cell immune profiling analysis 95%
- Segmenting functional tissue units across human organs using community-driven development of generalizable machine learning algorithms 94%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.