Compositional and interpretable representation of histology using AI foundation models and sparse autoencoders
Zhao, Z.; Maliga, Z.; Ogbonna, E. C.; Talemi, S. R.; Coy, S.; Gagne, A.; Lumamba, K.; Solomon, I. H.; Santagata, S.; Steyn, A. J. C.; Naidoo, T.; Sorger, P. K.
Show abstract
Light microscopy of tissue sections stained with hematoxylin and eosin (H&E) has been the foundation of histopathology for over 150 years and remains essential for diagnosis and research. The development of high-plex spatial profiling approaches able to measure protein and RNA expression at single-cell resolution augments but does not replace H&E imaging, even in research. Computational pathology (CPath) models based on deep learning promise to further increase the value of H&E imaging but interpreting these models in biological terms remains challenging. As a result, they are not widely used in spatial profiling studies. Here we describe a human-in-the-loop computational framework that leverages CPath foundation models (FMs) and sparse autoencoders (SAEs) to decompose FM embeddings and automatically identify diverse, human-interpretable histopathology features in H&E images. When FM-SAE modeling was applied to pulmonary diseases such as tuberculosis and lung cancer, human-machine interaction augmented and accelerated expert interpretation. Moreover, the resulting annotations provide a morphology-aware approach to integrating 2D and 3D mesoscale tissue architectures with molecular spatial profiling.
Matching journals
The top 4 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- PHARAOH: A collaborative crowdsourcing platform for PHenotyping And Regional Analysis Of Histology 96%
- STAIG: Spatial Transcriptomics Analysis via Image-Aided Graph Contrastive Learning for Domain Exploration and Alignment-Free Integration 96%
- Smart hybrid microscopy for cell-friendlydetection of rare events 96%
Similar papers in this journal
- Integrative, high-resolution analysis of single cell gene expression across experimental conditions with PARAFAC2-RISE 95%
- Simple visualization of submicroscopic protein clusters with a phase-separation-based fluorescent reporter. 94%
- Interpretable deep learning of label-free live cell images uncovers functional hallmarks of highly-metastatic melanoma 94%
Similar papers in this journal
- Identifying perturbations that boost T-cell infiltration into tumours via counterfactual learning of their spatial proteomic profiles 93%
- Defining and predicting transdiagnostic categories of neurodegenerative disease 93%
- Ultra-fast Prediction of Somatic Structural Variations by Reduced Read Mapping via Pan-Genome k-mer Sets 92%
Similar papers in this journal
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.