Back

Dissecting and directing pathology foundation models

Kim, C.; Kaczmarzyk, J.; Savant, D.; Zhao, Z.; Koo, P.; Lee, S.-I.

2026-06-16 pathology
10.64898/2026.06.12.731496 bioRxiv
Show abstract

Foundation models (FMs) are central to digital pathology, encoding histology images into dense embeddings for facilitating diagnostic classification, molecular alteration prediction, and clinical outcome modeling. However, the opacity of these embeddings renders FM-based systems "black boxes," limiting their trustworthiness for clinical translation and utility for scientific discovery. Here, we introduce PICASSO (Pathology Image Concept Atlas built via SparSe dictiOnary learning), a framework that makes pathology FMs interpretable and controllable. PICASSO decomposes FM embeddings into human-interpretable visual concepts using a sparse autoencoder. It is trained on more than 120 million tissue patches across 32 cancer types, producing the first pan-cancer atlas of histomorphological concepts. We demonstrate that PICASSO enables diverse downstream applications of FM embeddings by exposing interpretable structure within learned representations and supporting concept-level intervention. It enables auditing of clinical model behavior by revealing the morphological features driving predictions. Beyond transparency and validation, PICASSO enables the discovery of new biological insights; for example, it identified hobnailing epithelial morphology as a previously unrecognized biomarker of EGFR mutations in lung adenocarcinoma. By linking PICASSO-derived concepts with spatial transcriptomics, we uncover associations between morphological patterns and gene expression programs. Furthermore, PICASSO allows suppression of concepts associated with technical artifacts, thereby reducing model reliance on spurious signals. Finally, PICASSO enables controlled manipulation of learned concepts to generate counterfactual embeddings for exploratory therapeutic analysis, such as modulating tumour-infiltrating lymphocyte density to assess impacts on predict survival outcomes. Together, PICASSO provides a principled framework for transforming pathology FMs into platforms for mechanistic insight and discovery.

Matching journals

The top 5 journals account for 50% of the predicted probability mass.

1
Nature Machine Intelligence
70 papers in training set
Top 0.1%
18.3%
2
Nature Communications
5641 papers in training set
Top 16%
11.7%
3
Nature
645 papers in training set
Top 1%
10.9%
4
npj Digital Medicine
118 papers in training set
Top 0.8%
7.8%
5
Nature Methods
385 papers in training set
Top 1%
7.8%
50% of probability mass above
6
Cancer Discovery
66 papers in training set
Top 0.4%
6.2%
7
Nature Medicine
125 papers in training set
Top 0.4%
4.8%
8
Communications Medicine
113 papers in training set
Top 1%
3.2%
9
Modern Pathology
22 papers in training set
Top 0.1%
3.2%
10
Advanced Intelligent Systems
11 papers in training set
Top 0.1%
2.4%
11
Advanced Science
286 papers in training set
Top 5%
1.7%
12
Proceedings of the National Academy of Sciences
2444 papers in training set
Top 28%
1.7%
13
Cell Reports Methods
165 papers in training set
Top 2%
1.3%
14
Science Translational Medicine
127 papers in training set
Top 2%
1.1%
15
Nature Aging
60 papers in training set
Top 1%
1.1%
16
PLOS Computational Biology
1863 papers in training set
Top 18%
1.0%
17
Scientific Reports
3612 papers in training set
Top 71%
1.0%
18
Science Advances
1243 papers in training set
Top 31%
0.8%
19
eLife
5828 papers in training set
Top 66%
0.8%
20
Cancer Research
130 papers in training set
Top 3%
0.8%
21
Cell Systems
201 papers in training set
Top 5%
0.6%
22
Genome Medicine
183 papers in training set
Top 6%
0.6%
23
Cell
431 papers in training set
Top 11%
0.6%