Characterization of tumor heterogeneity through segmentation-free representation learning
Tan, J.; Le, H.; Deng, J.; Liu, Y.; Hao, Y.; Hollenberg, M.; Liu, W.; Wang, J. M.; Xia, B.; Ramaswami, S.; Mezzano, V.; Loomis, C.; Murrell, N.; Moreira, A. L.; Cho, K.; Pass, H. I.; Wong, K.-K.; Ban, Y.; Neel, B. G.; Tsirigos, A.; Fenyo, D.
Show abstract
The interaction between tumors and their microenvironment is complex and heterogeneous. Recent developments in high-dimensional multiplexed imaging have revealed the spatial organization of tumor tissues at the molecular level. However, the discovery and thorough characterization of the tumor microenvironment (TME) remains challenging due to the scale and complexity of the images. Here, we propose a self-supervised representation learning framework, CANVAS, that enables discovery of novel types of TMEs. CANVAS is a vision transformer that directly takes high-dimensional multiplexed images and is trained using self-supervised masked image modeling. In contrast to traditional spatial analysis approaches which rely on cell segmentations, CANVAS is segmentation-free, utilizes pixel-level information, and retains local morphology and biomarker distribution information. This approach allows the model to distinguish subtle morphological differences, leading to precise separation and characterization of distinct TME signatures. We applied CANVAS to a lung tumor dataset and identified and validated a monocytic signature that is associated with poor prognosis.
Matching journals
The top 6 journals account for 50% of the predicted probability mass.
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.