Deep Radiomics in 18F-FDG-PET/CT for Survival Prediction in Multiple Myeloma: Novel Embeddings Using A Foundational Models Memory
Guinea-Perez, J.; Uribe, S.; Peluso, S.; Castellani, G.; Nanni, C.; Alvarez, F.
Show abstract
PurposeTo test whether internal memory states from a medical founda-tional segmentation model can serve as compact, mask-aware embeddings for predicting progression-free survival (PFS) in multiple myeloma (MM) from whole-body [18F]FDG PET/CT, and how late fusion of PET, CT, and clinical data enhances prognostic performance. MethodsWe analyzed 227 newly diagnosed MM patients with PET/CT and clinical data. For two regions of interest (spine-dilated and full skeleton), we prompted MedSAM2 slice-wise using mask-derived bounding boxes and cached the final spatio-temporal memory tensor per modality. We compared two downsampling strategy to obtain per-study embeddings: channelxmemory averaging with a small CNN head, and depth-attention pooling. PET and CT embeddings were combined by late fusion and passed to a DeepSurv head. We evaluated image-only and multimodal (image+clinical) models with stratified 5-fold cross-validation. The primary endpoint was Harrells c-index (mean {+/-} SE across folds). ResultsImage-only models using the averaging downsampler achieved up to 0.659 {+/-} 0.015 c-index (PET, spine-dilated), comparable to baseline ra-diomics results. Multimodal models improved discrimination to 0.710{+/-}0.032 (CT, spine-dilated), with similar performance for other PET/CT+clinical variants (0.703-0.710), improving clinical-only baselines [~] 6.5%. Averaging consistently outperformed depth-attention; concatenation and gated fusion performed comparably. PET outperformed CT within the same mask in image-only settings. ConclusionMask-aware memory embeddings extracted from a founda-tional segmentation model provide effective, data-efficient imaging biomark-ers for MM PFS and, when fused with routine clinical covariates, significantly improve risk stratification over clinical-only or radiomics baselines. This of-fers a practical path to prognostic modeling on small medical cohorts without feature design.
Matching journals
The top 5 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Generative AI Enables Medical Image Segmentation in Ultra Low-Data Regimes 96%
- Segmenting functional tissue units across human organs using community-driven development of generalizable machine learning algorithms 94%
- IMC-Denoise: a content aware denoising pipeline to enhance Imaging Mass Cytometry 93%
Similar papers in this journal
- Encrypted federated learning for secure decentralized collaboration in cancer image analysis 95%
- Spatial Transcriptomics Expression Prediction from Histopathology Based on Cross-Modal Mask Reconstruction and Contrastive Learning 93%
- Clinical Validation of Saliency Maps for Understanding Deep Neural Networks in Ophthalmology 93%
Similar papers in this journal
- Deep learning inference of cell type-specific gene expression from breast tumor histopathology 93%
- A Deep Learning Model for Molecular Label Transfer that Enables Cancer Cell Identification from Histopathology Images 93%
- Generalizing AI-driven Assessment of Immunohistochemistry across Immunostains and Cancer Types: A Universal Immunohistochemistry Analyzer 92%
Similar papers in this journal
- STPath: A Generative Foundation Model for Integrating Spatial Transcriptomics and Whole Slide Images 95%
- Understanding the robustness of vision-language models to medical image artefacts 94%
- Biologically-informed deep neural networks provide quantitative assessment of intratumoral heterogeneity in post-treatment glioblastoma 94%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.