Multi-stain modelling of histopathology slides for breast cancer prognosis prediction

BackgroundPathologic assessment of the established biomarkers using standard hematoxylin & eosin (H&E) and immunohistochemical (IHC) stained whole slide images (WSIs) is central in routine breast cancer diagnostics and contributes prognostic and predictive information that guides clinical decision-making. However, other than only aggregated protein-expression values from IHC WSIs, a spatial combination of histo-morphological information from IHC and H&E WSIs can potentially improve prognosis prediction in breast cancer patients. In this study, we aim to develop a deep learning-based risk-stratification method for breast cancer using routine H&E and IHC-stained histopathology WSIs from resected tumours. MethodsThis is a retrospective study including WSIs from surgical resected specimens from 945 patients from the South General Hospital in Stockholm. One H&E and four IHC (ER, PR, HER2, and Ki-67) stained sections were included from each patient, retrieved from the same tumour block. The IHC WSIs with the H&E WSI were registered, and corresponding images patches (tiles) were extracted for each image modality. Features from the registered tiles were extracted using two existing and publicly available histopathology foundation models (UNI and CONCH). Using the extracted features together with time-to-event data, we optimised an attention-based multiple instance learning (MIL) model using the Cox loss (negative partial log-likelihood loss) and recurrence-free survival (RFS) as the survival endpoint. ResultsUsing cross-validation we observed a prognostic performance with a C-index of 0.65 (95%CI: 0.56 - 0.72) for the risk score prediction using only H&E WSIs and UNI as the tile-level feature extractor. Combinations of H&E with one or more IHC modalities were subsequently evaluated, with the highest performance observed in the model combining the H&E and PR WSI data and the model combining all the stains, obtaining a C-index of 0.72 (95% CI: 0.65 - 0.79) and 0.72 (95% CI: 0.64 - 0.79) respectively. ConclusionMultiple stain modalities are used in routine breast cancer pathology, but has not been considered together for prognostic modelling. The results in this study suggests that models combining morphological features extracted by histopathology foundation models across multiple stain modalities can improve prognostic risk-stratification performance compared to single-modality models.