Integration of Digital Histology and Gene Expression to Predict Treatment-Specific Responses in Breast Cancer

Deep learning models applied to digital histology can predict gene expression signatures (GES) and offer a low-cost, rapidly available alternative to molecular testing at the time of diagnosis. We optimized transformer-based models to infer GES results and applied this approach to pre-treatment H&E-stained biopsies from 1,940 breast cancer patients treated with neoadjuvant chemotherapy in clinical trial and real-world cohorts. The most predictive histology-derived GES for pathologic complete response (pCR) in the I-SPY2 trial was validated in four external cohorts: CALGB 40601, CALGB 40603, a trial of durvalumab plus CT, and standard-of-care CT-treated patients from the University of Chicago. Among HER2-negative patients, a transformer-based model trained using a signature composed of estrogen-regulated genes, proliferation, apoptosis, and interferon response genes predicted pCR with an AUC of 0.794, outperforming models based on clinical features alone (AUC 0.704, p = 0.001), pathologist TIL assessment, and a model trained directly to predict response from I-SPY2 cases. Tertiles of this signature stratify patients into clinically relevant groups with increasing likelihood of complete response, with pCR rates [≥]50% in the top tertile regardless of treatment or hormone receptor status. Additional transformer-based signature models predicted response to specific therapies (but not chemotherapy alone), including a HER2 signaling signature in IO-treated patients, and a claudin-low signature in bevacizumab treated patients. In HER2- cohorts with available gene expression data and histology, models trained on expression data performed similarly to digital histology predictions, but the combination of gene expression and histology outperformed histology alone. These findings suggest that histology-based GES provides additive information to RNA sequencing data and can inform precision treatment selection across breast cancer subtypes.