Multimodal Fusion of Pathology Free-Text and Clinical Data Enhances Complication-Risk Discrimination After Implant-Based Breast Reconstruction

Implant-based breast reconstruction is the most common surgical option following mastectomy for breast cancer. Despite its prevalence, up to one-third of patients develop complications within two years. Existing machine-learning models for predicting the complications rely solely on structured clinical data, over-looking prognostic information in narrative pathology reports. Recent advances in large language models (LLMs) enable extraction of numeric and semantic information from clinical text, offering opportunities to improve predictive performance and interpretability. We developed a fully on-premises, open-source model that extracts numeric morphometrics and contextual embeddings from free-text pathology reports, and fuses them with 63 structured variables via a CLIP-style dual encoder. In a single-center cohort of 963 patients (Jan 2007-Jan 2022), the multimodal model improved composite-complication risk discrimination (AUROC from 0.691 with logistic regression, increased to 0.740 with clinical features, and further improved to 0.764 with the addition of pathology report text features; p=0.027) and enhanced sensitivity and positive predictive value at clinical thresholds. The automated extraction module we developed for numeric morphometrics (e.g. mastectomy-specimen weight) from free-text pathology reports achieved an accuracy of 96.3%. SHAP analyses confirmed established risk factors--expander-to-implant interval, body-mass index, and total mastectomy weight--as dominant drivers. In subgroup analyses, model performance remained robust, with particularly strong discrimination observed among specific populations, such as shorter expander-to-implant interval (EII), the AUROC reached 0.796 and accuracy was 0.799. These results show that on-premises, open-source LLMs reliably extract and fuse textual and structured clinical features to achieve clinically meaningful gains in predicting complications after implant-based breast reconstruction. While traditional models are constrained by the limited scope of structured variables, pathology text--when analyzed with modern language models--adds new, clinically relevant signals. Even modest statistical gains yield more accurate identification of high-risk patients, potentially informing surgical planning, patient counseling, and postoperative follow-up. These findings demonstrate that privacy-preserving language models, when integrated with contrastive multimodal alignment, can unlock prognostic information embedded in narrative pathology reports and enable interpretable, patient-level decision support. This interpretable, privacy-preserving multimodal framework offers a generalizable approach for enhancing risk prediction and clinical decision-making across surgical oncology. Significance StatementThis study demonstrates that multimodal fusion of pathology free-text and structured clinical data, enabled by on-premises large language models, improves complication-risk discrimination after implant-based breast reconstruction. The approach uncovers prognostic signals hidden in narrative reports, offering an interpretable, privacy-preserving framework for precision risk prediction in surgical oncology.