Multimodal Multiple Instance Learning for Fibroepithelial Tumor Diagnosis in Breast Ultrasound
Abir, F. F.; Ali, M.; Hossain, M. S.; Holt, L.; Chamberlain, V.; Shern, T.; Ozmen, T.; Brattain, L.
Show abstract
Fibroepithelial breast lesions include fibroadenomas (FAs) and phyllodes tumors (PTs), which are difficult to distinguish preoperatively due to their overlapping clinical and ultrasound features. While FAs are typically managed with observation, PTs require complete surgical excision with negative margins due to their risk of local recurrence and malignant progression. Because core needle biopsy can be inconclusive in about 15% of patients, some patients undergo unnecessary surgical procedures. Breast ultrasound (BUS) is a widely available, noninvasive imaging modality, but visual interpretation alone is limited by the overlap in lesion appearance. Artificial intelligence (AI)-based BUS offers the potential to improve the preoperative differentiation of PTs from FAs, reducing unnecessary operations while ensuring appropriate treatment for patients with PTs. Most deep learning approaches operate at the image level, whereas clinical diagnosis is made at the patient level by integrating multiple ultrasound views with clinical context. We propose a patient-level multimodal multiple instance learning (MIL) framework for fibroepithelial tumor diagnosis in breast ultrasound. Each patient is represented as a bag of BUS images encoded by an ultrasound foundation model (USFM) and aggregated using attention-based MIL to produce an image-derived patient score. Clinical variables, including age, lesion size, and echogenicity, are modeled using a gradient-boosted clinical learner. The image and clinical scores are then combined using logistic score fusion. In patient-level 5-fold cross-validation, logistic score fusion achieved 0.780 AUC-ROC for classification, improving over clinical feature-only gradient boosting (GB) by +0.033 (95\% CI [+0.013, +0.053], p = 0.001), image-only MIL by +0.081 (95\% CI [+0.042, +0.119], p < 0.001), and MLP-based end-to-end multimodal fusion by +0.035 (95\% CI [+0.003, +0.067], p = 0.030). These results suggest that patient-level score fusion can combine complementary ultrasound and clinical evidence for PTs vs FAs.
Matching journals
The top 12 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Automated detection of the HER2 gene amplification status in Fluorescence in situ hybridization images for the diagnostics of cancer tissues 94%
- Incremental Learning Approach for Semantic Segmentation of Skin Histology Images 94%
- Automated and Manual Quantification of Tumour Cellularity in Digital Slides for Tumour Burden Assessment 93%
Similar papers in this journal
- Model uncertainty estimates for deep learning mammographic density prediction using ordinal and classification approaches 94%
- Mammographic density assessed using deep learning in women at high risk of developing breast cancer: the effect of weight change on density 90%
- Breast density prediction from low and standard dose mammograms using deep learning: effect of image resolution and model training approach on prediction quality 90%
Similar papers in this journal
- Artificial Intelligence System Reduces False-Positive Findings in the Interpretation of Breast Ultrasound Exams 95%
- An Ensembled Deep Learning Model Outperforms Human Experts in Diagnosing Biliary Atresia from Sonographic Gallbladder Images 92%
- Features fusion or not: harnessing multiple pathological foundation models using Meta-Encoder for downstream tasks fine-tuning 90%
Similar papers in this journal
- Unmasking the tissue microecology of ductal carcinoma in situ with deep learning 91%
- Predicting neoadjuvant chemotherapy benefit using deep learning from stromal histology in breast cancer 89%
- Investigating the relationship between breast cancer risk factors and an AI-generated mammographic texture feature in the Nurses' Health Study II 88%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.