Brain-SAM: A SAM-based Model Tailored for Brain MRI Lesion Segmentation

AO_SCPLOWBSTRACTC_SCPLOWMagnetic resonance imaging (MRI) is a cornerstone of modern neuroimaging, where accurate segmentation of brain structures and lesions is essential for diagnosis, treatment planning, and clinical research. However, most current foundation models are trained on mixed-organ datasets, while the anatomical structures of the brain differ substantially from those of other organs such as the lungs and kidneys. As a result, these models often struggle to adapt to the distinctive characteristics of brain tissue. In this work, we present Brain-SAM, a model tailored for brain MRI segmentation. Brain-SAM extends the Segment Anything Model 2 (SAM2) framework by enabling the Hiera encoder to directly process 3D volumetric data and introducing a UNETR-inspired decoder for hierarchical feature decoding. The model preserves the interactive segmentation paradigm of SAM while also supporting fully automatic segmentation. Trained on multiple brain MRI datasets covering brain tumors, stroke, and epilepsy, Brain-SAM demonstrated superior performance to state-of-the-art methods. Compared with nnU-Net, it achieved Dice scores improvements of 22%, 9%, and 6% on epileptic lesions, brain metastases, and meningiomas, respectively. Notably, Brain-SAM showed clear advantages in small-lesion segmentation, achieving 15%-18% higher Dice compared with other strong baseline models. We believe that Brain-SAM may offer a useful pre-trained model for downstream brain MRI analysis tasks, and could contribute to future research and clinical applications.Our code and models are available at https://github.com/DLbrainsam/Brain-SAM.