Deep generative models for vessel segmentation in CT angiography of the brain

Automated vessel segmentation in brain CT angiography (CTA) remains challenging despite the potential benefit of applications. Expert acquisition of reference vessel segmentations is a laborious task. We propose an unsupervised generative deep learning approach that can be trained for vessel segmentation in brain CTA using a large dataset (n=908) of unlabelled brain CTAs and non-contrast enhanced CTs (NCCTs). Our unsupervised approach uses a conditional generative adversarial network (GAN) for CTA to NCCT translation by generating a contrast map that allows for automatic extraction of vessel segmentations. Furthermore, we propose a 3D Frangi filter-based loss function to enhance tubular structures in the contrast map to improve vessel segmentations. We used a hold-out test set of 9 CTA volumes with manually annotated reference segmentations. We compared our unsupervised approach with a state-of-the-art supervised nnUnet, trained and evaluated with test set using 9-fold nested cross-validation. Evaluation metrics included voxel-wise Dice similarity coefficient (DSC), true positive rate (TPR), and false positive rate (FPR). The DSC was 4% lower for the unsupervised approach (DSC: 0.74) compared to the supervised nnUnet (DSC: 0.78). Both the TPR and FPR were higher for the unsupervised approach (TPR: 0.75, FPR/1000 voxels:2.05) compared to the supervised nnUnet (TPR:0.71, FPR/1000 voxels:0.87). Hence, the quantitative results showed that our unsupervised method approaches a supervised state-of-the-art segmentation network. The results demonstrate that an unsupervised generative deep learning approach for the segmentation of intracranial vessels is feasible without laborious manual segmentations. HighlightsO_LITo train supervised segmentation models laborious manual segmentations are needed C_LIO_LIUnsupervised generative deep learning does not require manual segmentations C_LIO_LIOur unsupervised method combines L1, adversarial, and a novel Frangiloss C_LIO_LIVarying loss function combinations can reduce false positives or false negatives C_LIO_LIOur method approached the performance of a state-of-the-art supervised nnUnet C_LI