Spatially Context-Aware Transformers Facilitate Modeling-Based Anomaly Detection of Subtle Lesions in Brain MRI Images
Schwarz, J.; Will, L.; Wellmer, J.; Mosig, A.
Show abstract
The detection of small and subtle lesions in high-resolution 3D volumes is a highly relevant, yet far from solved task in biomedical imaging. We here address a specific task in detecting certain types of epileptogenic lesions through our novel semi-supervised spatially context aware transformer (SpyCAT) approach to anomaly detection. SpyCAT is modeling-based in the sense that it builds on specific assumptions that constitute what is normal and what constitues relevant deviations from normality. We explicitly use these assumptions to justify the inductive bias of our anomaly detection approach. The resulting SpyCAT system is patch-based and uses a transformer architecture to process discrete tokens obtained from a vector quantizing variational autoencoder, which produces counterfactual patches through full 3D convolutions of each patch. We evaluate our approach on the grounds of point-annotations of two subtypes of epileptogenic lesions, using validation measures that build on the Metrics Reloaded framework, showing that SpyCAT can reliably identify and localize the lesion types under consideration, and outperforms state-of-the-art reference methods.
Matching journals
The top 7 journals account for 50% of the predicted probability mass.
Similar papers in this journal
Similar papers in this journal
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.