VPNet: Tractable and Explainable Classification with Probabilistic Circuits on ViT Features
Kusumoto, T.
Show abstract
We propose a novel explainable AI (XAI) model for classification tasks that can treat multi-modal and multidimensional information. Unlike traditional classification models based on convolutional neural networks or transformers, our approach combines probabilistic circuits with two vision transformers, DINO and CLIP, enabling probabilistic interpretability at the patch level, encoder level, and modality level. To demonstrate this capability, we developed a three-dimensional multimodal classification model for the brain age classification using 3D brain MRI volumes with T1 and T2 modalities. This model achieved 0.98 accuracy on the test domain while visualizing the patch-level probabilistic contribution. We expect that, in the future, scientists may uncover new findings using such explainable classification models by examining their decision processes. Our source code and training platform are available at https://github.com/tai2456/VPNet-Tractable-and-Explainable-Classification-with-Probabilistic-Circuits-on-ViT-Features, where users can train their own classification models on their own datasets.
Matching journals
The top 7 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Leveraging prior concept learning improves ability to generalize from few examples in computational models of human object recognition 95%
- Quantifying the Brain Predictivity of Artificial Neural Networks with Nonlinear Response Mapping 95%
- U-RISC: An Ultra-high Resolution EM Dataset Challenging Existing Deep Learning Algorithms 94%
Similar papers in this journal
Similar papers in this journal
- Generative AI Enables Medical Image Segmentation in Ultra Low-Data Regimes 94%
- Accurate recognition of colorectal cancer with semi-supervised deep learning on pathological images 93%
- Features fusion or not: harnessing multiple pathological foundation models using Meta-Encoder for downstream tasks fine-tuning 93%
Similar papers in this journal
- A Framework for Falsifiable Explanations of Machine Learning Models with an Application in Computational Pathology 95%
- Clinical Validation of Saliency Maps for Understanding Deep Neural Networks in Ophthalmology 95%
- STAMP: Simultaneous Training and Model Pruning for Low Data Regimes in Medical Image Segmentation 95%
Similar papers in this journal
- Graph attention-based fusion of pathology images and gene expression for prediction of cancer survival 95%
- Probabilistic Brain Extraction in MR Images via Conditional Generative Adversarial Networks 95%
- dFCExpert: Learning Dynamic Functional Connectivity Patterns with Modularity and State Experts 94%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.