Overlooked Trustworthiness of Explainability in Medical AI
Zhang, J.; Chao, H.; Kalra, M. K.; Wang, G.; Yan, P.
Show abstract
While various methods have been proposed to explain AI models, the trustworthiness of the generated explanation received little examination. This paper reveals that such explanations could be vulnerable to subtle perturbations on the input and generate misleading results. On the public CheXpert dataset, we demonstrate that specially designed adversarial perturbations can easily tamper saliency maps towards the desired explanations while preserving the original model predictions. AI researchers, practitioners, and authoritative agencies in the medical domain should use caution when explaining AI models because such an explanation could be irrelevant, misleading, and even adversarially manipulated without changing the model output. AI researchers, practitioners, and authoritative agencies in the medical domain should use caution when explaining AI models because such an explanation could be irrelevant, misleading, and even adversarially manipulated without changing the model output.
Matching journals
The top 6 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Toward Understanding COVID-19 Pneumonia: A Deep-learning-based Approach for Severity Analysis and Monitoring the Disease 94%
- Assisting Scalable Diagnosis Automatically via CT Images in the Combat against COVID-19 94%
- Tracking And Predicting COVID-19 Radiological Trajectory Using Deep Learning On Chest X-Rays: Initial Accuracy Testing 94%
Similar papers in this journal
- BenchXAI: Comprehensive Benchmarking of Post-hoc Explainable AI Methods on Multi-Modal Biomedical Data 94%
- Deep learning ensemble for abdominal aortic calcification scoring from lumbar spine X-ray and DXA images 93%
- MultiHeadGAN: A Deep Learning Method for Low Contrast Retinal Pigment Epithelium Cells Segmentation in Fluorescent Flatmount Microscopy Images 93%
Similar papers in this journal
- An Accurate and Explainable Deep Learning System Improves Interobserver Agreement in the Interpretation of Chest Radiograph 93%
- The tempest in a cubic millimeter: Image-based refinements necessitate the reconstruction of 3D microvasculature from a large series of damaged alternately-stained histological sections 93%
- SN-FPN: Self-attention Nested Feature Pyramid Network for Digital Pathology Image Segmentation 92%
Similar papers in this journal
- Mistic: an open-source multiplexed image t-SNE viewer 91%
- RATING: Medical-knowledge-guided rheumatoid arthritis assessment from multimodal ultrasound images via deep learning 91%
- Structuring clinical text with AI: old vs. new natural language processing techniques evaluated on eight common cardiovascular diseases 90%
Similar papers in this journal
- Phase Recognition in Contrast-Enhanced CT Scans based on Deep Learning and Random Sampling 94%
- SCU-Net: A deep learning method for segmentation and quantification of breast arterial calcifications on mammograms 92%
- Necessity and Impact of Specialization of Large Foundation Model for Medical Segmentation Tasks 92%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.