Quantifying Explainable AI Methods in Medical Diagnosis: A study in skin cancer

Deep learning models have shown substantial promise in assisting medical diagnosis, offering the potential to improve patient outcomes and reduce clinician workloads. However, the widespread adoption of these models in clinical practice has been hindered by concerns surrounding their trustworthiness, transparency, and interpretability. Addressing these challenges requires not only the development of explainable AI (xAI) techniques but also quantitative metrics to evaluate their effectiveness. This study presents a comprehensive framework for training, explaining, and quantitatively assessing deep learning models for skin cancer diagnosis. Leveraging the HAM10000 dataset of seven diagnostic skin lesion categories, multiple convolutional neural network architectures--including custom CNNs, DenseNet, MobileNet, and ResNet--were trained and optimized using augmentation, oversampling, and hyperparameter tuning. Following model training, explainability techniques such as SHAP, LIME, and Integrated Gradients were deployed to generate post hoc explanations. Critically, the primary contribution of this work is the quantitative evaluation of these explanation methods using metrics related to faithfulness, robustness, and complexity. All code, models, and results are publicly available, providing a reproducible pathway toward more trustworthy, explainable diagnostic tools.