Email this article Deep Learning Approaches for Automated Medical Image Diagnosis: A Focus on Explainability (XAI)
Abstract
Deep learning has transformed computer-aided medical image diagnosis with record-breaking performance on a range of tasks such as the detection of tumors, segmentation of lesions, and classification of diseases. However, the dominance of extremely complex neural architectures—most prominently convolutional neural networks (CNNs), vision transformers (ViTs), and future foundation models—has generated anxiety about their "black-box" status. The primary challenge is no longer whether artificial intelligence (AI) will match or even surpass clinicians in generating diagnostic decisions, but whether these models can be trusted in high-risk clinical practice. This review discusses explainable artificial intelligence (XAI) as the path to filling the trust deficit between technical innovation and medical adoption. We categorize XAI methods into model-specific methods, i.e., attention mechanisms and explainable architectures, and post-hoc methods such as SHAP, LIME, Grad-CAM, and counterfactual explanations, and examine critically their strengths and weaknesses in medical imaging. Beyond technical quality, the review emphasizes clinical utility, asking if explanations enhance decision-making, reveal biases, or enable human-in-the-loop processes. We further examine open issues such as reproducibility of explanation, absence of standard benchmarks, and growing need to adapt XAI frameworks to future architectures like diffusion and multimodal foundation models. By highlighting both progress and the long-standing gaps, this paper presents a path forward for aligning deep learning innovations with clinical trust, usability, and regulatory preparedness.
Full Text:
PDFReferences
Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., & Kim, B. (2018). Sanity checks for saliency maps. Advances in Neural Information Processing Systems, 31, 9505–9515.
Amann, J., Blasimme, A., Vayena, E., Frey, D., & Madai, V. I. (2022). Explainability for artificial intelligence in healthcare: A multidisciplinary perspective. BMC Medical Informatics and Decision Making, 22, 46. https://doi.org/10.1186/s12911-022-01766-5
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., … Herrera, F. (2020). Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion, 58, 82–115. https://doi.org/10.1016/j.inffus.2019.12.012
Arun, N., Gaw, N., Singh, P., Chang, K., Aggarwal, M., Chen, B., … Kalpathy-Cramer, J. (2021). Assessing the trustworthiness of saliency maps for localizing abnormalities in medical imaging. Radiology: Artificial Intelligence, 3(6), e200267. https://doi.org/10.1148/ryai.2021200267
Azizi, S., Mustafa, B., Ryan, F., Beaver, Z., Freyberg, J., Deaton, J., … Norouzi, M. (2023). Robust and generalizable vision transformers for medical image analysis. Nature Biomedical Engineering, 7, 1102–1117. https://doi.org/10.1038/s41551-023-01052-9
Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., … Zhou, Y. (2022). TransUNet: Transformers make strong encoders for medical image segmentation. Medical Image Analysis, 82, 102615. https://doi.org/10.1016/j.media.2022.102615
Doshi-Velez, F., & Kim, B. (2017). Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608. https://arxiv.org/abs/1702.08608
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., … Houlsby, N. (2021). An image is worth 16×16 words: Transformers for image recognition at scale. International Conference on Learning Representations (ICLR). https://arxiv.org/abs/2010.11929
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017). Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542, 115–118. https://doi.org/10.1038/nature21056
Ghassemi, M., Oakden-Rayner, L., & Beam, A. L. (2021). The false hope of current approaches to explainable artificial intelligence in health care. The Lancet Digital Health, 3(11), e745–e750. https://doi.org/10.1016/S2589-7500(21)00208-9
Ghosal, S., Banerjee, I., & Mitra, S. (2023). Counterfactual explanations in medical imaging: A survey. Artificial Intelligence in Medicine, 141, 102592. https://doi.org/10.1016/j.artmed.2023.102592
Holzinger, A., Carrington, A., & Müller, H. (2022). Measuring the quality of explanations: The system causability scale (SCS). KI-Künstliche Intelligenz, 36, 1–7. https://doi.org/10.1007/s13218-021-00741-0
Jetley, S., Lord, N. A., Lee, N., & Torr, P. H. (2018). Learn to pay attention. International Conference on Learning Representations (ICLR).
Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., & Sayres, R. (2018). Interpretability beyond feature attribution: Testing with concept activation vectors (TCAV). International Conference on Machine Learning (ICML), 2673–2682.
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., … van Ginneken, B. (2017). A survey on deep learning in medical image analysis. Medical Image Analysis, 42, 60–88. https://doi.org/10.1016/j.media.2017.07.005
Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30, 4765–4774.
Moor, M., Banerjee, O., Abad, Z. S. H., Krumholz, H. M., Leskovec, J., & Topol, E. J. (2023). Foundation models for generalist medical artificial intelligence. Nature, 616, 259–265. https://doi.org/10.1038/s41586-023-05881-4
Raghu, M., Unterthiner, T., Kornblith, S., Zhang, C., & Dosovitskiy, A. (2021). Do vision transformers see like convolutional neural networks? Advances in Neural Information Processing Systems, 34, 12116–12128.
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). “Why should I trust you?”: Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135–1144. https://doi.org/10.1145/2939672.2939778
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-CAM: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 618–626. https://doi.org/10.1109/ICCV.2017.74
Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Müller, K.-R. (2021). Explaining deep neural networks and beyond: A review of methods and applications. Proceedings of the IEEE, 109(3), 247–278. https://doi.org/10.1109/JPROC.2021.3060483
Shen, Y., Yang, Y., Guo, X., Xu, T., & Zhang, H. (2023). Deep learning in medical imaging: Progress, challenges, and future trends. Artificial Intelligence in Medicine, 135, 102523. https://doi.org/10.1016/j.artmed.2023.102523
Tjoa, E., & Guan, C. (2021). A survey on explainable artificial intelligence (XAI): Toward medical XAI. IEEE Transactions on Neural Networks and Learning Systems, 32(11), 4793–4813. https://doi.org/10.1109/TNNLS.2020.3027314
Tonekaboni, S., Joshi, S., McCradden, M. D., & Goldenberg, A. (2019). What clinicians want: Contextualizing explainable machine learning for clinical end use. Machine Learning for Healthcare Conference, 359–380.
Wu, M., Zhang, H., Luo, J., & Jiang, M. (2023). Evaluating stability and reliability of explainability methods in deep learning models for medical imaging. Scientific Reports, 13, 8754. https://doi.org/10.1038/s41598-023-35471-3
Yang, Z., Li, K., Zhao, W., & Wang, J. (2024). Benchmarking interpretability methods in deep medical image analysis: Stability, fidelity, and usability. Medical Image Analysis, 92, 103064. https://doi.org/10.1016/j.media.2024.103064
Yeh, C.-C., Hsieh, T.-Y., Lin, C.-Y., & Wang, W.-C. (2022). On the reproducibility of explainable AI methods in medical imaging. IEEE Access, 10, 70391–70405. https://doi.org/10.1109/ACCESS.2022.3188854
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016). Learning deep features for discriminative localization. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2921–2929. https://doi.org/10.1109/CVPR.2016.319. (ICLR).
Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., & Sayres, R. (2018). Interpretability beyond feature attribution: Testing with concept activation vectors (TCAV). International Conference on Machine Learning (ICML), 2673–2682.
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., … van Ginneken, B. (2017). A survey on deep learning in medical image analysis. Medical Image Analysis, 42, 60–88. https://doi.org/10.1016/j.media.2017.07.005
Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30, 4765–4774.
Moor, M., Banerjee, O., Abad, Z. S. H., Krumholz, H. M., Leskovec, J., & Topol, E. J. (2023). Foundation models for generalist medical artificial intelligence. Nature, 616, 259–265. https://doi.org/10.1038/s41586-023-05881-4
Raghu, M., Unterthiner, T., Kornblith, S., Zhang, C., & Dosovitskiy, A. (2021). Do vision transformers see like convolutional neural networks? Advances in Neural Information Processing Systems, 34, 12116–12128.
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). “Why should I trust you?”: Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135–1144. https://doi.org/10.1145/2939672.2939778
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-CAM: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 618–626. https://doi.org/10.1109/ICCV.2017.74
Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Müller, K.-R. (2021). Explaining deep neural networks and beyond: A review of methods and applications. Proceedings of the IEEE, 109(3), 247–278. https://doi.org/10.1109/JPROC.2021.3060483
Shen, Y., Yang, Y., Guo, X., Xu, T., & Zhang, H. (2023). Deep learning in medical imaging: Progress, challenges, and future trends. Artificial Intelligence in Medicine, 135, 102523. https://doi.org/10.1016/j.artmed.2023.102523
Tjoa, E., & Guan, C. (2021). A survey on explainable artificial intelligence (XAI): Toward medical XAI. IEEE Transactions on Neural Networks and Learning Systems, 32(11), 4793–4813. https://doi.org/10.1109/TNNLS.2020.3027314
Tonekaboni, S., Joshi, S., McCradden, M. D., & Goldenberg, A. (2019). What clinicians want: Contextualizing explainable machine learning for clinical end use. Machine Learning for Healthcare Conference, 359–380.
Wu, M., Zhang, H., Luo, J., & Jiang, M. (2023). Evaluating stability and reliability of explainability methods in deep learning models for medical imaging. Scientific Reports, 13, 8754. https://doi.org/10.1038/s41598-023-35471-3
Yang, Z., Li, K., Zhao, W., & Wang, J. (2024). Benchmarking interpretability methods in deep medical image analysis: Stability, fidelity, and usability. Medical Image Analysis, 92, 103064. https://doi.org/10.1016/j.media.2024.103064
Yeh, C.-C., Hsieh, T.-Y., Lin, C.-Y., & Wang, W.-C. (2022). On the reproducibility of explainable AI methods in medical imaging. IEEE Access, 10, 70391–70405. https://doi.org/10.1109/ACCESS.2022.3188854
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016). Learning deep features for discriminative localization. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2921–2929. https://doi.org/10.1109/CVPR.2016.319.
Refbacks
- There are currently no refbacks.