A Unified Explainable AI Framework for Multimodal Chest X-Ray and Clinical Note Diagnosis

SHOBHANJALY P NAIR; LAVANYA JAYARAMAN

doi:10.63363/aijfr.2026.v07i05.7978

A Unified Explainable AI Framework for Multimodal Chest X-Ray and Clinical Note Diagnosis

Author(s)	Ms. SHOBHANJALY P NAIR, Ms. LAVANYA JAYARAMAN
Country	India
Abstract	Chest radiography is the most common first-line imaging modality for diagnosing respiratory disease, but deep learning classifiers built for this task are largely opaque, which restricts their clinical adoption. This paper proposes a multimodal explainable-AI (XAI) framework that couples a DenseNet-121 convolutional network for chest X-ray classification with a RadBERT transformer for clinical-note analysis inside a single interpretability pipeline. Grad-CAM produces spatial saliency maps for the imaging branch, while SHAP-based token attribution explains the predictions of the language branch, whose pathology label is obtained through cosine similarity against class-prototype embeddings. On a representative case, the text branch assigned an 80.8% similarity score to pneumonia and highlighted clinical indicators such as fever and consolidation alongside demographic terms such as age and diabetes history. Rather than reporting a marginal accuracy improvement, the central contribution is a cross-modal consistency check that compares the imaging and textual explanations for the same patient and automatically surfaces discrepancies — for example, a mismatch between the anatomical location implicated by the image and the one described in the note — that neither modality reveals in isolation. Outputs are exposed through a standardized JSON schema together with a natural-language summary to support clinician review.
Keywords	Explainable Artificial Intelligence, Grad-CAM, SHAP, DenseNet-121, RadBERT, Multimodal Learning, Chest X-Ray Classification, Clinical Decision Support
Field	Computer Applications
Published In	Volume 7, Issue 5, September-October 2026
Published On	2026-09-27
DOI	https://doi.org/10.63363/aijfr.2026.v07i05.7978

View / Download PDF File

E-ISSN 3048-7641

doi

CrossRef DOI is assigned to each research paper published in our journal.

AIJFR DOI prefix is
10.63363/aijfr

Downloads

Research Paper Format Copyright Permission Form and Undertaking Form Cover Page Vol 7 Isu 6 Cover Page Vol 7 Isu 4 Cover Page Vol 7 Isu 3

All research papers published on this website are licensed under Creative Commons Attribution-ShareAlike 4.0 International License, and all rights belong to their respective authors/researchers.

CC-BY-SA

About AIJFR Fees & Payment Current Issue Publication Archive	Submit Research Paper Track Submission Status Publication Guidelines Publication Ethics Peer Review & Plagiarism	Join as a Reviewer Editors & Reviewers Reviewer Referral Program Get Reviewer Membership Certi.	Website/Journal Policies Usage Policy Content Policies Privacy Policy

Contact Us		+91-9898-79-39-59	editor@aijfr.com

Advanced International Journal for Research

A Widely Indexed Open Access Peer Reviewed Multidisciplinary Bi-monthly Scholarly International Journal

A Unified Explainable AI Framework for Multimodal Chest X-Ray and Clinical Note Diagnosis

Share this