CliQ-RRG: Clinical-Knowledge Guided Disease-aware Visual-Textual Alignment for QA-Style Radiology Report Generation

Chandravardhan Singh Raghaw; Shahid Shafi Dar; Arnav Nishith Sharma; Shrey Modi; Nagendra Kumar

cs.CLOct 9, 2026

CliQ-RRG: Clinical-Knowledge Guided Disease-aware Visual-Textual Alignment for QA-Style Radiology Report Generation

Authors: Chandravardhan Singh Raghaw, Shahid Shafi Dar, Arnav Nishith Sharma, Shrey Modi, Nagendra Kumar

Organizations: Indian Institute of Technology Indore, Indore, India · University of Texas at Dallas, USA · University of Wisconsin - Madison, USA

Abstract

Radiology reports are essential for diagnostic reasoning and patient care, yet their manual preparation is time-consuming and cognitively demanding. Automatic radiology report generation (RRG) offers a scalable alternative, but existing models often produce lengthy, unstructured narratives that overlook diagnostic cues and multi-view information. We present Clinical-Knowledge Guided Disease-aware Visual-Textual Alignment for QA-Style Radiology Report Generation (CliQ-RRG), a unified two-stage framework for interpretable and clinically grounded reporting. In Stage 1, CliQ-RRG employs a Disease-aware Visual-Textual Alignment module that aligns image and text representations using predicted disease embeddings, reinforced by a Prior-Guided Attention Module (PrAM) to capture multi-view dependencies across current and prior scans. In Stage 2, GraphRAG-guided retrieval augments intermediate textual representations with graph-grounded clinical evidence, and a large language model restructures them into concise, interpretable question-answer (QA) pairs with diagnostic summaries. Experiments on two public chest X-ray benchmarks demonstrate that CliQ-RRG consistently outperforms prior methods across both lexical and clinical metrics, generating accurate and clinically coherent QA-style radiology reports. Code is available at: https://github.com/chandravardhan-singh-raghaw/CliQ-RRG.

Figures & tables

Figure 1: Comparison with existing methods. Existing approaches generate unstructured reports, overlook prior scans, lack clinical knowledge, and exhibit limited alignment. In contrast, CliQ-RRG generates QA-style reports with clinical knowledge enrichment, disease-aware alignment, and prior-guided attention.

Figure 2: Overview of the proposed CliQ-RRG framework. Stage 1 : Disease-aware Visual–Textual Alignment aligns image and text representations using predicted disease embeddings, while the Prior-Guided Attention Module (PrAM) integrates multi-view information from current and prior scans. Stage 2 : Knowledge-Guided QA-Style Report Generation enriches the intermediate report with graph-connected clinical evidence retrieved from a medical knowledge graph and employs a large language model to produce concise QA pairs and a diagnostic summary.

Figure 3: Disease-aware Visual–Textual Contrast (Di-VTC). Positive visual–textual pairs with shared disease labels (green) are pulled closer, while unrelated pairs (red) are pushed apart, enforcing disease-consistent alignment in the shared embedding space.

Figure 4: Qualitative analysis of CliQ-RRG on MIMIC-CXR. Report comparison across models given input and prior scans, with ground truth (GT). Shared medical terms use consistent color coding, and Grad-CAM highlights anatomically and pathologically relevant regions.

Figure 5: Parameter sensitivity analysis on MIMIC-CXR showing the effect of generation loss weight ( $\lambda_{1}$ ), classification loss weight ( $\lambda_{2}$ ), and temperature ( $\tau$ ).

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Comparison of LLMs for QA restructuring under zero-shot, few-shot, and QLoRA fine-tuned settings on MIMIC-CXR and IU X-Ray using RadGraph-F1, CheXbert-F1, and BERTScore.

Radiology Report Generation

Radiology report generation uses medical images and clinical context to produce structured, clinically accurate findings and impressions.

See all 150 papers

Multimodal Large Language Model driven Radiology Report Generation with Clinical Knowledge Enhancement
Mar 11, 2024Miaojing Shi, Tianyu Cen, Zijie Yue +3Radiology Report Generation Radiology VLMs
College of Electronic and Information Engineering, Tongji University, Shanghai, China
RIHA: Report-Image Hierarchical Alignment for Radiology Report Generation
Apr 30, 2026Yucheng Chen, Yang Yu, Yufei Shi +3Radiology Report Generation Radiology VLMs
MedVisAI Lab, Lee Kong Chian School of Medicine, Nanyang Technological University (NTU), and Centre of AI in Medicine, Singapore · Department of Computer Science and Engineering, the Chinese University of Hong Kong, Hong Kong SAR, China · Department of Machine Intellection, Institute for Infocomm Research (I2R), Agency for Science, Technology and Research, (A*STAR), Singapore
Graph-Supervised Hierarchical Clinical Alignment for Radiology Report Generation with Large Language Models
Aug 25, 2026Yingshu Li, Yunyi Liu, Zhanyu Wang +4Radiology Report Generation Radiology VLMs
University of Sydney Sydney, Australia · University of Wollongong Wollongong, Australia · University of Adelaide Adelaide, Australia

Mar 11, 2024J/K move · Enter open · S save

Multimodal Large Language Model driven Radiology Report Generation with Clinical Knowledge Enhancement

Miaojing Shi, Tianyu Cen, Zijie Yue, Meng Wei+2

College of Electronic and Information Engineering, Tongji University, Shanghai, China

Radiology Report Generation Radiology VLMs Multimodal Large Language Models

Open paper AskPDF arXiv

Radiology report generation (RRG) has attracted significant attention due to its potential to reduce the workload of radiologists. The performance of current RRG approaches remains unsatisfactory against clinical standards. This paper introduces a novel RRG method, MLLM-RRG, that integrates multimodal large language models (MLLMs) with various types of clinical knowledge to generate accurate and comprehensive chest X-ray reports. Our method first designs a referring anatomical feature extractor that leverages anatomical knowledge to analyze different regions of the chest X-ray image and extract visual features without explicitly detecting regions. Next, based on the MLLM's decoder, we develop a multimodal report generator that leverages multimodal prompts constructed from dedicated visual features and textual instructions to produce the radiology report in an auto-regressive way. Finally, we introduce a disease-oriented clinical classification and alignment scheme in a multi-task learning manner to leverage disease knowledge to better preserve the clinical relevance among the generated reports. Once the model is trained, we also introduce a novel clinical quality reinforcement learning strategy to enhance the MLLM with report knowledge, further refining the tones of the generated reports towards radiologists. Extensive experiments on the MIMIC-CXR and IU X-Ray datasets demonstrate the superiority of our method over the state of the art. Our codes will be available at https://github.com/viscom-tongji/MLLM-RRG.

Abstract

Radiology Report Generation

Explore similar work

Multimodal Large Language Model driven Radiology Report Generation with Clinical Knowledge Enhancement

RIHA: Report-Image Hierarchical Alignment for Radiology Report Generation

Graph-Supervised Hierarchical Clinical Alignment for Radiology Report Generation with Large Language Models