NeuroSymb-MRG: Differentiable Abductive Reasoning with Active Uncertainty Minimization for Radiology Report Generation
Authors: Rong Fu, Yiqing Lyu, Chunlei Meng, Muge Qi, Yabin Jin, Qi Zhao, Li Bao, Juntao Gao, +4 more
Organizations: University of Macau · Tsinghua University · Fudan University · Peking University · The First People’s Hospital of Foshan · Capital Medical University · Rutgers Cancer Institute, NJ, USA · NYU Langone Health, NY, USA · Techno International New Town, Kolkata, India
Automatic generation of radiology reports seeks to reduce clinician workload while improving documentation consistency. Existing methods that adopt encoder-decoder or retrieval-augmented pipelines achieve progress in fluency but remain vulnerable to visual-linguistic biases, factual inconsistency, and lack of explicit multi-hop clinical reasoning. We present NeuroSymb-MRG, a unified framework that integrates NeuroSymbolic abductive reasoning with active uncertainty minimization to produce structured, clinically grounded reports. The system maps image features to probabilistic clinical concepts, composes differentiable logic-based reasoning chains, decodes those chains into templated clauses, and refines the textual output via retrieval and constrained language-model editing. An active sampling loop driven by rule-level uncertainty and diversity guides clinician-in-the-loop adjudication and promptbook refinement. Experiments on standard benchmarks demonstrate consistent improvements in factual consistency and standard language metrics compared to representative baselines.
Figures & tables
Figure 1: Architectural overview of the NeuroSymb-MRG framework for transparent and clinically grounded radiology report generation. The pipeline initiates with Visual Perception , utilizing a self-supervised visual encoder fve to extract patch-level features X . In the Neuro-Symbolic Reasoning module, these features are mapped to probabilistic concept activations c^ , which serve as leaves for a Differentiable Logic Layer that composes multi-hop reasoning chains through soft logical operators (AND, OR, NOT). The Multi-Agent Orchestration hub employs a request-response protocol among specialized agents (Knowledge, Verifier, Reasoning) to cross-reference findings against a Medical Knowledge Graph (UMLS) . The Hybrid Generation phase merges rule-decoded clauses L with retrieval-augmented evidence R via a template inventory T and constrained LLM refinement. Finally, an Active Uncertainty Minimization loop utilizes predictive entropy and a Feedback Simulator to identify high-value cases for clinician adjudication, iteratively refining the promptbook and model parameters.
Method
IU X-ray [ 12 ]
MIMIC-CXR [ 11 ]
B-1
B-2
B-3
B-4
R-L
MTR
B-1
B-2
B-3
B-4
R-L
MTR
Show-Tell [ 1 ]
0.243
0.130
0.108
0.078
0.307
0.157
0.308
0.190
0.125
0.088
0.256
0.122
Transformer [ 2 ]
0.372
0.251
0.147
0.136
0.317
0.168
0.316
0.199
0.140
0.092
0.267
0.129
Att2in [ 31 ]
0.248
0.134
0.116
0.091
0.309
0.162
0.314
0.198
0.133
0.095
0.264
0.122
AdaAtt [ 3 ]
0.284
0.207
0.150
0.126
0.311
0.165
0.314
0.198
0.132
0.094
0.267
0.128
Up-Down [ 4 ]
–
–
–
–
–
–
0.317
0.195
0.130
0.092
0.267
0.128
Table 1: Quantitative comparison with representative prior methods on IU X-ray and MIMIC-CXR. Best and second-best values are in bold and underlined , respectively.
Model Variant
BLEU-1
BLEU-2
BLEU-3
BLEU-4
ROUGE-L
METEOR
Δ B-1
Full NeuroSymb-MRG
0.487
0.332
0.234
0.175
0.362
0.225
–
I. Core Architectural Components
w/o Self-Supervised Visual Encoder (Random Init)
0.412
0.262
0.175
0.124
0.298
0.168
-15.4%
w/o Concept Attention Head (Raw Patch Features)
0.401
0.251
0.168
0.118
0.288
0.159
-17.7%
w/o Differentiable Logic Layer (End-to-End MLP)
0.428
0.278
0.190
0.136
0.315
0.185
-12.1%
w/o Rule-Guided Decoder (Direct LLM Generation)
0.439
0.289
0.198
0.144
0.321
0.191
-9.9%
Table 2: Comprehensive ablation on architectural components and key mechanisms of NeuroSymb-MRG evaluated on MIMIC-CXR. The symbol ↓ indicates performance drop relative to the full model.
Training & Retrieval Strategy
BLEU-1
BLEU-2
BLEU-3
BLEU-4
ROUGE-L
METEOR
Full Strategy
0.487
0.332
0.234
0.175
0.362
0.225
Loss weighting: Fixed weights
0.471
0.318
0.222
0.165
0.348
0.213
Loss weighting: GradNorm
0.480
0.327
0.230
0.172
0.357
0.221
No rule supervision ( Lrule removed)
0.435
0.285
0.197
0.145
0.322
0.192
Active sampling: Random selection
0.463
0.310
0.217
0.161
0.342
0.208
Active sampling: Entropy only
0.475
0.323
0.228
0.170
0.353
0.217
Table 3: Ablation on training strategies and retrieval settings. Dynamic loss weighting and combined active sampling are most effective.
Logical Reasoning Configuration
BLEU-1
BLEU-2
BLEU-3
BLEU-4
ROUGE-L
METEOR
Full: Soft Tree (Prod. T-Norm + Prob. Sum + Gate)
0.487
0.332
0.234
0.175
0.362
0.225
Operator ablation: Min/Max T-norm / Conorm
0.479
0.325
0.228
0.170
0.355
0.219
Operator ablation: Łukasiewicz T-norm / Conorm
0.473
0.319
0.223
0.166
0.350
0.214
Operator ablation: Gödel T-norm / Conorm
0.468
0.315
0.219
0.162
0.345
0.210
Composition ablation: Fixed α=1.0 (AND only)
0.452
0.299
0.206
0.151
0.332
0.200
Composition ablation: Fixed α=0.0 (OR only)
0.461
0.308
0.214
0.158
0.340
0.207
Table 4: Ablation on the differentiable logic module and decoding strategies evaluated on MIMIC-CXR.
Radiologists generate diagnostic reports through iterative and selective revisiting of suspicious regions to refine their interpretations. Recent multimodal large language models (MLLMs) for radiology report generation (RRG) have shifted from text-only reasoning toward a ``Thinking-with-Images'' paradigm, incorporating visual evidence into the reasoning process. However, existing methods provide static visual evidence without a dynamic revisit mechanism during reasoning, neglecting how radiologists re-examine uncertain observations. To this end, we propose an Uncertainty-aware Revisit Reasoning MLLM (UR2-MLLM) framework that dynamically revisits uncertain regions during reasoning for RRG. UR2-MLLM is first equipped with uncertainty perception by training on an uncertainty-aware dataset. We then construct a multimodal reasoning trajectory dataset together with a detect-and-copy mechanism, which guides when and where to revisit. Finally, a visual grounding reward refines this behavior through reinforcement learning, aligning the revisited regions with corresponding anatomical structures. Experiments on MIMIC-CXR and IU-Xray show that UR2-MLLM achieves state-of-the-art performance, highlighting the value of uncertainty-aware visual revisit reasoning for reliable and clinically aligned report generation.
Yucheng Chen, Yang Yu, Jiazhou Zhou +5
MedVisAI Lab · Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore · Centre of AI in Medicine, Singapore +3
Radiology reports are essential for diagnostic reasoning and patient care, yet their manual preparation is time-consuming and cognitively demanding. Automatic radiology report generation (RRG) offers a scalable alternative, but existing models often produce lengthy, unstructured narratives that overlook diagnostic cues and multi-view information. We present Clinical-Knowledge Guided Disease-aware Visual-Textual Alignment for QA-Style Radiology Report Generation (CliQ-RRG), a unified two-stage framework for interpretable and clinically grounded reporting. In Stage 1, CliQ-RRG employs a Disease-aware Visual-Textual Alignment module that aligns image and text representations using predicted disease embeddings, reinforced by a Prior-Guided Attention Module (PrAM) to capture multi-view dependencies across current and prior scans. In Stage 2, GraphRAG-guided retrieval augments intermediate textual representations with graph-grounded clinical evidence, and a large language model restructures them into concise, interpretable question-answer (QA) pairs with diagnostic summaries. Experiments on two public chest X-ray benchmarks demonstrate that CliQ-RRG consistently outperforms prior methods across both lexical and clinical metrics, generating accurate and clinically coherent QA-style radiology reports. Code is available at: https://github.com/chandravardhan-singh-raghaw/CliQ-RRG.
Vision-language models (VLMs) have shown potential for automated radiology report generation, yet existing approaches rely on global embedding compression of volumetric data, often leading to hallucinated findings and limited anatomical grounding in 3D CT imaging. We introduce MedScribe, a hypothesis-driven framework that reformulates report generation as an iterative evidence acquisition process rather than a single-pass encoding task. MedScribe models reporting as a sequential decision process in which a large language model dynamically invokes pathology-specific diagnostic tools to extract localized volumetric features. These structured features are used to query a multidimensional retrieval space aligned with pathology-specific textual evidence. By explicitly accumulating quantitative evidence prior to synthesis, the framework enforces fine-grained grounding and reduces unsupported claims. Without task-specific fine-tuning, MedScribe improves clinical accuracy, factual consistency, and interpretability on CT-RATE and RadChestCT compared to state-of-the-art 2D and 3D VLMs, demonstrating the value of hypothesis-driven reasoning for reliable medical image reporting.
Giuseppe A. Orlando, Paolo Papotti, Maria A. Zuluaga +2
Inria Centre at Université Côte d’Azur Sophia Antipolis, France · EURECOM Biot, France · Dep. of Nuclear Medicine, Centre Antoine Lacassagne Nice, France Université Côte d’Azur, CNRS, Inserm, iBV