LEAD: Layer-wise Expert-aligned Decoding for Faithful Radiology Report Generation

Ruixiao Yang; Yuanhe Tian; Di Dong; Xu Yang; Huiqi Li; Yan Song

cs.CLFeb 4, 2026

LEAD: Layer-wise Expert-aligned Decoding for Faithful Radiology Report Generation

Authors: Ruixiao Yang, Yuanhe Tian, Di Dong, Xu Yang, Huiqi Li, Yan Song

Organizations: Beijing Institute of Technology · Zhongguancun Academy · Zhongguancun Institute of Artificial Intelligence · University of Science and Technology of China

Abstract

Radiology Report Generation aims to produce accurate and coherent diagnostics from medical images. Although large vision-language models improve report fluency and accuracy, they still suffer from hallucinations by generating plausible pathological descriptions that are not supported by the input images. Existing methods primarily rely on external knowledge guidance to facilitate the alignment between generated text and visual information. However, these approaches often ignore the inherent decoding priors and vision-language alignment biases in pretrained models and lack robustness due to reliance on constructed guidance. In this paper, we propose Layer-wise Expert-aligned Decoding, a method that directly intervenes in the internal decoding process of large vision-language models. A pathology-specific expert module is designed to extract discriminative pathological features, which are then injected into each decoder layer through a gated mechanism. This architecture enables the large language model to progressively incorporate expert features during generation through a learned layer-wise gating function, thereby mitigating decoding biases and steering generation toward factual consistency. Experiments on multiple public datasets demonstrate that the proposed method improves clinical accuracy and factual consistency while maintaining competitive report generation quality.

Figures & tables

Fig. 1: Conceptual illustration of Layer-wise Expert-aligned Decoding (LEAD). Intrinsic language priors may affect standard VLMs and existing methods (top and middle), whereas LEAD introduces pathology expert signals during decoding. Adaptively injecting signals into intermediate representations at each decoder layer, LEAD guides the trajectory toward visual evidence and promotes factual consistency (bottom).

Fig. 2: Overview of LEAD for radiology report generation. Built upon Qwen3-VL, LEAD aggregates features from pathology classifiers according to confidence and projects the resulting expert embedding separately for each decoder layer. Context-Adaptive Gated Fusion (CAGF) integrates the embedding into token hidden states through gated residual fusion, as detailed on the right.

Dataset	Training	Validation	Test
CheXpert Plus	40,463	5,780	11,562
MIMIC-CXR	270,790	2,130	3,858
IU X-Ray	2,069	296	590

TABLE I: Dataset statistics for CheXpert Plus [ 26 ] , MIMIC-CXR [ 2 ] , and IU X-Ray [ 27 ] .

Method	Publication/Setting	Decoder	R-L	M	C	F1	GREEN
R2Gen † [ 3 ]	EMNLP20	Transformer	0.246	0.113	0.077	0.181	0.168
R2GenCMN † [ 30 ]	ACL21	Transformer	0.256	0.127	0.102	0.231	0.197
R2GenRL † [ 31 ]	ACL22	Transformer	0.186	0.101	0.012	0.196	–
XProNet † [ 32 ]	ECCV22	Transformer	0.265	0.146	0.121	0.259	–
CvT2DistilGPT2 † [ 34 ]	AIM23	GPT2	0.238	0.118	0.101	0.246	0.185
R2GenGPT-Llama2 [ 6 ]	Meta-Rad.23	Llama2-7B	0.239	0.132	0.122	0.216	0.217

TABLE II: Comparison with prior baselines on CheXpert Plus. Metrics include Natural Language Generation scores ( R-L : ROUGE-L, M : METEOR, C : CIDEr), the macro-averaged Clinical Efficacy score F1 , and the factual consistency score GREEN . Bold indicates the best performance overall, and underlined denotes the second-best result. Among prior methods,

\dagger

denotes results from published papers, whereas unmarked results are reproduced using publicly available implementations.

Method	Publication/Setting	M	R-L	C	F1
R2Gen [ 3 ]	EMNLP20	0.145	0.274	0.147	0.201
CvT2DistilGPT2 [ 34 ]	AIM23	0.139	0.265	0.128	0.180
R2GenGPT-Llama2 [ 6 ]	Meta-Rad.23	0.151	0.276	0.145	0.249
R2GenGPT-Llama3 [ 6 ]	Meta-Rad.23	0.153	0.269	0.158	0.252
Token-Mixer [ 33 ]	IEEE TMI24	0.148	0.275	0.161	0.243
SILC [ 39 ]	IEEE TMI24	0.150	0.277	0.163	0.232

TABLE III: Comparison with representative methods on MIMIC-CXR. Metrics are METEOR ( M ), ROUGE-L ( R-L ), CIDEr ( C ), and macro-averaged clinical F1 . Bold and underlining mark the best and second-best results, respectively. All baselines were reproduced with public implementations.

Dataset	Scale	Method	R-L	M	C	F1
CheXpert Plus	2B	Baseline	0.249	0.138	0.105	0.236
	2B	LEAD	0.258	0.140	0.138	0.242
	4B	Baseline	0.256	0.146	0.142	0.248
	4B	LEAD	0.262	0.148	0.153	0.260
	8B	Baseline	0.259	0.147	0.159	0.274
	8B	LEAD	0.275 ∗	0.154 ∗	0.199 ∗	0.346 ∗

TABLE IV: Scalability and cross-dataset generalization of LEAD across Qwen3-VL model scales [ 24 ] . “Baseline” denotes the corresponding vanilla Qwen3-VL model without the proposed LEAD modules, trained under the same training configuration as its LEAD counterpart. ∗ denotes a statistically significant improvement of LEAD over the corresponding baseline (

p<0.05

Fig. 3: Training configuration analysis on CheXpert Plus. The vanilla Qwen3-VL baseline and LEAD are compared under four training configurations. V and L denote the vision tower and LLM LoRA parameters, respectively. “+” and “-” indicate trainable and frozen parameters. The LEAD modules remain trainable in all LEAD configurations.

Dataset	Backbone	Method	R-L	M	C	F1
CheXpert Plus	Qwen3-VL [ 24 ]	Baseline	0.259	0.147	0.159	0.274
	Qwen3-VL [ 24 ]	LEAD	0.275	0.154	0.199	0.346
	Ministral3 [ 46 ]	Baseline	0.255	0.143	0.128	0.223
	Ministral3 [ 46 ]	LEAD	0.261	0.143	0.138	0.283
	InternVL3.5 [ 47 ]	Baseline	0.269	0.149	0.171	0.267
	InternVL3.5 [ 47 ]	LEAD	0.272	0.150	0.179	0.306

TABLE V: Effectiveness of LEAD across backbone families. Each backbone is evaluated with and without LEAD under identical final training settings on three datasets. “Baseline” denotes the vanilla model without LEAD modules.

Configuration			Metrics
Exp.	Proj.	Fuse.	R-L	M	C	F1
-	-	-	0.259	0.147	0.159	0.274
✓	-	-	0.265	0.141	0.160	0.275
✓	Shared	Gate	0.265	0.147	0.168	0.281
✓	Layer	Add	0.270	0.146	0.174	0.279
✓	Prompt	Input	0.272	0.147	0.182	0.291

TABLE VI: Component ablation study on CheXpert Plus. “Exp.” indicates the presence of the visual expert module. “Proj.” denotes the projection strategy, where “Shared” uses a single projection shared across all layers and “Layer” uses distinct layer-wise projections. “Fuse.” compares our context-adaptive “Gate” with direct “Add”.

Fig. 4: Qualitative comparison of generated reports. “Baseline” denotes the vanilla Qwen3-VL model trained under the same hybrid fine-tuning configuration as LEAD. Green text indicates correct pathological findings, red highlights factual errors, and yellow denotes missed findings.

Dataset	maAUC	miAUC	ACC
CheXpert Plus	0.779	0.893	0.879
MIMIC-CXR	0.761	0.860	0.865

TABLE VII: Pathology expert classification performance on CheXpert Plus and MIMIC-CXR. Macro AUROC (maAUC), micro AUROC (miAUC), and accuracy (ACC) assess the reliability of pathology signals injected into the decoder.

Explore similar work

Learning to See Locally and Align Clinically with Pathology Semantics for Radiology Report Generation
Jul 31, 2026Xuan Cuong NgoRadiology Report Generation Medical Vision-Language Models
Calibrated Confidence Expression for Radiology Report Generation
Mar 31, 2026David Bani-Harouni, Chantal Pellegrini, Julian Lüers +6Radiology Report Generation Medical Vision-Language Models
Computer Aided Medical Procedures, Technical University of Munich, Germany · Munich Center for Machine Learning (MCML), Germany · Department of Diagnostic and Interventional Radiology, TUM Klinikum rechts der Isar, Germany +3
RIHA: Report-Image Hierarchical Alignment for Radiology Report Generation
Apr 30, 2026Yucheng Chen, Yang Yu, Yufei Shi +3Radiology Report Generation Digitally Reconstructed Radiographs
MedVisAI Lab, Lee Kong Chian School of Medicine, Nanyang Technological University (NTU), and Centre of AI in Medicine, Singapore · Department of Computer Science and Engineering, the Chinese University of Hong Kong, Hong Kong SAR, China · Department of Machine Intellection, Institute for Infocomm Research (I2R), Agency for Science, Technology and Research, (A*STAR), Singapore

Jul 31, 2026, eess.IVJ/K move · Enter open · S save

Learning to See Locally and Align Clinically with Pathology Semantics for Radiology Report Generation

Xuan Cuong Ngo

Radiology Report Generation Medical Vision-Language Models Pathology Foundation Models Vision-Language Alignment Contrastive Learning

Open paper AskPDF arXiv

Recent radiology-adapted vision-language models have achieved strong performance on standard report generation benchmarks, yet their robustness and generalization remain constrained by imperfect alignment and correlation between visual and textual features. Existing methods connect image and text either implicitly through autoregressive report supervision or explicitly through contrastive learning. However, autoregressive supervision alone is insufficient to establish reliable image-text alignment, while contrastive learning can push apart unpaired reports that describe related pathologies simply because they are not paired with the same image. This is problematic in radiology, where different reports may share compatible pathology semantics rather than being true negatives. As a result, the learned representation may fail to organize images and reports around shared pathology concepts, causing the decoder to rely on pretrained language priors and generate clinically plausible reports that are not fully supported by radiographic evidence. To address this issue, we propose PALM, a pathology-aware alignment framework for radiology report generation. Instead of directly matching each image-report pair while separating all others, PALM aligns visual and textual features through shared pathology prototypes. These prototypes provide a clinically meaningful bridge between radiographic evidence and textual findings, allowing cases with similar pathology semantics to move toward common concepts without separating compatible cases. In addition, we introduce Masked Evidence Modeling to strengthen the image encoder sensitivity to local radiographic evidence by learning semantic changes caused by masked image regions. Experiments on MIMIC-CXR, IU X-Ray, and MIMIC-ABN show that PALM consistently improves both report generation and abnormality-focused robustness.

Abstract

Explore similar work

Learning to See Locally and Align Clinically with Pathology Semantics for Radiology Report Generation

Calibrated Confidence Expression for Radiology Report Generation

RIHA: Report-Image Hierarchical Alignment for Radiology Report Generation