TIM: Temporal Decoupling with Iterative Mutual-Refinement Model for Longitudinal Radiology Report Generation

Yiheng Dong; Yi Lin; Shilong Huang; Xiyan Yang; Xin Yang

cs.CVOct 19, 2026

TIM: Temporal Decoupling with Iterative Mutual-Refinement Model for Longitudinal Radiology Report Generation

Authors: Yiheng Dong, Yi Lin, Shilong Huang, Xiyan Yang, Xin Yang

Organizations: Huazhong University of Science and Technology, Wuhan, China · HKUST, Hong Kong, China

Abstract

Automatic radiology report generation (RRG) aims to translate medical images into diagnostic text, reducing radiologists' workload and standardizing clinical documentation. Nonetheless, existing approaches mainly focus on single-timepoint analysis and fail to capture temporal disease evolution across longitudinal examinations. While recent longitudinal RRG (LRRG) approaches incorporate historical data, they often combine images from different time points within a single representation space, leading to blurred semantics and inconsistent temporal reasoning. In this work, we propose a Temporal Decoupling with Iterative Mutual-Refinement Model (TIM), a two-stage framework that explicitly decouples spatial pathology from temporal progression and iteratively refines reports through mutual feedback. Stage I performs temporal-decoupled representation learning, separating temporal evolution patterns from disease-specific features and generating radiology reports for both prior and current studies. Stage II introduces a mutual report refinement mechanism that identifies diagnostic inconsistencies within prior reports and iteratively rectifies both prior and current reports through error-sensitive feedback. Experiments on the Longitudinal-MIMIC dataset demonstrate that TIM surpasses existing single-image and longitudinal baselines, achieving new state-of-the-art performance across both language and clinical metrics. Code is available at https://github.com/yihengd/TIM.

Figures & tables

Figure 1: Comparison between TIM and previous work. Previous methods typically use a single representation branch, limiting their ability to capture disease progression. In contrast, TIM decouples the longitudinal radiology report generation into static pathology recognition and dynamic progression modeling , allowing precise characterization of both prior and current pathological states, as well as their longitudinal evolution.

Figure 2: Overview of the proposed TIM. Our proposed model comprises two stages: (I) temporal-decoupled representation learning and (II) mutual report refinement. During Stage I, TIM generates dual-timepoint reports using static pathological recognition and dynamic progression modeling. In Stage II, TIM identifies errors in reports and iteratively performs mutual refinement, generating clinically accurate and temporally coherent reports.

Input	Method	Venue	NLG Metrics						CE Metrics
Input	Method	Venue	B-1	B-2	B-3	B-4	MTR	R-L	P	R	F1
Single Image	R2Gen [ 8 ]	EMNLP 2020	0.308	0.190	0.126	0.089	0.124	0.266	0.457	0.290	0.355
	R2GenCMN [ 7 ]	ACL 2021	0.328	0.203	0.135	0.096	0.133	0.273	0.512	0.359	0.422
	R2GenGPT [ 46 ]	Meta-Rad 2023	0.396	0.243	0.161	0.108	0.155	0.260	0.495	0.385	0.404
	PromptMRG [ 19 ]	AAAI 2024	0.384	0.229	0.149	0.104	0.146	0.261	0.517	0.453	0.439
	EKAGen [ 5 ]	CVPR 2024	0.405	0.245	0.156	0.105	0.154	0.264	0.472	0.404	0.411

Table 1: Experimental results of our model and previous methods on Longitudinal-MIMIC. We report the natural language generation (NLG) metrics, including BLEU-n (B-n), METEOR (MTR), and ROUGE-L (R-L), and clinical efficiency (CE) metrics, including precision (P), recall (R), and F1-score. The best results are bolded, and the second-best results are underlined.

Model	DPM	DRG	$\mathcal{L}_{spr}$	$\mathcal{L}_{dpm}$	NLG Metrics						CE Metrics
Model	DPM	DRG	$\mathcal{L}_{spr}$	$\mathcal{L}_{dpm}$	B-1	B-2	B-3	B-4	MTR	R-L	P	R	F1
Base	✗	✗	✗	✗	0.396	0.233	0.151	0.108	0.155	0.260	0.495	0.385	0.404
(a)	✓	✗	✗	✗	0.414	0.241	0.158	0.110	0.156	0.266	0.508	0.422	0.427
(b)	✓	✓	✗	✗	0.416	0.245	0.164	0.114	0.161	0.271	0.509	0.452	0.446
(c)	✓	✓	✓	✗	0.419	0.250	0.167	0.117	0.165	0.270	0.521	0.468	0.460
(d)	✓	✓	✗	✓	0.418	0.251	0.169	0.117	0.170	0.271	0.517	0.451	0.451

Table 2: Ablation study of Stage I . DPM: Decoupled Bilateral Representation Modeling. DRG: Dual-timepoint Report Generation mechanism.

\mathcal{L}_{spr}

: Static Pathology Recognition loss,

\mathcal{L}_{dpm}

: Dynamic Progression Modeling loss. The best results are in bold.

PDI	$\mathcal{L}_{\Delta}$	NLG Metrics			CE Metrics
PDI	$\mathcal{L}_{\Delta}$	B-1	B-4	R-L	P	R	F1
✗	✗	0.419	0.118	0.269	0.522	0.452	0.453
✓	✗	0.424	0.121	0.274	0.551	0.495	0.487
✓	✓	0.430	0.124	0.287	0.563	0.505	0.511

Table 3: Ablation study of Stage II . PDI: Prior Difference Information,

\mathcal{L}_{\Delta}

: Mutual-Refinement loss. The best results are in bold.

Figure 3: Case study. On the left part, we connect the attention areas of the perception frames to the corresponding sentences in the GT report. On the right part, the red, blue, and green fonts respectively indicate statements that are consistent with, absent from, and conflicting with GT. Besides, the underlined sentences describe progression and change.

Panel of Figure 4 — Figure 4: Visualization of report embeddings before and after mutual refinement. (a) t-SNE of embedding pairs. (b) Distribution of distance between GT and generated report embeddings.

Figure 5: Comparision between different LLMs.

Figure 6: Hyper-parameter Analysis. (Above) Performance with different numbers of perception frames. (Below) Results after different numbers of refinement iterations.

Panel of Figure 7 — Figure 7: Stage I hyperparameter analysis , including $\lambda_{1}$ and $\lambda_{2}$ in Eq. ( 8 ). (a) ROUGE-L of different $\lambda_{1}$ values. (b) F1-score of different $\lambda_{1}$ values. (c) ROUGE-L of different $\lambda_{2}$ values. (d) F1 of different $\lambda_{2}$ values.

Panel of Figure 8 — Figure 8: Stage II hyperparameter analysis , including $\beta$ in Eq. ( 11 ) and $\lambda_{\Delta}$ in Eq. ( 12 ). (a) ROUGE-L of different $\beta$ values. (b) F1-score of different $\beta$ values. (c) ROUGE-L of different $\lambda_{\Delta}$ values. (d) F1 of different $\lambda_{\Delta}$ values.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Model	F1-score
Model	Stable	Worsening	Improving	Avg.
TIM w/o Progression Branch	0.85	0.30	0.52	0.78
TIM w/o Progression Loss	0.89	0.31	0.58	0.81
TIM-stage I	0.89	0.33	0.61	0.82
TIM-stage II	0.91	0.35	0.65	0.84

Appendix

Table 4: Evaluation of Spatiotemporal Consistency, conducting an ablation study on the components related to dynamic modeling.

Figure 9: Specific Case Study 1. Red and green fonts indicate statements that are consistent with and conflicting with the ground truth, respectively. Blue numbers in the predicted observations mark classification errors.

Figure 10: Specific Case Study 2. Red and green fonts indicate statements that are consistent with and conflicting with the ground truth, respectively. Blue numbers in the predicted observations mark classification errors.

Figure 11: Specific Case Study 3. Red and green fonts indicate statements that are consistent with and conflicting with the ground truth, respectively. Blue numbers in the predicted observations mark classification errors.

Medical Condition	Source Literature	Annotation Prompt
Enlarged Cardiomediastinum	Klein et al. [ 39 ]	Enlarged cardiomediastinum on chest x-ray refers to the widening or abnormal enlargement of the combined cardiac and mediastinal contours on chest radiography.
Cardiomegaly	Siwik et al. [ 40 ] Cardinale et al. [ 6 ]	Cardiomegaly on chest x-ray refers to an abnormally enlarged cardiac silhouette, typically identified when the cardiac-thoracic ratio (CTR) exceeds 0.5, meaning the heart width occupies more than half of the thoracic cage width.
Lung Opacity	Turk et al. [ 43 ]	Lung opacity on chest x-ray refers to an area in the normally dark lung that appears whiter, hazy, or denser due to increased tissue density from the replacement of air by substances such as fluid, fibrosis, or other material.
Lung Lesion	Feyisa et al. [ 14 ]	Lung lesion on chest x-ray refers to pulmonary lesions presenting as localized nodules, masses, or cavitary abnormalities within the lung parenchyma, often associated with structural distortion of surrounding lung tissue.
Edema	Siwik et al. [ 40 ]	Pulmonary edema on chest x-ray refers to interstitial or alveolar fluid accumulation, often producing distortion in the hilar image, thickening of interlobar or interlobular fissures, and bilateral higher density opacities in central and basal lung areas.
Consolidation	Bankier et al. [ 3 ] Cardinale et al. [ 6 ]	Consolidation on chest x-ray refers to a homogeneous increase in attenuation caused by alveolar filling with fluid, pus, blood, or cells, resulting in homogeneous opacification of the lung parenchyma on radiographs, often accompanied by air bronchograms.

Appendix

Table 5: Examples of clinical descriptions used for disease-aware text construction.

Radiology Report Generation

Radiology report generation uses medical images and clinical context to produce structured, clinically accurate findings and impressions.

See all 151 papers

STRIVE: Multi-Agent Structured Temporal Reasoning with Integrated Verification for Longitudinal Radiology Report Generation
Aug 25, 2026Junyeong Maeng, Eunsong Kang, Heung-Il SukRadiology Report Generation Temporal Reasoning in Language Models
Department of Artificial Intelligence, Korea University, Seoul, Republic of Korea · Graduate School of Data Science, Kangwon National University, Chuncheon, Republic of Korea
Standardizing Longitudinal Radiology Report Evaluation via Large Language Model Annotation
Jan 23, 2026Xinyi Wang, Grazziela Figueredo, Ruizhe Li +1Medical Report Generation LLM-Assisted Annotation
School of Computer Science, The University of Nottingham, Nottingham, NG7 2RD, United Kingdom. · School of Medicine, The University of Nottingham, Nottingham, NG7 2RD, United Kingdom.
DRRG: A Discrete Diffusion Framework for Radiology Report Generation
Aug 25, 2026Shaoyang Zhoua, Yingshu Li, Yunyi Liu +4Radiology Report Generation Radiology VLMs
School of Electrical and Computer Engineering, The University of Sydney, Sydney, NSW, 2006, Australia · Department of Radiology, The First Affiliated Hospital of Soochow University, Suzhou, Jiangsu, 215006, China · School of Computer Science, The University of Adelaide, Adelaide, SA, 5005, Australia +1

Aug 25, 2026J/K move · Enter open · S save

STRIVE: Multi-Agent Structured Temporal Reasoning with Integrated Verification for Longitudinal Radiology Report Generation

Junyeong Maeng, Eunsong Kang, Heung-Il Suk

Department of Artificial Intelligence, Korea University, Seoul, Republic of Korea · Graduate School of Data Science, Kangwon National University, Chuncheon, Republic of Korea

Radiology Report Generation Temporal Reasoning in Language Models Multi-Agent Reasoning RL for Language Model Reasoning

Open paper AskPDF arXiv

Longitudinal radiology report generation (LRRG) requires identifying both current findings and their changes relative to a prior study. Existing methods jointly model diagnosis, attribute estimation, temporal comparison, and language generation within implicit representations, which can cause task interference, obscure the evidence underlying each decision, and limit error traceability. They also model progression states as independent labels, ignoring their ordered structure and thus treating missed changes and direction reversals equally. We present STRIVE, Multi-Agent Structured Temporal Reasoning with Integrated Verification for LRRG, which decomposes clinical reasoning into specialized Diagnosis, Attribute, and Temporal Change Agents that produce explicit intermediate evidence. In particular, the Temporal Change Agent is further post-trained using Progression-Aware GRPO, a verifiable, shaped reward that assigns partial credit to direction-preserving errors while scoring direction reversals lowest. STRIVE performs verification at two stages: a deterministic Consistency Gate reconciles the agent outputs before report generation, and a Validation Agent checks whether the generated report is supported by the aggregated clinical evidence. On Longitudinal-MIMIC, STRIVE attains the best clinical efficacy among recent methods and more than doubles Longitudinal Change Concordance (LCC), a measure of temporal agreement with the reference report, over the strongest baseline.

Abstract

Radiology Report Generation

Explore similar work

STRIVE: Multi-Agent Structured Temporal Reasoning with Integrated Verification for Longitudinal Radiology Report Generation

Standardizing Longitudinal Radiology Report Evaluation via Large Language Model Annotation

DRRG: A Discrete Diffusion Framework for Radiology Report Generation