Personalized Longitudinal Medical Report Generation via Temporally-Aware Federated Adaptation

He Zhu; Ren Togo; Takahiro Ogawa; Kenji Hirata; Minghui Tang; Takaaki Yoshimura; Hiroyuki Sugimori; Noriko Nishioka

cs.CVFeb 23, 2026

Personalized Longitudinal Medical Report Generation via Temporally-Aware Federated Adaptation

Authors: He Zhu, Ren Togo, Takahiro Ogawa, Kenji Hirata, Minghui Tang, Takaaki Yoshimura, Hiroyuki Sugimori, Noriko Nishioka,

Organizations: Hokkaido University

Abstract

Longitudinal medical report generation is clinically important yet remains challenging due to strict privacy constraints and the evolving nature of disease progression. Although federated learning (FL) enables collaborative training without data sharing, existing FL methods largely overlook longitudinal dynamics by assuming stationary client distributions, making them unable to model temporal shifts across visits or patient-specific heterogeneity-ultimately leading to unstable optimization and suboptimal report generation. We introduce Federated Temporal Adaptation (FTA), a federated setting that explicitly accounts for the temporal evolution of client data. Building upon this setting, we propose FedTAR, a framework that integrates demographic-driven personalization with time-aware global aggregation. FedTAR generates lightweight LoRA adapters from demographic embeddings and performs temporal residual aggregation, where updates from different visits are weighted by a meta-learned temporal policy optimized via first-order MAML. Experiments on J-MID (1M exams) and MIMIC-CXR demonstrate consistent improvements in linguistic accuracy, temporal coherence, and cross-site generalization, establishing FedTAR as a robust and privacy-preserving paradigm for federated longitudinal modeling.

Figures & tables

Panel of Figure 1 — Figure 1: Visualization of semantic structure evolution in medical reports. All data are derived from the J-MID dataset. (a) Ridgeline kernel-density plots of LDA-derived topic components for annual reports (2018–2024) from one institution. (b) T-SNE embedding of TF-IDF vectors for reports from five institutions.

Panel of Figure 2 — Figure 2: Overview of the problem and our approach. (a) Client- and temporal-drift challenges in federated longitudinal medical report generation. (b) The proposed FedTAR framework.

	BLEU-1	BLEU-2	BLEU-3	BLEU-4	ROUGE-L	CIDEr
FedAvg [ 24 ]	38.40	24.68	16.16	10.98	28.54	31.70
FedProx [ 21 ]	38.32	24.48	15.95	11.00	28.58	31.62
SCAFFOLD [ 18 ]	35.40	22.38	14.80	10.10	27.12	24.82
FedAdam [ 31 ]	38.26	24.54	16.05	10.93	28.61	30.62
FedYogi [ 31 ]	35.39	22.37	13.72	10.46	27.91	24.79
DRFA [ 8 ]	36.80	24.02	16.63	11.60	28.75	29.51

Table 1: Comparison of our proposed method (FedTAR) with standard FL baselines on the chest CT report generation task. Evaluation metrics include BLEU-

n

(

n

=1–4), ROUGE-L, and CIDEr, which collectively reflect linguistic precision, recall, and content relevance. FedTAR achieves consistent improvements, demonstrating the effectiveness of demographics- and temporally-aware adaptation.

	BLEU-1	BLEU-2	BLEU-3	BLEU-4	ROUGE-L	CIDEr
FedTAR w/o GMM	40.38	25.08	17.59	12.07	28.65	41.56
FedTAR w/o temporal	38.83	25.82	17.28	11.21	29.50	41.88
FedTAR	40.08	25.94	17.80	12.40	29.54	42.80

Table 2: Ablation study evaluating the contributions of the Gaussian Mixture Model (GMM)-based demographics embedding and the temporal residual aggregation module. Removing either component leads to consistent performance degradation across all metrics, confirming their necessity for capturing patient-specific heterogeneity and longitudinal dynamics.

	CE-P	CE-R	CE-F1	BLEU-1	BLEU-2	BLEU-3	BLEU-4	ROUGE-L	CIDEr
FedAvg [ 24 ]	68.58	22.63	28.43	29.16	19.49	14.36	11.00	35.77	37.54
FedProx [ 21 ]	69.83	24.59	30.26	27.16	17.80	12.85	9.70	34.11	37.38
SCAFFOLD [ 18 ]	44.58	22.6	21.47	28.00	17.23	14.21	9.86	38.05	99.11
FedAdam [ 31 ]	58.74	25.95	31.80	39.17	28.56	22.42	18.39	43.08	97.07
FedYogi [ 31 ]	52.02	23.36	29.90	35.55	22.57	18.94	13.84	37.03	94.11
DRFA [ 8 ]	47.67	26.66	30.30	36.81	26.94	21.30	17.64	42.71	108.10

Table 3: Quantitative evaluation on the MIMIC-CXR dataset. Higher values indicate better performance.

	CE-P	CE-R	CE-F1	BLEU-1	BLEU-2	BLEU-3	BLEU-4	ROUGE-L	CIDEr
CT2RepLong	13.37	13.12	13.24	47.20	28.46	24.26	10.57	35.63	25.68
FedTAR (ours)	17.65	12.76	14.47	40.08	25.94	20.54	12.14	29.54	42.80

Table 4: Comparison with centralized generation models on the J-MID dataset. Higher values are better.

Statistic	CE-P	CE-R	CE-F1	BLEU-1	BLEU-2	BLEU-3	BLEU-4	CIDEr	ROUGE-L
95% CI	(-0.02,0.08)	(0.01,0.10)	(0.00,0.09)	(-0.00,0.02)	(0.01,0.03)	(0.01,0.03)	(0.01,0.03)	(0.17,0.57)	(0.01,0.03)
$p$ -value	0.25	0.02	0.04	0.09	<0.001	<0.001	<0.001	<0.001	<0.001
Win-rate (%)	88.98	98.84	97.76	95.71	100.00	100.00	100.00	100.00	100.00
Significant	No	Yes	Yes	No	Yes	Yes	Yes	Yes	Yes

Table 5: Paired significance analysis between FedTAR and FedAdam on MIMIC-CXR. Significant results: 95% CI excludes 0 and

p<0.05

Panel of Figure 3 — Figure 3: Overall caption describing both subfigures.

Property	FedAvg	Online FL	FTA (Ours)
Data Dist.	$P_{k}$ (Static)	$Q_{k,\tau}$ (Streaming)	$P_{k,t}$ (Evolving)
Temporal Structure	Ignored	Optimization Time	Physical Time
Sequential Modeling	$\times$	$\times$	$\checkmark$
Objective	Expectation	Regret / Stream	Longitudinal Sum

Table 6: Comparison of Federated Learning Paradigms.

Medical Report Generation

Medical report generation turns medical images, clinical records, or conversations into structured, clinically useful reports.

See all 82 papers

FedKPer: Tackling Generalization and Personalization in Medical Federated Learning via Knowledge Personalization
May 1, 2026Zoe Fowler, Ghassan AlRegibFederated Learning Aggregation Cross-Domain Generalization
Georgia Institute of Technology
TIM: Temporal Decoupling with Iterative Mutual-Refinement Model for Longitudinal Radiology Report Generation
Oct 19, 2026Yiheng Dong, Yi Lin, Shilong Huang +2Radiology Report Generation Radiology VLMs
Huazhong University of Science and Technology, Wuhan, China · HKUST, Hong Kong, China
Understanding Trajectory Heterogeneity in Federated World Model Learning
Oct 2, 2026Yipan Wei, Zhaokun Yan, Ziming Hong +2World Model Learning Heterogeneous Federated Learning
Wuhan University · China Academy of Information and Communications Technology · The University of Sydney +2

May 1, 2026J/K move · Enter open · S save

FedKPer: Tackling Generalization and Personalization in Medical Federated Learning via Knowledge Personalization

Zoe Fowler, Ghassan AlRegib

Georgia Institute of Technology

Federated Learning Aggregation Cross-Domain Generalization Federated Learning Heterogeneous Federated Learning Catastrophic Forgetting

Open paper AskPDF arXiv

Federated learning (FL) holds great potential for medical applications. However, statistical heterogeneity across healthcare institutions poses a major challenge for FL, as the global model struggles both to generalize across unseen patient populations and to adapt to the unique data distributions of individual hospitals. This heterogeneity also exacerbates forgetting at both the global and local level, resulting in previous learned patient patterns to be misclassified after model updates. While prior work has largely treated generalization and personalization as separate challenges, we show that a better balance between the two can be achieved through selective alignment with the global model and a modified aggregation scheme, which together mitigate the effects of statistical heterogeneity. Specifically, we introduce FedKPer, which introduces knowledge personalization into the training stage of each local device. Afterwards, generalization is considered via the global model aggregation process, where local updates that are reliable and label-diverse are emphasized. We evaluate the performance of FedKPer, devising additional metrics that relate to common consequences of forgetting. Overall, we demonstrate FedKPer improves the generalization-personalization trade-off without sacrificing retention.

Abstract

Medical Report Generation

Explore similar work

FedKPer: Tackling Generalization and Personalization in Medical Federated Learning via Knowledge Personalization

TIM: Temporal Decoupling with Iterative Mutual-Refinement Model for Longitudinal Radiology Report Generation

Understanding Trajectory Heterogeneity in Federated World Model Learning