Longitudinal medical report generation is clinically important yet remains challenging due to strict privacy constraints and the evolving nature of disease progression. Although federated learning (FL) enables collaborative training without data sharing, existing FL methods largely overlook longitudinal dynamics by assuming stationary client distributions, making them unable to model temporal shifts across visits or patient-specific heterogeneity-ultimately leading to unstable optimization and suboptimal report generation. We introduce Federated Temporal Adaptation (FTA), a federated setting that explicitly accounts for the temporal evolution of client data. Building upon this setting, we propose FedTAR, a framework that integrates demographic-driven personalization with time-aware global aggregation. FedTAR generates lightweight LoRA adapters from demographic embeddings and performs temporal residual aggregation, where updates from different visits are weighted by a meta-learned temporal policy optimized via first-order MAML. Experiments on J-MID (1M exams) and MIMIC-CXR demonstrate consistent improvements in linguistic accuracy, temporal coherence, and cross-site generalization, establishing FedTAR as a robust and privacy-preserving paradigm for federated longitudinal modeling.
Figures & tables
Figure 1: Visualization of semantic structure evolution in medical reports. All data are derived from the J-MID dataset. (a) Ridgeline kernel-density plots of LDA-derived topic components for annual reports (2018–2024) from one institution. (b) T-SNE embedding of TF-IDF vectors for reports from five institutions.
Figure 2: Overview of the problem and our approach. (a) Client- and temporal-drift challenges in federated longitudinal medical report generation. (b) The proposed FedTAR framework.
BLEU-1
BLEU-2
BLEU-3
BLEU-4
ROUGE-L
CIDEr
FedAvg [ 24 ]
38.40
24.68
16.16
10.98
28.54
31.70
FedProx [ 21 ]
38.32
24.48
15.95
11.00
28.58
31.62
SCAFFOLD [ 18 ]
35.40
22.38
14.80
10.10
27.12
24.82
FedAdam [ 31 ]
38.26
24.54
16.05
10.93
28.61
30.62
FedYogi [ 31 ]
35.39
22.37
13.72
10.46
27.91
24.79
DRFA [ 8 ]
36.80
24.02
16.63
11.60
28.75
29.51
Table 1: Comparison of our proposed method (FedTAR) with standard FL baselines on the chest CT report generation task. Evaluation metrics include BLEU- n ( n =1–4), ROUGE-L, and CIDEr, which collectively reflect linguistic precision, recall, and content relevance. FedTAR achieves consistent improvements, demonstrating the effectiveness of demographics- and temporally-aware adaptation.
BLEU-1
BLEU-2
BLEU-3
BLEU-4
ROUGE-L
CIDEr
FedTAR w/o GMM
40.38
25.08
17.59
12.07
28.65
41.56
FedTAR w/o temporal
38.83
25.82
17.28
11.21
29.50
41.88
FedTAR
40.08
25.94
17.80
12.40
29.54
42.80
Table 2: Ablation study evaluating the contributions of the Gaussian Mixture Model (GMM)-based demographics embedding and the temporal residual aggregation module. Removing either component leads to consistent performance degradation across all metrics, confirming their necessity for capturing patient-specific heterogeneity and longitudinal dynamics.
CE-P
CE-R
CE-F1
BLEU-1
BLEU-2
BLEU-3
BLEU-4
ROUGE-L
CIDEr
FedAvg [ 24 ]
68.58
22.63
28.43
29.16
19.49
14.36
11.00
35.77
37.54
FedProx [ 21 ]
69.83
24.59
30.26
27.16
17.80
12.85
9.70
34.11
37.38
SCAFFOLD [ 18 ]
44.58
22.6
21.47
28.00
17.23
14.21
9.86
38.05
99.11
FedAdam [ 31 ]
58.74
25.95
31.80
39.17
28.56
22.42
18.39
43.08
97.07
FedYogi [ 31 ]
52.02
23.36
29.90
35.55
22.57
18.94
13.84
37.03
94.11
DRFA [ 8 ]
47.67
26.66
30.30
36.81
26.94
21.30
17.64
42.71
108.10
Table 3: Quantitative evaluation on the MIMIC-CXR dataset. Higher values indicate better performance.
CE-P
CE-R
CE-F1
BLEU-1
BLEU-2
BLEU-3
BLEU-4
ROUGE-L
CIDEr
CT2RepLong
13.37
13.12
13.24
47.20
28.46
24.26
10.57
35.63
25.68
FedTAR (ours)
17.65
12.76
14.47
40.08
25.94
20.54
12.14
29.54
42.80
Table 4: Comparison with centralized generation models on the J-MID dataset. Higher values are better.
Statistic
CE-P
CE-R
CE-F1
BLEU-1
BLEU-2
BLEU-3
BLEU-4
CIDEr
ROUGE-L
95% CI
(-0.02,0.08)
(0.01,0.10)
(0.00,0.09)
(-0.00,0.02)
(0.01,0.03)
(0.01,0.03)
(0.01,0.03)
(0.17,0.57)
(0.01,0.03)
p -value
0.25
0.02
0.04
0.09
<0.001
<0.001
<0.001
<0.001
<0.001
Win-rate (%)
88.98
98.84
97.76
95.71
100.00
100.00
100.00
100.00
100.00
Significant
No
Yes
Yes
No
Yes
Yes
Yes
Yes
Yes
Table 5: Paired significance analysis between FedTAR and FedAdam on MIMIC-CXR. Significant results: 95% CI excludes 0 and p<0.05 .
Figure 3: Overall caption describing both subfigures.
Figure 9Figure 10
Figure 6: fig:generated report
Property
FedAvg
Online FL
FTA (Ours)
Data Dist.
Pk (Static)
Qk,τ (Streaming)
Pk,t (Evolving)
Temporal Structure
Ignored
Optimization Time
Physical Time
Sequential Modeling
×
×
✓
Objective
Expectation
Regret / Stream
Longitudinal Sum
Table 6: Comparison of Federated Learning Paradigms.
Federated learning (FL) holds great potential for medical applications. However, statistical heterogeneity across healthcare institutions poses a major challenge for FL, as the global model struggles both to generalize across unseen patient populations and to adapt to the unique data distributions of individual hospitals. This heterogeneity also exacerbates forgetting at both the global and local level, resulting in previous learned patient patterns to be misclassified after model updates. While prior work has largely treated generalization and personalization as separate challenges, we show that a better balance between the two can be achieved through selective alignment with the global model and a modified aggregation scheme, which together mitigate the effects of statistical heterogeneity. Specifically, we introduce FedKPer, which introduces knowledge personalization into the training stage of each local device. Afterwards, generalization is considered via the global model aggregation process, where local updates that are reliable and label-diverse are emphasized. We evaluate the performance of FedKPer, devising additional metrics that relate to common consequences of forgetting. Overall, we demonstrate FedKPer improves the generalization-personalization trade-off without sacrificing retention.
Automatic radiology report generation (RRG) aims to translate medical images into diagnostic text, reducing radiologists' workload and standardizing clinical documentation. Nonetheless, existing approaches mainly focus on single-timepoint analysis and fail to capture temporal disease evolution across longitudinal examinations. While recent longitudinal RRG (LRRG) approaches incorporate historical data, they often combine images from different time points within a single representation space, leading to blurred semantics and inconsistent temporal reasoning. In this work, we propose a Temporal Decoupling with Iterative Mutual-Refinement Model (TIM), a two-stage framework that explicitly decouples spatial pathology from temporal progression and iteratively refines reports through mutual feedback. Stage I performs temporal-decoupled representation learning, separating temporal evolution patterns from disease-specific features and generating radiology reports for both prior and current studies. Stage II introduces a mutual report refinement mechanism that identifies diagnostic inconsistencies within prior reports and iteratively rectifies both prior and current reports through error-sensitive feedback. Experiments on the Longitudinal-MIMIC dataset demonstrate that TIM surpasses existing single-image and longitudinal baselines, achieving new state-of-the-art performance across both language and clinical metrics. Code is available at https://github.com/yihengd/TIM.
Yiheng Dong, Yi Lin, Shilong Huang +2
Huazhong University of Science and Technology, Wuhan, China · HKUST, Hong Kong, China
World models learn state evolution from trajectories, making access to temporal context a central training requirement. Federated learning can use distributed records, while ownership boundaries within a trajectory restrict the examples each client can construct. Our study benchmarks this cross-time setting through hourly action-conditioned clinical prediction on eight MIMIC-IV disease cohorts, comprising 40.87 million transition memberships. We specify severity-based client ownership, patient-separated construction, local history and future-window rules, and paired rollout evaluation from one to 32 hours. A matrix of ten federated algorithms covers 32 disease--partition configurations under five rounds of ten-percent participation. Three findings emerge from existing results and training logs. First, client ownership and participation jointly restrict long-window coverage: only 7.55%--21.36% of pooled-available 32-step windows have a locally complete anchor visited during training, averaged across diseases. Second, finer severity partitions accompany higher FedAvg error in 15 of 16 paired comparisons, while algorithm gains are small and horizon-dependent: FedProx reduces mean error by 0.56%, with no consistent improvement at 32 steps. Third, algorithm labels conceal distinct update behavior, including inactive extrapolation and orders-of-magnitude differences in update scale. Cached-update performance also varies strongly across trajectory partitions under the same benchmark protocol. These results establish temporal access, participation coverage, optimization behavior, and horizon-resolved prediction as complementary dimensions for evaluating federated clinical world models.
Yipan Wei, Zhaokun Yan, Ziming Hong +2
Wuhan University · China Academy of Information and Communications Technology · The University of Sydney +2