Clinical deployment of chest radiograph classifiers requires models that can be updated as new datasets become available without retraining on previously observed data or degrading validated performance. We study a task-incremental continual learning setting for chest radiograph classification under task-unknown inference, where heterogeneous chest X-ray datasets arrive sequentially and task identity is unavailable at deployment time. We propose CARL-CXR, a continual adapter-based routing framework that maintains a fixed high-capacity backbone while incrementally introducing lightweight task-specific adapters and classifier heads. A latent task selector operates on adapter-conditioned features to dynamically route each input to the most relevant task pathway, leveraging compact task prototypes and feature-level experience replay to preserve task identity across sequential updates without storing raw images. Experiments on MIMIC-CXR and CheXpert two large-scale datasets with distinct patient populations, imaging devices, and annotation pipelines demonstrate that CARL-CXR achieves minimal catastrophic forgetting (0.012 AUROC drop), representing a 6X and 11X reduction over established continual learning baselines LwF and EWC respectively, while maintaining competitive diagnostic performance (AUROC 0.74). Under task unknown deployment, CARL-CXR outperforms joint training by 12.5 points in routing accuracy (75.0% vs. 62.5%): unlike LwF and EWC, which require explicit task identifiers at inference and provide no routing mechanism.
Figures & tables
Figure 1: CARL-CXR framework: Architectural overview of continual chest radiograph learning with a frozen backbone, incremental task-specific adapters, and routing-based task selection during inference.
Figure 2: CARL-CXR diagnostic performance across training phases. (a) Forgetting on Task 1 is 0.012, indicating strong retention under sequential updates. (b) Under task-unknown inference, sequential training achieves 75.0% weighted routing accuracy versus 62.5% for joint training, reflecting stronger task-specific feature separation.
Training Phase
MIMIC AUC
CheXpert AUC
Forgetting
Memory (MB)
Task 1 (MIMIC)
0.752
–
–
1.1
Task 2 (CheXpert)
0.740
0.748
0.012
2.3
Table 1: CARL-CXR performance under sequential continual learning. AUROC reported as macro-average across 14 clinical findings.
Method
MIMIC AUC
CheXpert AUC
Forgetting ↓
Balanced Routing (%)
EWC [ 9 ]
0.645
–
0.133
N/A
✗
LwF [ 12 ]
0.707
0.823
0.072
N/A
✗
Joint Training (upper bound)
0.740
0.730
–
53.8
✗
CARL-CXR (ours)
0.740
0.748
0.012
65.3
✓
Table 2: Comparison with continual learning baselines on sequential MIMIC → CheXpert. Forgetting = AUROC drop on Task 1 after learning Task 2. Task-unknown routing is not applicable (N/A) for EWC and LwF as both require task identity at inference.
Figure 3: Routing accuracy under task-unknown inference. CARL-CXR achieves 75.0% weighted and 65.3% balanced routing accuracy, compared to 62.5% and 53.8% respectively for joint training, demonstrating stronger task-specific feature separation under the sequential continual setting.
Figure 4: Confusion matrix for task-unknown routing under the main CARL-CXR configuration. Misrouting is balanced across both directions, confirming the selector does not exhibit bias toward the larger MIMIC-CXR dataset.
Setting
Routing Strategy
Balanced Acc (%)
Weighted Acc (%)
Prototypes only
Selector-based (MLP)
13.1
14.3
Prototypes + Replay
Selector-based (MLP)
65.3
75.0
Prototypes only
Entropy-based
57.2
57.3
Table 3: Impact of experience replay on task-unknown routing.
Routing Strategy
MIMIC Acc (%)
CheXpert Acc (%)
Balanced Acc (%)
Weighted Acc (%)
AUC
Memory-based
88.0
11.0
49.5
50.0
0.721
Entropy-based
66.0
65.0
65.5
65.0
0.740
Selector + Replay
65.6
64.7
65.3
75.0
0.748
Table 4: Inference-time routing strategy comparison under task-unknown deployment.
Buffer Capacity
MIMIC Acc
CheXpert Acc
Balanced Acc
Weighted Acc
0
0.520
0.835
0.678
0.556
1000
0.690
0.549
0.620
0.674
2500
0.546
0.795
0.671
0.575
5000
0.778
0.523
0.651
0.748
10000
0.726
0.517
0.622
0.702
Table 5: Routing accuracy under different replay buffer capacities.
Class-incremental learning (CIL) requires a model to incrementally learn tasks that contain new classes without accessing earlier training data while preserving the ability to recognize all seen classes. Recently, pretrained-model-based approaches have become prevalent by adapting a frozen backbone with additional lightweight trainable modules. Existing methods, however, exhibit limitations: task-specific adapters learn explicit per-task representations but are parameter- and computation-inefficient, while LoRA-based merging methods combine per-task LoRA parameters into a single model whose static aggregated weights cause representation interference during inference. To address these problems, we present FACET: task-conditioned FeAture transformation with ConditionEd feature consisTency, achieving excellent parameter efficiency while producing highly discriminative features during inference. When continually trained on a task sequence, FACET learns a single shared adapter that employs a dynamic task-conditioned feature transformation, shaping the overall feature distribution of the adapter into a mixture of overlap-reduced task-specific components. On the other hand, we propose an efficient replay-free task-conditioned feature consistency loss, aiming to mitigate catastrophic forgetting of the learned mixture distribution in the adapter's feature space. Even when maintaining only a single adapter, FACET demonstrates robust scalability. On both very long task sequences (e.g., 200 tasks) and standard short task sequences (e.g., 20 tasks), our method achieves superior performance while using significantly fewer trainable parameters and GFLOPs. The code will be made open source upon acceptance.
Yunxiang Fu, Meng Lou, Yizhou Yu
School of Computing and Data Science The University of Hong Kong
Recent advances in vision--language pretraining have enabled strong medical foundation models, yet most analyze radiographs in isolation, overlooking the key clinical task of comparing prior and current images to assess interval change. For chest radiographs (CXRs), capturing interval change is essential, as radiologists must evaluate not only the static appearance of findings but also how they evolve over time. We introduce TILA (Temporal Inversion-aware Learning and Alignment), a simple yet effective framework that uses temporal inversion, reversing image pairs, as a supervisory signal to enhance the sensitivity of existing temporal vision-language models to directional change. TILA integrates inversion-aware objectives across pretraining, fine-tuning, and inference, complementing conventional appearance modeling with explicit learning of temporal order. We also propose a unified evaluation protocol to assess order sensitivity and consistency under temporal inversion, and introduce MS-CXR-Tretrieval, a retrieval evaluation set constructed through a general protocol that can be applied to any temporal CXR dataset. Experiments on public datasets and real-world hospital cohorts demonstrate that TILA consistently improves progression classification and temporal embedding alignment when applied to multiple existing architectures.
Hanbin Ko, Kyungmin Jeon, Doowoong Choi +1
Interdisciplinary Program in Bioengineering, Seoul National University Graduate School · Integrated Major in Innovative Medical Science, Seoul National University Graduate School · Dept. of Radiology, Seoul National University Hospital +2
This challenge tackles multi-label classification for known chest X-ray (CXR) lesions and zero-shot classification for unseen ones. To handle diverse CXR projections, we integrate projection-specific models via a classification network into a unified framework. For zero-shot classification (Task 2), we extend CheXzero with a novel dual-branch architecture that combines contrastive learning, Asymmetric Loss (ASL), and LLM-generated descriptive prompts. This effectively mitigates severe long-tail imbalances and maximizes zero-shot generalization. Additionally, strong data and test-time augmentations (TTA) ensure robustness across both tasks.