Free-Flow Class-Incremental Learning: Towards Robust CIL under Variable Class Arrivals

Zhiming Xu; Baile Xu; Jian Zhao; Furao Shen; Suorong Yang

cs.LGApr 3, 2026

Free-Flow Class-Incremental Learning: Towards Robust CIL under Variable Class Arrivals

Authors: Zhiming Xu, Baile Xu, Jian Zhao, Furao Shen, Suorong Yang

Organizations: National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · School of Artificial Intelligence, Nanjing University, Nanjing, China · School of Electronic Science and Engineering, Nanjing University, China · School of Computer Science, Nanjing University, Nanjing, China

Abstract

Class-incremental learning (CIL) is commonly evaluated under predefined schedules with fixed or nearly equal class increments, leaving irregular class-arrival scenarios underexplored. However, practical CIL systems may need to update whenever new categories emerge, without forcing them into balanced task partitions. We formalize this setting as Free-Flow Class-Incremental Learning (FFCIL), where the number of newly arriving classes can vary substantially across learning stages. We show that variable class arrivals alter the class composition of incremental training, the reliability of new-class classifier statistics, and the consistency of representations learned across increments, causing clear performance degradation in both conventional and pre-trained model (PTM)-based CIL methods. To improve robustness under FFCIL, we introduce a general framework consisting of Class-Wise Mean (CWM), which replaces instance-wise loss aggregation with class-wise averaging; Dynamic Intervention Weight Alignment (DIWA), which adjusts new-class weight calibration according to the current increment size; and Head-Agnostic Alignment (HA), which performs feature-level correction for PTM-based methods using current and previous-class feature supervision. Extensive experiments across diverse methods, datasets, and class-arrival schedules demonstrate the general impact of FFCIL and the consistent effectiveness of our framework.

Figures & tables

Panel of Figure 1 — Figure 1: Illustration of FFCIL. (a) Unlike equal-size tasks, FFCIL allows variable per-step class increments. (b) Existing CIL methods could experience a substantial accuracy drop under FFCIL, even with the same total number of classes, learning stages, and global class permutation.

Figure 2: Overview of the proposed framework for FFCIL. Class-Wise Mean (CWM) replaces instance-wise loss aggregation with class-wise averaging to reduce the dependence of class contributions on step-specific batch composition. Dynamic Intervention Weight Alignment (DIWA) adjusts the strength of new-class weight calibration according to the number of classes introduced at the current step. Head-Agnostic Alignment (HA) performs a lightweight feature-level correction before the original prediction head and uses previous-class feature statistics to retain supervision over the accumulated label space.

Non-PTM	CIFAR-100						ImageNet-100
	Equ.T		FF. org		FF. ours		Equ.T		FF. org		FF. ours
	$\overline{A}$	$A_{T}$	$\overline{A}$	$A_{T}$	$\overline{A}$	$A_{T}$	$\overline{A}$	$A_{T}$	$\overline{A}$	$A_{T}$	$\overline{A}$	$A_{T}$
iCaRL	$70.25$ [-0.65mm] $\pm 0.12$	$49.80$ [-0.65mm] $\pm 0.22$	$55.90$ [-0.65mm] $\pm 1.50\;{\color[rgb]{0.6992,0.1328,0.1328}\downarrow 14.35}$	$42.96$ [-0.65mm] $\pm 1.50\;{\color[rgb]{0.6992,0.1328,0.1328}\downarrow 6.84}$	$62.60$ [-0.65mm] $\pm 1.30\;{\color[rgb]{0.1328,0.5469,0.1328}\uparrow 6.70}$	$49.24$ [-0.65mm] $\pm 1.40\;{\color[rgb]{0.1328,0.5469,0.1328}\uparrow 6.28}$	$69.86$ [-0.65mm] $\pm 0.12$	$51.54$ [-0.65mm] $\pm 0.24$	$54.48$ [-0.65mm] $\pm 1.90\;{\color[rgb]{0.6992,0.1328,0.1328}\downarrow 15.38}$	$36.44$ [-0.65mm] $\pm 2.50\;{\color[rgb]{0.6992,0.1328,0.1328}\downarrow 15.10}$	$60.73$ [-0.65mm] $\pm 1.70\;{\color[rgb]{0.1328,0.5469,0.1328}\uparrow 6.25}$	$43.10$ [-0.65mm] $\pm 2.20\;{\color[rgb]{0.1328,0.5469,0.1328}\uparrow 6.66}$
DER	$74.44$ [-0.65mm] $\pm 0.06$	$60.55$ [-0.65mm] $\pm 0.11$	$71.87$ [-0.65mm] $\pm 1.20\;{\color[rgb]{0.6992,0.1328,0.1328}\downarrow 2.57}$	$58.21$ [-0.65mm] $\pm 1.60\;{\color[rgb]{0.6992,0.1328,0.1328}\downarrow 2.34}$	$72.28$ [-0.65mm] $\pm 0.80\;{\color[rgb]{0.1328,0.5469,0.1328}\uparrow 0.41}$	$60.21$ [-0.65mm] $\pm 1.00\;{\color[rgb]{0.1328,0.5469,0.1328}\uparrow 2.00}$	$77.62$ [-0.65mm] $\pm 0.08$	$70.98$ [-0.65mm] $\pm 0.16$	$76.40$ [-0.65mm] $\pm 1.30\;{\color[rgb]{0.6992,0.1328,0.1328}\downarrow 1.22}$	$66.30$ [-0.65mm] $\pm 1.70\;{\color[rgb]{0.6992,0.1328,0.1328}\downarrow 4.68}$	$77.91$ [-0.65mm] $\pm 0.90\;{\color[rgb]{0.1328,0.5469,0.1328}\uparrow 1.51}$	$69.26$ [-0.65mm] $\pm 1.10\;{\color[rgb]{0.1328,0.5469,0.1328}\uparrow 2.96}$
MEMO	$73.75$ [-0.65mm] $\pm 0.16$	$59.07$ [-0.65mm] $\pm 0.30$	$70.84$ [-0.65mm] $\pm 1.10\;{\color[rgb]{0.6992,0.1328,0.1328}\downarrow 2.91}$	$56.52$ [-0.65mm] $\pm 1.40\;{\color[rgb]{0.6992,0.1328,0.1328}\downarrow 2.55}$	$71.42$ [-0.65mm] $\pm 0.80\;{\color[rgb]{0.1328,0.5469,0.1328}\uparrow 0.58}$	$58.34$ [-0.65mm] $\pm 1.00\;{\color[rgb]{0.1328,0.5469,0.1328}\uparrow 1.82}$	$74.47$ [-0.65mm] $\pm 0.16$	$63.34$ [-0.65mm] $\pm 0.34$	$72.05$ [-0.65mm] $\pm 1.50\;{\color[rgb]{0.6992,0.1328,0.1328}\downarrow 2.42}$	$57.90$ [-0.65mm] $\pm 2.00\;{\color[rgb]{0.6992,0.1328,0.1328}\downarrow 5.44}$	$72.48$ [-0.65mm] $\pm 1.00\;{\color[rgb]{0.1328,0.5469,0.1328}\uparrow 0.43}$	$61.70$ [-0.65mm] $\pm 1.40\;{\color[rgb]{0.1328,0.5469,0.1328}\uparrow 3.80}$
TagFex	$82.98$ [-0.65mm] $\pm 0.06$	$71.91$ [-0.65mm] $\pm 0.13$	$80.04$ [-0.65mm] $\pm 1.00\;{\color[rgb]{0.6992,0.1328,0.1328}\downarrow 2.94}$	$70.02$ [-0.65mm] $\pm 1.20\;{\color[rgb]{0.6992,0.1328,0.1328}\downarrow 1.89}$	$80.38$ [-0.65mm] $\pm 0.70\;{\color[rgb]{0.1328,0.5469,0.1328}\uparrow 0.34}$	$71.56$ [-0.65mm] $\pm 0.90\;{\color[rgb]{0.1328,0.5469,0.1328}\uparrow 1.54}$	$77.05$ [-0.65mm] $\pm 0.11$	$70.14$ [-0.65mm] $\pm 0.19$	$74.88$ [-0.65mm] $\pm 1.30\;{\color[rgb]{0.6992,0.1328,0.1328}\downarrow 2.17}$	$67.58$ [-0.65mm] $\pm 1.60\;{\color[rgb]{0.6992,0.1328,0.1328}\downarrow 2.56}$	$75.50$ [-0.65mm] $\pm 0.80\;{\color[rgb]{0.1328,0.5469,0.1328}\uparrow 0.62}$	$68.74$ [-0.65mm] $\pm 1.10\;{\color[rgb]{0.1328,0.5469,0.1328}\uparrow 1.16}$

Table 1:

\overline{A}

and

A_{T}

under Equ.T, FF.org, and FF.ours. Results are reported as mean

\pm

standard deviation over five seeds for each applicable randomness: the global class permutation for Equ.T, and the global class permutation, class-arrival spectrum, and temporal schedule for FFCIL. Arrows denote the drop from Equ.T to FF.org and the recovery from FF.org to FF.ours.

CWM	DIWA	iCaRL		DER
CWM	DIWA	$\overline{A}$	$A_{T}$	$\overline{A}$	$A_{T}$
$\times$	$\times$	55.90	42.96	71.87	58.21
✓	$\times$	58.88	45.29	72.17	60.01
✓	✓	62.60	49.24	72.28	60.21
CWM	HA	MOS		TUNA
CWM	HA	$\overline{A}$	$A_{T}$	$\overline{A}$	$A_{T}$

Table 2: Ablation study of FFCIL components.

Panel of Figure 3 — Figure 3: Computational impact study of the proposed CWM loss and HA.

Panel of Figure 4 — Figure 4: Impact of FFCIL class-arrival spectra under different values of the spectrum intensity parameter $\alpha$ .

Panel of Figure 5 — Figure 5: Impact of FFCIL step schedules under ascending, descending, and jumbled schedules.

Class-Incremental Learning

Class-incremental learning studies how models can add new categories over time while retaining the ability to recognize previously learned classes.

See all 159 papers

Sep 30, 2026

Dynamic LoRA-Experts and Prototype-Ensemble Matching for Class-Incremental Learning

Class-Incremental Learning (CIL) aims to continuously learn new classes without forgetting previously acquired knowledge. Parameter-efficient fine-tuning with pre-trained models reduces parameter overhead but can suffer from cumulative interference and suboptimal alignment between inference samples and specialized modules. We propose Dynamic LoRA-Experts and Prototype-Ensemble Matching (DLEPEM), a two-stage rehearsal-free framework. DLEPEM allocates a task-specific LoRA-Expert for each incremental task to reduce cross-task interference, then combines frozen pre-trained-model prototypes with task-adaptive LoRA-Expert prototypes for reliable task-level discrimination. Experiments on standard CIL and Few-Shot CIL benchmarks demonstrate strong performance under the evaluated protocols.

Hongwei Zhao, Rui Liu, Yansong Liu

School of Computer Science and Engineering, Beihang University

Class-Incremental Learning Prototype-Based Classification Continual Learning Low-Rank Adaptation Parameter-Efficient Fine-Tuning

Ask PDF

Jul 20, 2026

Miles: Metric Learning with Expandable Subspace for Pre-Trained Model-Based Class-Incremental Learning

Class Incremental Learning (CIL) aims to learn new concepts consistently from a data stream without forgetting. Unlike typical CIL methods which need to learn a model from scratch, pre-trained model (PTM) can easily adapt to a new task with fine-tuning. However, existing PTM-based CIL methods fail to achieve a trade-off between performance and computational expenditure, i.e., they either adopt the same parameter space so that leading catastrophic forgetting, or expand a new branch for each task but adding more computational cost. To this end, we propose MetrIc Learning with Expandable Subspace (Miles) to harness the prior information within pre-trained knowledge, thereby orchestrating an efficient expansion of the parameter space through guided optimization. Specifically, it decouples the learnable modules with the pre-trained model, exploiting prior information from intermediate features of the backbone network to enable more flexible parameter expansion. Then, a central loss is adopted to guide the new category to cluster towards the corresponding prototype in the new task subspace while incorporating an auxiliary distance regularization term to maintain metric equilibrium across tasks. Extensive experiments on six benchmark datasets demonstrate that Miles achieves state-of-the-art performance in various CIL settings.

Kai Jiang, Zisong Lin, Hongyuan Zhang +2

National Key Laboratory of Radar Signal Processing, Xidian University, Xi’an 710071, China · The University of Hong Kong, Hong Kong SAR, China · Institute of Artificial Intelligence (TeleAI) of China Telecom, China

Class-Incremental Learning Metric Learning Incremental Learning

Ask PDF

May 28, 2026

Non-Forgetting Knowledge Allocation with Bi-level Competition for Class-Incremental Learning

Class-Incremental Learning (CIL) with pre-trained models (PTMs) aims to sequentially adapt PTMs to new categories without forgetting old knowledge. Built upon PTMs, existing adapter-based methods mainly train models via distinct task-specific adapters, and present a uniform knowledge allocation for each adapter during inference. However, this allocation mechanism ignores the nature of task discrepancy and leads to suboptimal utilization of adapters. Also, under CIL constraint, an allocator is prone to forgetting when tasks evolve. To address these issues, we propose a Non-Forgetting Allocation with Bi-Level Competition (NoFA-BC). NoFA-BC constructs a non-forgetting allocator (NFA) by transforming the allocator training into a recursive least-squares problem and achieves an allocator equivalent to that trained with all data. Based on the NFA, a Bi-Level Competition (BLC) including an intra-task level Winner-Takes-All (WTA) mechanism and inter-task Last-Ones-Fall (LOF) elimination is proposed to provide better allocation of adapter knowledge. WTA extracts the most significant logit within a task to represent the adapter's contribution and LOF suppresses the irrelevant adapters. With BLC, participation ratio of each adapter can be tailored for each input. Moreover, a Stability Enhancement (SE) process is incorporated to further improve the performance of old tasks.

Xiang Tan, Run He, Yawen Cui +6

South China University of Technology · Hong Kong Polytechnic University · Agency for Science, Technology and Research +3

Class-Incremental Learning Incremental Learning Catastrophic Forgetting

Ask PDF

Abstract

Class-Incremental Learning

Explore similar work

Dynamic LoRA-Experts and Prototype-Ensemble Matching for Class-Incremental Learning

Miles: Metric Learning with Expandable Subspace for Pre-Trained Model-Based Class-Incremental Learning

Non-Forgetting Knowledge Allocation with Bi-level Competition for Class-Incremental Learning