Causally Sufficient and Necessary Feature Expansion for Class-Incremental Learning

Zhen Zhang; Jielei Chu; Jiangtao Hu; Bin Liu; Jie Wang; Ya Liu; Tianrui Li

cs.LGMar 10, 2026

Causally Sufficient and Necessary Feature Expansion for Class-Incremental Learning

Authors: Zhen Zhang, Jielei Chu, Jiangtao Hu, Bin Liu, Jie Wang, Ya Liu, Tianrui Li

Organizations: School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu 611756, China. · Engineering Research Center of Sustainable Urban Intelligent Transportation, Ministry of Education, Chengdu, Sichuan 611756, China · National Engineering Laboratory of Integrated Transportation Big Data Application Technology, Southwest Jiaotong University, Chengdu, Sichuan 611756, China. · Sichuan International Travel Healthcare Center, Chengdu Customs, Chengdu, 610041, Sichuan, China ·

Abstract

Current expansion-based methods for Class Incremental Learning (CIL) effectively mitigate catastrophic forgetting by freezing old features. However, such task-specific features learned from the new task may collide with the old features. From a causal perspective, spurious feature correlations are the main cause of this collision, manifesting in two scopes: (i) guided by empirical risk minimization (ERM), intra-task spurious correlations cause task-specific features to rely on shortcut features. These non-robust features are vulnerable to interference, inevitably drifting into the feature space of other tasks; (ii) inter-task spurious correlations induce semantic confusion between visually similar classes across tasks. To address this, we propose a Probability of Necessity and Sufficiency (PNS)-based regularization method to guide feature expansion in CIL. Specifically, we first extend the definition of PNS to expansion-based CIL, termed CPNS, which quantifies both the causal completeness of intra-task representations and the separability of inter-task representations. We then introduce a dual-scope counterfactual generator based on twin networks to ensure the measurement of CPNS, which simultaneously generates: (i) intra-task counterfactual features to minimize intra-task PNS risk and ensure causal completeness of task-specific features, and (ii) inter-task interfering features to minimize inter-task PNS risk, ensuring the separability of inter-task representations. Theoretical analyses confirm its reliability. The regularization is a plug-and-play method for expansion-based CIL to mitigate feature collision. Extensive experiments demonstrate the effectiveness of the proposed method.

Figures & tables

Fig. 1 : (a) and (b): Old→New misclassification rates grouped by semantic overlap on CUB200. Old classes are partitioned into low-, medium-, and high-overlap groups according to their maximum attribute similarity to the current new classes. A higher error rate in the high-overlap group indicates that semantically similar new classes more easily intrude into old-class decision regions. (c): CKA feature similarity analysis. Our method possesses high similarity in shallow layers (indicating shared causal semantics) while maintaining discriminability in deep layers.

Fig. 2 : Illustration of feature suppression and collision. ERM and the diversity strategy drive the model to learn shortcut features (e.g., ear vs. eyes) for semantically similar classes, leading to a fragmented feature space.

Fig. 3 : Structural Causal Model (SCM) for expansion-based CIL. Left: causal generating mechanism, Right: the learning process.

Dataset	CIFAR-100				ImageNet-100				ImageNet-1000
Scenarios	10-10		50-10		10-10		50-10		100-100
Methods	Last	Avg	Last	Avg	Last	Avg	Last	Avg	Last	Avg
DER	64.35	75.36	65.27	72.60	66.71	77.18	71.08	77.71	58.83	66.87
w/ CPNS	66.21	76.93	67.31	74.24	68.12	78.16	72.54	79.26	60.25	67.74
FOSTER	62.20	74.49	59.80	67.54	65.68	76.74	71.60	77.37	59.03	67.18
w/ CPNS	64.25	75.48	61.28	68.77	66.98	77.82	72.77	78.54	60.12	67.77

TABLE I : Performance results.

Fig. 4 : Accuracy curves for CPNS on various scenarios and baselines.

Methods	CUB200 100-20		Birds525 100-50		Flower102 10-10		Food101 10-10
Methods	Last	Avg	Last	Avg	Last	Avg	Last	Avg
DER	52.88	53.07	77.40	85.21	48.70	55.94	61.10	72.05
w/ CPNS	56.52 ( +3.64 )	55.20 ( +2.13 )	79.52 ( +2.12 )	87.73 ( +2.52 )	51.06 ( +2.36 )	57.65 ( +1.71 )	63.29 ( +2.19 )	73.98 ( +1.93 )
FOSTER	53.48	54.01	78.06	85.36	49.64	57.77	61.96	72.54
w/ CPNS	55.93 ( +2.45 )	56.26 ( +2.25 )	80.02 ( +1.96 )	87.51 ( +2.15 )	51.99 ( +2.35 )	59.94 ( +2.17 )	63.88 ( +1.92 )	74.09 ( +1.55 )
BEEF	51.30	51.98	76.98	85.64	48.67	56.04	60.99	71.76

TABLE II : Performance results on various fine-grained datasets.

Method	Top- $k$ Parts Masked					Avg. Drop $\downarrow$
Method	0	1	2	3	5	Avg. Drop $\downarrow$
DER	52.88	40.52	32.15	25.33	17.65	7.05
w/ CPNS	56.52	46.21	39.14	33.25	25.12	6.28
TagFex	54.37	43.12	35.41	28.85	20.34	6.81
w/ CPNS	56.13	47.05	40.32	34.81	27.45	5.74

TABLE III : Intervention-based evaluation on CUB200 dataset. We report the last accuracy (%) after progressively masking the top-

k

most important semantic parts.

Fig. 5 : Validation of intra-task and inter-task counterfactual representations on CIFAR-100

Fig. 6 : Examples of intra-task and inter-task counterfactual sample construction.

Fig. 7 : Complementary comparison with counterfactual generation methods.

Baseline:DER			CIFAR-100		ImageNet-100
$\text{PNS}_{\text{intra}}$	$\text{PNS}_{\text{inter}}$	2-Stage	10-10	50-10	10-10	50-10
✗	✗	✗	75.36	72.60	77.18	77.71
✓	✗	✗	76.31	73.64	77.78	78.64
✗	✓	✗	74.98	71.84	76.65	77.21
✗	✓	✓	75.28	73.59	77.43	78.51
✓	✓	✗	75.16	72.09	76.98	77.79

TABLE IV : Ablation study on CIFAR-100 and ImageNet-100 in terms of average accuracy. The baseline is DER.

Dataset	CIFAR-100
Metric Strategy	10-10		50-10
Metric Strategy	Last	Avg	Last	Avg
DER (Baseline)	64.35	75.36	65.27	72.60
w/ MSE	65.12	76.05	65.88	73.15
w/ Cross-Entropy	65.45	76.28	66.10	73.45
w/ Wasserstein	65.80	76.55	66.52	73.80

TABLE V : Ablation study on different distance metrics for counterfactual generation within the CPNS framework.

Fig. 8 : Hyperparameter sensitivity analysis on CIFAR-100 10-10.

Fig. 9 : The parameter sensitivity experiment of $\beta$ (EQ. 9 ) in the CIFAR100 10-10 scenario.

Fig. 10 : t-SNE visualization on the CUB200 dataset. Panels (a)–(d) correspond to the baseline DER,

\text{PNS}_{\text{intra}}

\text{PNS}_{\text{inter}}

, and w/ CPNS, respectively.

Fig. 11 : Grad-CAM visualization on the CUB200 dataset. Compared to the baseline DER, our CPNS framework forces the model to focus on causally complete attributes.

Class-Incremental Learning

Class-incremental learning studies how models can add new categories over time while retaining the ability to recognize previously learned classes.

See all 160 papers

Width Expansion as a Method for Class Incremental Learning
Sep 29, 2026A. L. S. Conde, Y. Elkhatib, C. M. RanieriClass-Incremental Learning Dynamic Neural Networks
Institute of Geosciences and Exact Sciences, São Paulo State University (UNESP), Av. 24-A, 1515, Rio Claro, 13506-692, SP, Brazil · School of Computing Science, University of Glasgow, Glasgow, G12 8QQ, Scotland, United Kingdom
Dynamic LoRA-Experts and Prototype-Ensemble Matching for Class-Incremental Learning
Sep 30, 2026Hongwei Zhao, Rui Liu, Yansong LiuClass-Incremental Learning Prototype-Based Classification
School of Computer Science and Engineering, Beihang University
Miles: Metric Learning with Expandable Subspace for Pre-Trained Model-Based Class-Incremental Learning
Jul 20, 2026Kai Jiang, Zisong Lin, Hongyuan Zhang +2Class-Incremental Learning Metric Learning
National Key Laboratory of Radar Signal Processing, Xidian University, Xi’an 710071, China · The University of Hong Kong, Hong Kong SAR, China · Institute of Artificial Intelligence (TeleAI) of China Telecom, China

Sep 29, 2026J/K move · Enter open · S save

Width Expansion as a Method for Class Incremental Learning

A. L. S. Conde, Y. Elkhatib, C. M. Ranieri

Institute of Geosciences and Exact Sciences, São Paulo State University (UNESP), Av. 24-A, 1515, Rio Claro, 13506-692, SP, Brazil · School of Computing Science, University of Glasgow, Glasgow, G12 8QQ, Scotland, United Kingdom

Class-Incremental Learning Dynamic Neural Networks Incremental Learning

Open paper AskPDF arXiv

Class Incremental Learning (Class-IL) requires models to learn new classes over time while preserving previously acquired knowledge without access to past data or task identity. This setting intensifies the stability-plasticity dilemma and makes catastrophic forgetting a central challenge. Existing approaches include regularization, knowledge distillation, replay, and architectural expansion. However, many expansion methods rely on explicit task identifiers or predefined growth strategies, limiting their applicability when task boundaries are unavailable at inference time. This work proposes a dynamic width expansion method that increases the number of neurons within existing layers according to a normalized loss criterion, without requiring task-specific information. An attention mechanism with persistent key-value memory is also incorporated to stabilize feature representations and reduce interference between previously learned and newly introduced classes. The approach is evaluated on Split MNIST and Split CIFAR-100 under the standard Class-IL protocol. Experiments compare fixed-capacity and dynamically expanding architectures, both with and without attention, combined with established continual learning methods including EWC, LwF, and A-GEM. Results show that progressive width expansion consistently improves performance over fixed architectures, particularly when combined with functional methods and A-GEM. The combination of width expansion and attention provides the most consistent gains. Overall, dynamic width expansion based on representational demand provides an effective and flexible strategy for Class-IL, although uncontrolled growth may increase overfitting and computational cost.

Abstract

Class-Incremental Learning

Explore similar work

Width Expansion as a Method for Class Incremental Learning

Dynamic LoRA-Experts and Prototype-Ensemble Matching for Class-Incremental Learning

Miles: Metric Learning with Expandable Subspace for Pre-Trained Model-Based Class-Incremental Learning