Co2In: a Bi-level Memory Incremental Learning Framework with Knowledge Encoding, Consolidation, and Integration

Wenju Sun; Qingyong Li; Boyang Li; Wen Wang; Siyu Zhang; Yi Wei; Yangliao Geng

cs.AIOct 15, 2026

Co2In: a Bi-level Memory Incremental Learning Framework with Knowledge Encoding, Consolidation, and Integration

Authors: Wenju Sun, Qingyong Li, Boyang Li, Wen Wang, Siyu Zhang, Yi Wei, Yangliao Geng

Organizations: Key Laboratory of Big Data & Artificial Intelligence in Transportation (Ministry of Education), Beijing Jiaotong University, Beijing, China, 100044. · College of Computing and Data Science, Nanyang Technological University, Singapore, 639798. · Fujian Key Laboratory of Big Data Application and Intellectualization for Tea Industry, Wuyi University, Fujian, China, 354300.

Abstract

Incremental learning (IL) aims to continually acquire new knowledge (plasticity) while retaining previously learned information (stability). However, striking a balance between plasticity and stability remains a significant challenge for intelligent systems. The human brain achieves exceptional balance, owing to various memory units that collaboratively encode and store information. Inspired by human memory mechanisms, this paper introduces an IL framework with knowledge enCoding, Consolidation, and Integration (Co2In). Co2In is designed as a bi-level memory architecture: a working memory for adaptive knowledge acquisition and a long-term memory dedicated to the persistent retention of information. The two memory modules work cooperatively during IL. The working memory first learns from new data to encode knowledge into parameters. Subsequently, Co2In performs a consolidation process to identify underlying patterns in the learned parameters and re-express them into a compact knowledge representation. Next, the knowledge representation and the identified patterns are transformed into separate network layers and integrated into the long-term memory. These designs empower Co2In to accumulate knowledge with high plasticity and stability. We evaluate Co2In on CIFAR-10, CIFAR-100, and Tiny-ImageNet under exemplar-free Class-IL and Task-IL settings. Experimental results show that Co2In achieves state-of-the-art performance with efficient memory consumption.

Figures & tables

Fig. 1: Diagram of memory-based IL methods, Atkinson-Shiffrin human memory model [ 8 ] , and the proposed Co 2 In . (a) Memory-based IL methods prevent the model from forgetting when learning by replaying old-task exemplars or restricting gradient directions. (b) In the human brain, information is temporally stored in short-term memory. After repeated rehearsal, the information will be permanently remembered in long-term memory. (c) Inspired by the human brain, Co 2 In acquires new knowledge with working memory and integrates it into long-term memory for storage.

Notation	Description
General Notations
$\left\\|.\right\\|_{1}$	$\ell_{1}$ norm: sum of absolute values of all elements.
$\left\\|.\right\\|_{F}$	Frobenius norm: $\sqrt{\sum_{i,j}\|a_{ij}\|^{2}}$ for a matrix $\bm{A}$ .
$\text{SVD}(.)$	Singular Value Decomposition: $\text{SVD}(\bm{X})=\bm{U}\bm{\Sigma}\bm{V}^{\top}$ , where $\bm{U}$ and $\bm{V}$ are orthogonal, and $\bm{\Sigma}$ is diagonal.
$L_{ce}(.)$	The cross-entropy loss.
$[.,.]$	The concatenation operation.

TABLE I: Mathematical notations and their descriptions.

Fig. 2: Illustration of the learning process of Co 2 In . (a) The task- $t$ training process of Co 2 In , including knowledge encoding, knowledge consolidation, and knowledge integration steps. (b) In the knowledge consolidation step, Co 2 In recognizes new pattern basis and projects the parameters to form compact knowledge representations. (c) In the knowledge integration step, Co 2 In reorganizes the long-term memory into a unified neural network.

Fig. 3: Diagram of the zero-padding operation during multiplying the task-

t

pattern basis

\bm{B}_{t}^{l}\in\mathbb{R}^{p^{l+1}\times r_{t}^{l}}

with previously learned task-

j

(

j<t

) knowledge representation

\bm{A}_{j}^{l}\in\mathbb{R}^{r_{j}^{l}\times p^{l}}

in Eq. ( 6 ).

Fig. 4: Diagram of converting a convolutional layer into a linear layer.

w^{l}

h^{l}

, and

c^{l}

denote the width, height, and channel of the input tensor, respectively;

w^{l+1}

h^{l+1}

, and

c^{l+1}

correspond to the width, height, and channel of the output feature tensor, respectively, and

k^{l}

signifies the size of the convolution kernel.

Fig. 5: Diagram of the

l

-th convolutional layer of long-term memory after knowledge integration, which turns into a group convolutional layer (corresponding to knowledge representations) and a

1\times 1

convolutional layer (corresponding to pattern basis). Between them, we supplement an additional zero-channel padding operation to match the dimension.

Fig. 6: Contour map for logits corresponding to a task-1 class (cluster with the red star) output by a piece-wise linear or quadratic classifier trained with task-1 samples. The linear classifier accurately identifies task-1 samples (a). Due to linear extrapolation, it produces relatively high logits for task-2 samples (b), which could result in misclassification. In contrast, the logits output by the quadratic classifier can not only contribute reasonable classification within task-1 samples (c) but is also suitable for task-2 samples (d).

Methods	Venue	Buffer	CIFAR-10		CIFAR-100		Tiny-ImageNet
Methods	Venue	Buffer	Class-IL	Task-IL	Class-IL	Task-IL	Class-IL	Task-IL
Joint	-	-	91.29	98.07	70.31	91.18	58.07	82.01
GEM [ 10 ]	NIPS2017	500	23.66 $\pm$ 1.2	85.14 $\pm$ 2.1	11.76 $\pm$ 0.7	62.80 $\pm$ 2.7	-	-
DER [ 4 ]	NIPS2020	500	70.57 $\pm$ 1.0	93.41 $\pm$ 0.3	34.24 $\pm$ 1.4	73.26 $\pm$ 1.3	17.75 $\pm$ 1.1	51.78 $\pm$ 0.9
DER++ [ 4 ]	NIPS2020	500	73.48 $\pm$ 0.3	94.10 $\pm$ 0.3	36.52 $\pm$ 2.0	74.86 $\pm$ 1.1	19.38 $\pm$ 1.4	51.91 $\pm$ 0.7
Co2L † [ 25 ]	ICCV2021	500	74.26	95.90	30.45	79.61	20.12	53.04

TABLE II: Average accuracy (%) and the standard deviation over five runs with random seeds. Joint is the model jointly trained with data from all tasks. (-) means that the result was unavailable, due to the intractable training time by our implementation (GEM) or unsuitable setting (GPM and Adam-NSCL for Class-IL). (

{\dagger}

) and (

{\ddagger}

) imply the results are quoted from HCR and the origin paper, where the standard deviations is not provided. Bold and underlined denote the best and the second-best performance. Results with (*) imply the leveraging of rotate class augmentation.

Fig. 7: Memory usage (MB) on CIFAR-100 and Tiny-ImageNet.

Methods	CIFAR-100		Tiny-ImageNet
Methods	Class-IL	Task-IL	Class-IL	Task-IL
(a) Co 2 In w/o representation compaction regularizer	34.75 $\pm$ 0.5	83.04 $\pm$ 0.5	27.05 $\pm$ 2.0	65.00 $\pm$ 2.2
(b) Co 2 In w/o basis updating	6.88 $\pm$ 5.9	36.48 $\pm$ 17.7	2.90 $\pm$ 1.0	27.23 $\pm$ 3.6
(c) Co 2 In w/o retraining	36.12 $\pm$ 0.7	81.78 $\pm$ 3.0	28.91 $\pm$ 0.7	65.68 $\pm$ 1.9
(d) Co 2 In w/o quadratic classifier	16.90 $\pm$ 0.5	79.65 $\pm$ 0.4	8.81 $\pm$ 0.1	69.31 $\pm$ 0.5
Co 2 In	37.54 $\pm$ 0.9	84.21 $\pm$ 1.0	31.23 $\pm$ 0.5	68.75 $\pm$ 0.7

TABLE III: Ablation study of Co 2 In on CIFAR-100 and Tiny-ImageNet.

Fig. 8: Incremental learning results of Co 2 In with different $\lambda$ (first row) and SVD ratio (second row) on CIFAR-100 and Tiny-ImageNet.

Dataset	Methods	Class-IL	Task-IL
CIFAR-10	OWM [ 69 ]	47.50 $\pm$ 0.5	81.00 $\pm$ 0.1
	PASS [ 70 ]	53.12 $\pm$ 0.3	86.07 $\pm$ 0.2
	EFT [ 51 ]	55.12 $\pm$ 1.9	95.09 $\pm$ 0.3
	ABD [ 14 ]	57.64 $\pm$ 0.3	95.11 $\pm$ 0.3
	FAS [ 71 ]	43.36 $\pm$ 2.8	90.89 $\pm$ 1.3
	DCPOC [ 31 ]	44.30 $\pm$ 0.6	90.43 $\pm$ 0.3

TABLE IV: Incremental learning result (ACC %) of Co 2 In with and without self-supervised technique on CIFAR-10. “RCA” indicates the rotate class augmentation technique.

Method	Partition	CIFAR-100
		Class-IL (%)	Task-IL (%)
Co 2 In	10-Split	37.54	84.21
Co 2 In w/ RCA	10-Split	29.58	84.18
Co 2 In	25-Split	21.75	91.45
Co 2 In w/ RCA	25-Split	27.37	91.03
Co 2 In	50-Split	5.02	91.53

TABLE V: Incremental learning result (ACC %) of Co 2 In with and without rotate class augmentation (RCA) with different numbers of tasks.

Fig. 9: Memory usage (MB) versus Task-IL performance (%) on CIFAR-100 and Tiny-ImageNet.

Fig. 10: Confusion matrix of fine-tuning, BMKP, and Co 2 In on CIFAR-100 and Tiny-ImageNet.

Method	CIFAR-100		Tiny-ImageNet
	Class-IL	Task-IL	Class-IL	Task-IL
LWF [ 33 ]	5.57 $\times 10^{8}$	5.57 $\times 10^{8}$	2.22 $\times 10^{9}$	2.22 $\times 10^{9}$
DER++ [ 4 ]	5.57 $\times 10^{8}$	5.57 $\times 10^{8}$	2.22 $\times 10^{9}$	2.22 $\times 10^{9}$
Co 2 In w/o knowledge integration	1.45 $\times 10^{9}$	1.45 $\times 10^{8}$	3.23 $\times 10^{9}$	3.23 $\times 10^{8}$
Co 2 In	3.49 $\times 10^{8}$	3.49 $\times 10^{7}$	1.43 $\times 10^{9}$	1.43 $\times 10^{8}$

TABLE VI: FLOPs for different methods on CIFAR-100 and Tiny-ImageNet.

Fig. 11: Diagram of ACC (%) of each task during incremental learning on CIFAR-100 and Tiny-ImageNet.

Methods	5-split CIFAR-100	10-split CIFAR-100	20-split CIFAR-100
DGR [ 28 ] †	14.4	8.1	4.1
LwF [ 33 ] †	17.0	9.2	4.7
DI [ 29 ] †	18.8	10.9	5.7
ABD [ 14 ] †	43.9	33.7	20.0
PASS [ 70 ]	45.2	30.8	17.4
CRHD [ 74 ]	46.1	34.3	21.8

TABLE VII: Class Incremental learning results (ACC %) on CIFAR100 for various numbers of tasks (5, 10, 20). Results with “

{\dagger}

” stem from ABD [ 14 ] .

Methods	20-split CIFAR-100	25-split Tiny-ImageNet
InstAParam † [ 75 ]	51.04	34.64
PackNet † [ 42 ]	67.50	52.93
BLIP † [ 76 ]	68.17	47.63
GEM † [ 10 ]	68.89	-
A-GEM † [ 11 ]	61.91	53.32
EWC † [ 38 ]	71.66	52.33

TABLE VIII: Task Incremental learning results (ACC %) on 20-split CIFAR-100 and 25-split Tiny-ImageNet. Results with (

{\dagger}

) stem from VDFD [ 36 ] .

Method	Classifier	CIFAR-100		Tiny-ImageNet
		Class-IL	Task-IL	Class-IL	Task-IL
LWF [ 33 ]	Linear	10.05 $\pm$ 0.6	63.78 $\pm$ 4.3	9.36 $\pm$ 0.1	58.61 $\pm$ 1.8
LWF [ 33 ]	Quadratic	FAIL	FAIL	FAIL	FAIL
DER++ [ 4 ]	Linear	7.42 $\pm$ 0.5	62.21 $\pm$ 2.6	6.77 $\pm$ 0.1	60.91 $\pm$ 1.3
DER++ [ 4 ]	Quadratic	FAIL	FAIL	FAIL	FAIL
GPM [ 5 ]	Linear	8.19 $\pm$ 0.1	70.93 $\pm$ 0.9	7.04 $\pm$ 0.1	59.84 $\pm$ 0.2

TABLE IX: Accuracy (%) of methods with linear and quadratic classifiers.

Number of Expanded Basis	Linear Classifier	Quadratic Classifier
0	8.98	8.50
1	12.96	24.85
2	13.84	36.14
4	14.17	36.70
8	14.86	37.10
No Restriction	16.90	37.54

TABLE X: Class-IL performance (%) of Co 2 In using linear vs. quadratic classifiers on CIFAR-100. “Number of Expanded Basis” indicates the maximum number of new basis allowed to be added in each task.

Method	Backbone	CIFAR-100					Tiny-ImageNet
		Class-IL		Task-IL		Memory (MB)	Class-IL		Task-IL		Memory (MB)
		Accuracy (%)	Flops	Accuracy (%)	Flops		Accuracy (%)	Flops	Accuracy (%)	Flops
LWF [ 33 ]	standard ResNet18	10.05	5.57 $\times 10^{8}$	63.78	5.57 $\times 10^{8}$	42.50	9.36	2.22 $\times 10^{9}$	58.61	2.22 $\times 10^{9}$	42.50
DER++ [ 4 ]	standard ResNet18	36.52	5.57 $\times 10^{8}$	74.86	5.57 $\times 10^{8}$	48.35	19.38	2.22 $\times 10^{9}$	51.91	2.22 $\times 10^{9}$	65.94
Co 2 In	standard ResNet18	39.25	3.24 $\times 10^{9}$	84.78	3.24 $\times 10^{9}$	309.32	33.25	1.19 $\times 10^{10}$	70.08	1.19 $\times 10^{9}$	270.81
Co 2 In	narrower ResNet18	37.54	3.49 $\times 10^{8}$	84.21	3.49 $\times 10^{7}$	25.53	31.23	1.43 $\times 10^{9}$	68.75	1.43 $\times 10^{8}$	23.74

TABLE XI: Comparison of FLOPs, memory usage, and performance (Class-IL and Task-IL accuracy) on CIFAR-100 and Tiny-ImageNet.

	CIFAR-100
Non-decomp. layers	Class-IL		Task-IL		Memory (MB)
	Accuracy (%)	Flops	Accuracy (%)	Flops
All decomposed	37.54	3.49 $\times 10^{8}$	84.21	3.49 $\times 10^{7}$	25.53
First 5	35.87	2.62 $\times 10^{8}$	83.37	4.34 $\times 10^{7}$	25.46
First 10	35.60	2.13 $\times 10^{8}$	81.09	5.03 $\times 10^{7}$	23.95
First 15	31.46	1.45 $\times 10^{8}$	75.33	5.53 $\times 10^{7}$	21.19

TABLE XII: Comparison with different numbers of non-decomposed layers.

Method	CIFAR-100			Tiny-ImageNet
	Class-IL	Task-IL	Memory	Class-IL	Task-IL	Memory
ACMap [ 59 ]	89.91%	98.35%	63.28 MB	86.01%	96.12%	63.28 MB
InfoLoRA [ 60 ]	85.83%	98.62%	63.28 MB	84.62%	93.58%	63.28 MB
Naive LoRA	83.44%	98.03%	63.28 MB	79.30%	91.42%	63.28 MB
Co 2 In (ours)	85.96%	98.74%	63.98 MB	84.38%	96.72%	60.38 MB

TABLE XIII: Comparison of pretrained model-based IL methods.

Task-Incremental Learning

See all 8 papers

Grow, Assess, Compress: Adaptive Backbone Scaling for Memory-Efficient Class Incremental Learning
Mar 9, 2026Adrian Garcia-Castañeda, Jon Irureta, Jon Imaz +1Storage-Efficient Continual Learning Class-Incremental Learning
Ikerlan Technology Research Center, Arrasate, Spain · Goi Eskola Politeknikoa, Mondragon Unibertsitatea, Arrasate, Spain · University of the Basque Country UPV/EHU, Spain
Knowing Beyond the Known: Reinforced Knowledge Specification for Multi-Label Class-Incremental Learning
Aug 31, 2026Aoting Zhang, Dongbao Yang, Chang Liu +3Class-Incremental Learning Multi-Label Classification
Institute of Information Engineering, Chinese Academy of2026 Sciences, Beijing, 100085, China · School of Cyber Security, University of Chinese Academy ofAug Sciences, Beijing, 101408, China · Nankai University, Tianjin, 300350, China +1
SPARS: A Stability-Plasticity-Aware Region Selection Framework for Continual Learning
Oct 14, 2026Teif Aldaajani, Huda Hakami, Maha A. ThafarRegularization-Based Continual Learning Incremental Learning
Department of Computer Science, College of Computers and Information Technology, Taif University, Taif, 21944, Saudi Arabia .

Mar 9, 2026J/K move · Enter open · S save

Grow, Assess, Compress: Adaptive Backbone Scaling for Memory-Efficient Class Incremental Learning

Adrian Garcia-Castañeda, Jon Irureta, Jon Imaz, Aizea Lojo

Ikerlan Technology Research Center, Arrasate, Spain · Goi Eskola Politeknikoa, Mondragon Unibertsitatea, Arrasate, Spain · University of the Basque Country UPV/EHU, Spain

Storage-Efficient Continual Learning Class-Incremental Learning Incremental Learning

Open paper AskPDF arXiv

Class Incremental Learning (CIL) poses a fundamental challenge: maintaining a balance between the plasticity required to learn new tasks and the stability needed to prevent catastrophic forgetting. While expansion-based methods effectively mitigate forgetting by adding task-specific parameters, they suffer from uncontrolled architectural growth and memory overhead. In this paper, we propose a novel dynamic scaling framework that adaptively manages model capacity through a cyclic "GRow, Assess, ComprEss" (GRACE) strategy. Crucially, we supplement backbone expansion with a novel saturation assessment phase that evaluates the utilization of the model's capacity. This assessment allows the framework to make informed decisions to either expand the architecture or compress the backbones into a streamlined representation, preventing parameter explosion. Experimental results demonstrate that our approach achieves state-of-the-art performance across multiple CIL benchmarks, while reducing memory footprint by up to a 73% compared to purely expansionist models.

Abstract

Task-Incremental Learning

Explore similar work

Grow, Assess, Compress: Adaptive Backbone Scaling for Memory-Efficient Class Incremental Learning

Knowing Beyond the Known: Reinforced Knowledge Specification for Multi-Label Class-Incremental Learning

SPARS: A Stability-Plasticity-Aware Region Selection Framework for Continual Learning