Socialized Division and Collaboration: Rethinking Class-Incremental Learning under Optimization Conflicts

In this work, we formulate decentralized continual learning within a multi-objective optimization framework. For a given inference task t (corresponding to a common minimizer shared by the cost functions of all agents), agents collecting data in a distributed and streamed manner are only allowed to perform local computations and to exchange information with neighboring agents over the underlying communication graph. As tasks evolve sequentially over time, agents must adapt to newly arriving tasks while retaining knowledge acquired from previously learned ones. This requirement leads to the wellknown stability plasticity dilemma, where stability refers to the ability to retain previous knowledge, while plasticity refers to the ability to learn and adapt to new tasks. To address the stability challenge, agents store subsets of samples from past tasks in local memory buffers. Then, through an appropriate multiobjective formulation, the stored information is incorporated into the learning process so that parameter updates account jointly for the current task and previously learned tasks. The proposed decentralized continual learning approach is analyzed in the mean square error sense under general assumptions on the individual cost functions and gradient noise processes. The analysis reveals that cooperation among agents improves the performance of continual learning. In particular, by exchanging information with neighboring agents, decentralized collaborative learning can exploit the diversity of locally observed data and memory buffers to improve the network average mean-square deviation (MSD) across tasks. Finally, simulations illustrate the theoretical findings and the effectiveness of the method in reducing forgetting and improving the average MSD across tasks.

Class-incremental learning (CIL) is commonly evaluated under predefined schedules with fixed or nearly equal class increments, leaving irregular class-arrival scenarios underexplored. However, practical CIL systems may need to update whenever new categories emerge, without forcing them into balanced task partitions. We formalize this setting as Free-Flow Class-Incremental Learning (FFCIL), where the number of newly arriving classes can vary substantially across learning stages. We show that variable class arrivals alter the class composition of incremental training, the reliability of new-class classifier statistics, and the consistency of representations learned across increments, causing clear performance degradation in both conventional and pre-trained model (PTM)-based CIL methods. To improve robustness under FFCIL, we introduce a general framework consisting of Class-Wise Mean (CWM), which replaces instance-wise loss aggregation with class-wise averaging; Dynamic Intervention Weight Alignment (DIWA), which adjusts new-class weight calibration according to the current increment size; and Head-Agnostic Alignment (HA), which performs feature-level correction for PTM-based methods using current and previous-class feature supervision. Extensive experiments across diverse methods, datasets, and class-arrival schedules demonstrate the general impact of FFCIL and the consistent effectiveness of our framework.

Class-Incremental Learning (CIL) requires a model to continually learn new classes without forgetting old ones. A common and efficient solution freezes a pre-trained model and employs lightweight adapters, whose parameters are often forced to be orthogonal to prevent inter-task interference. However, we argue that this parameter-constraining method is detrimental to plasticity. To this end, we propose Semantic-Guided Dynamic Sparsification (SGDS), a novel method that proactively guides the activation space by governing the orientation and rank of its subspaces through targeted sparsification. Specifically, SGDS promotes knowledge transfer by encouraging similar classes to share a compact activation subspace, while simultaneously preventing interference by assigning non-overlapping activation subspaces to dissimilar classes. By sculpting class-specific sparse subspaces in the activation space, SGDS effectively mitigates interference without imposing rigid constraints on the parameter space. Extensive experiments on various benchmark datasets demonstrate the state-of-the-art performance of SGDS.