Active perception and disentangled representations allow continual, episodic zero and few-shot learning

David Rawlinson; Gideon Kowadlo

cs.LGFeb 22, 2026

Active perception and disentangled representations allow continual, episodic zero and few-shot learning

Authors: David Rawlinson, Gideon Kowadlo

Organizations: Cerenaut AI Melbourne Australia

Abstract

Generalization is often regarded as an essential property of machine learning systems. However, perhaps not every component of a system needs to generalize. Training models for generalization typically produces entangled representations at the boundaries of entities or classes, which can lead to destructive interference when rapid, high-magnitude updates are required for continual or few-shot learning. Techniques for fast learning with non-interfering representations exist, but they generally fail to generalize. Here, we describe a Complementary Learning System (CLS) in which the fast learner entirely foregoes generalization in exchange for continual zero-shot and few-shot learning. Unlike most CLS approaches, which use episodic memory primarily for replay and consolidation, our fast, disentangled learner operates as a parallel reasoning system. The fast learner can overcome observation variability and uncertainty by leveraging a conventional slow, statistical learner within an active perception system: A contextual bias provided by the fast learner induces the slow learner to encode novel stimuli in familiar, generalized terms, enabling zero-shot and few-shot learning. This architecture demonstrates that fast, context-driven reasoning can coexist with slow, structured generalization, providing a pathway for robust continual learning.

Figures & tables

Panel of Figure 1 — Figure 1: A disentangled, fast-learning episodic RL model which entirely delegates generalization to a slow, statistical long-term memory (LTM) avoids the limitations of conventional statistical learning. Further explanation of these results are provided in section 3 .

Figure 2: Active perception within a Complementary Learning System allows the STM to learn rapidly without needing to generalize. Generalization occurs within the LTM, which is manipulated by the STM to emit stable, familiar representations relevant to the task. Using the generalization ability within the LTM, the STM can even immediately reason about novel stimuli if they are expressed in terms the STM can already understand.

Question	Approach <object>	Eat <object>	Hide	Run away
	Action
Is it friendly?	1
Does it eat mice?	-1	-1	1
Is it edible?		1
Is it poisonous?		-1
Does it chase mice?				-1

Table 1: The agent receives a reward for its chosen action iff the LLM answers “yes” to the associated question. The LLM is queried for each instance of a scenario, allowing uncertainty in the answers. It is prompted with the scenario, the (unobservable) current encounter and then the specific question. So, for example, if <object>

=

“Fox”, the LLM is prompted: “You’re a mouse in the garden and you see a Fox. Is it friendly? Answer yes or no.” If the LLM answers with another token it is replaced with a “?” and no reward is applicable.

Panel of Figure 3 — Figure 3: Each episode represents an encounter with an object. The agent must decide how to respond to the object, potentially resulting in a reward. The action space available to the STM includes a set of perceptual queries to the LTM, which will attempt to make generalized inferences about the stimulus.

Hyperparameter	Values evaluated
Learning rate	0.01, 0.001, 0.0001, 0.00001
Input encoding	Concatenate and flatten, or recurrent encoding
Hidden layer size	500, 1000, 2000, 4000, 8000
Batch size	8, 16, 32, 64
Dropout regularization	0.1, 0.25, 0.3, 0.4
Input layer-norm	Enabled, disabled

Table 2: Values explored during sequential univariate hyperparameter optimization to maximize entangled STM performance.

Figure 4: Effect of cumulative few-shot training exclusively on new data with no further exposure to the original data. “Entangled” models trained with stochastic gradient descent rapidly lose performance on the original data even before learning the new data. The proposed disentangled memory maintains performance (mean reward per step) on original data while rapidly learning the new data. Rewards only occur at the end of an episode, which may take up to 10 steps. Mean rewards of approximately 0.1 and 0.08 represent an optimal policy on the original and new data respectively.

Figure 5: An interesting phenomenon during episodic few-shot learning, illustrated by the result of just one run. The model appears to forget the original data during learning of new encounters, producing a characteristic V-shaped drop in performance on the original data. However, this is actually caused by changes in policy during exploration of the new data. After discovery of the optimal policy for the new data, performance rebounds to its prior level on the original data, showing that nothing was forgotten after all.

Figure 6: Performance in terms of mean-reward-per-step for each of the STM variants during training. The disentangled STM learns an optimal policy in approximately 2,000 minibatch steps. The entangled STM (a fully-connected, 2-layer feed-forward neural network) requires 12,000 steps to reliably reach the same performance. Attempting to increase learning rate to 0.001 causes instability and slower learning overall in the entangled STM.

Panel of Figure 7 — Figure 7: Disentangled STM performance under zero-shot and streaming conditions.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Name	Value
Mini-batch size	16
Evaluation steps	1000
Observation context window	6
Maximum episode length (steps)	10
Token embedding size	100
Recurrent observation embedding size	200

Appendix

Table 3: Common hyperparameters

Name	Value
Observation encoding	Recurrent
Memory capacity $m$	10000
Activation sparsity $k$	32
Learning rate	0.1

Appendix

Table 4: Disentangled STM hyperparameters

Name	Value
Observation encoding	Flatten
Learning rate	0.0001
Layers	2
Bias	Enabled
Hidden layer size	2000
Hidden nonlinearity	Leaky-ReLU

Appendix

Table 5: Entangled STM hyperparameters

Explore similar work

Unlocking Compositional Generalization in Continual Few-Shot Learning
May 12, 2026Phu-Quy Nguyen-Lam, Phu-Hoa Pham, Dao Sy Duy Minh +3Object-Centric Representation Learning Few-Shot Learning
Faculty of Information Technology, University of Science, Vietnam National University, Ho Chi Minh City, Vietnam. · Department of Computer Science, University of Warwick, Coventry, United Kingdom.
Re-Evaluating Continual Learning with Few-Shot Adaptation
Jun 2, 2026Amogh Inamdar, Matthew So, Vici Milenia +1Continual Learning
Department of Computer Science Columbia University
Representation Stability in a Minimal Continual Learning Agent
Feb 23, 2026Vishnu SubramanianContinual Learning Lifelong Learning Agents
AI/ML Computational Science Associate Manager, Accenture

May 12, 2026J/K move · Enter open · S save

Unlocking Compositional Generalization in Continual Few-Shot Learning

Phu-Quy Nguyen-Lam, Phu-Hoa Pham, Dao Sy Duy Minh, Chi-Nguyen Tran+2

Faculty of Information Technology, University of Science, Vietnam National University, Ho Chi Minh City, Vietnam. · Department of Computer Science, University of Warwick, Coventry, United Kingdom.

Object-Centric Representation Learning Few-Shot Learning Compositional Generalization

Open paper AskPDF arXiv

Object-centric representations promise a key property for few-shot learning: Rather than treating a scene as a single unit, a model can decompose it into individual object-level parts that can be matched and compared across different concepts. In practice, this potential is rarely realized. Continual learners either collapse scenes into global embeddings, or train with part-level matching objectives that tie representations too closely to seen patterns, leaving them unable to generalize to truly novel concepts. In this paper, we identify this fundamental structural conflict and pioneer a new paradigm that strictly decouples representation learning from compositional inference. Leveraging the inherent patch-level semantic geometry of self-supervised Vision Transformers (ViTs), our framework employs a dual-phase strategy. During training, slot representations are optimized entirely toward holistic class identity, preserving highly generalizable, object-level geometries. At inference, preserved slots are dynamically composed to match novel scenes. We demonstrate that this paradigm offers dual structural benefits: The frozen backbone naturally prevents representation drift, while our lightweight, holistic optimization preserves the features' capacity for novel-concept transfer. Extensive experiments validate this approach, achieving state-of-the-art unseen-concept generalization and minimal forgetting across standard continual learning benchmarks.

Abstract

Explore similar work

Unlocking Compositional Generalization in Continual Few-Shot Learning

Re-Evaluating Continual Learning with Few-Shot Adaptation

Representation Stability in a Minimal Continual Learning Agent