Homeostatic Continual Learning

Yue Jin

cs.AISep 12, 2026

Homeostatic Continual Learning

Authors: Yue Jin

Organizations: Nokia Bell Labs France

Abstract

In this paper, I formulate a Continual Learning problem and propose a method named "Homeostatic Continual Learning" that enables an AI agent to learn continuously in a changing environment without catastrophic forgetting. The core of the method is to find outliers in the environment data when the agent experiences an outlier in its output. Through this method, the agent gradually completes its model and policy and performs well in more and more contexts. I also suggest that we may use the method to build a world model where the agent factorizes the objects in the world into features, abstract objects into comparable instances of concepts and map concepts to intents through features. I discuss the works needed to render the method practical, the connections to many fields in Artificial Intelligence and the broader implications of the method.

Figures & tables

figure 1: The problem: the agent needs to identify the data streams relevant for its intent

figure 2: Problem formulation. The agent observes the data stream $X$ . It needs to identify the relevant ones $X^{r}$ and define the relevant features $F^{r}$ . It needs to learn the model $y=y(u,X^{r})$ and policy $u=\pi(X^{r},r)$ to satisfy the intent $y=r$ .

figure 3: Context. Combinations of active and static data streams form contexts.

figure 4: The Example. The agent decides the number of servers

u

to control the response time

y

in a system. It has an intent to satisfy the requirement of the response time

y=r

. It observes a data stream

X

. It needs to identify the relevant ones

X^{r}

, define the relevant features

F^{r}

, and learn the model and policy.

figure 5: Data Stream $X=(x_{0},x_{1},x_{2},x_{3})$ . Data stream $x_{0}$ is generated by the arrival rate of packets. Data stream $x_{1}$ is generated by the packet size. Data streams $x_{2}$ and $x_{3}$ are irrelevant. The initial context $c_{0}$ is from time index 0 to 39. The new context $c_{1}$ is from time index 40 to 79. Time indices from 80 to 99 have a mixture of contexts $c_{0}$ and $c_{1}$ for testing.

figure 6: Use the policy $\pi_{0}$ in 3 time indices. The subplot “Data Stream” shows the data streams. The subplot “Control” shows the action $u$ and the response time $y$ achieved. The subplot “Learning” shows the model the agent thinks the sample belongs to and whether the agent performs learning.

figure 7: Detect new context. The new context arises at the start of time index 40. At first, the agent applies the policy $\pi_{0}$ . It observes an outlier $y\neq r$ in the intent outcome. It detects a new context from this outlier. It steps away from the model $y_{0}$ and starts learning a new model and policy. Towards the end of time index 40, it satisfies the intent again. The same operation repeats in time index 41.

figure 8: Use updated model in 3 time indices. The values of $x_{1}$ are different. The agent selects the model and applies the policy accordingly and observes that the intent is satisfied.

figure 9: Learning process. From the contexts, the agent identifies the relevant data streams $X^{r}$ , defines the features $F^{r}$ out of $X^{r}$ and learns models $y(u,X^{r})$ and policies $\pi(X^{r},r)$ to satisfy its intent $y=r$ . Through the features $F^{r}$ , it abstracts individual objects into comparable instances of concepts.

figure 10: Learned knowledge. Concepts have features $F^{r}$ . Features $F^{r}$ generate data streams $X^{r}$ . The model $y(u,X^{r})$ and policy $\pi(X^{r},r)$ are learned based on the data streams $X^{r}$ to satisfy the intent $y=r$ .

	Dry road	Icy road
Flat road	X	X
Steep road	X	(?)

table 1: An example of combining features to better adapt to new contexts

figure 11: World Model. Features link concepts to intents.

figure 12: Use Spike-Timing-Dependent Plasticity. When the intent outcome has an outlier, the agent looks for outliers in the data streams. It links all the data streams that have outliers to a node and uses the node to represent the feature. It links the feature to the intent.

Explore similar work

Position: Modular Memory is the Key to Continual Learning Agents
Mar 2, 2026Vaggelis Dorovatas, Malte Schwerin, Andrew D. Bagdanov +21Continual Learning for LLM Agents Memory-Augmented Agents
Toyota Motor Europe · University of Bremen · University of Florence +17
Harness Continual Learning: Continual Adaptation Beyond Model Parameters
Aug 19, 2026Borui Kang, Jinrui Gu, Junhan Lv +3Continual Learning for LLM Agents Continual Learning
State Key Laboratory for Novel Software Technology, Nanjing University, China · University of Wollongong, Australia
When Does Continual Learning Require Learning
Jul 8, 2026Anne Harrington, Nayan Saxena, Michael Murphy +7Continual Learning for LLMs Concept Drift
1UC Berkeley · 2Independent · 3Capital Fund Management

Mar 2, 2026J/K move · Enter open · S save

Position: Modular Memory is the Key to Continual Learning Agents

Vaggelis Dorovatas, Malte Schwerin, Andrew D. Bagdanov, Lucas Caccia+20

Toyota Motor Europe · University of Bremen · University of Florence+17

Continual Learning for LLM Agents Memory-Augmented Agents

Open paper AskPDF arXiv

Foundation models have transformed machine learning through large-scale pretraining and increased test-time compute. Despite surpassing human performance in several domains, these models remain fundamentally limited in continuous operation, experience accumulation, and personalization, capabilities that are central to adaptive intelligence. While continual learning research has long targeted these goals, its historical focus on in-weight learning (IWL), i.e., updating a single model's parameters to absorb new knowledge, has rendered catastrophic forgetting a persistent challenge. Our position is that combining the strengths of In-Weight Learning (IWL) and the newly emerged capabilities of In-Context Learning (ICL) through the design of modular memory is the missing piece for continual adaptation at scale. We outline a conceptual framework for modular memory-centric architectures that leverage ICL for rapid adaptation and knowledge accumulation, and IWL for stable updates to model capabilities, charting a practical roadmap toward continually learning agents.

Abstract

Explore similar work

Position: Modular Memory is the Key to Continual Learning Agents

Harness Continual Learning: Continual Adaptation Beyond Model Parameters

When Does Continual Learning Require Learning