Organizations: Creative Machines Lab, Department of Mechanical Engineering, Columbia University, New York, NY · Creative Machines Lab, Department of Computer Science, Columbia University, New York, NY
A key challenge to understanding self-awareness has been a principled way of quantifying whether an intelligent system has a concept of a "self", and if so how to differentiate the "self" from other cognitive structures. We propose that the "self" can be isolated by seeking the invariant portion of cognitive process that changes relatively little compared to more rapidly acquired cognitive skills - because our self is the most persistent aspect of our experiences. We used this principle to analyze the cognitive structure of robots under two conditions: One robot learns a constant task, while a second undergoes continual learning under variable tasks. We find that robots subjected to continual learning develop an invariant subnetwork that is significantly more stable (p < 0.001) compared to the control, and that this subnetwork is also functionally important: preserving it aids adaptation while damaging it impairs performance. We validate this pattern across three different robots spanning locomotion and manipulation.
Figures & tables
Figure 1: Continual learning produces a stable self-like core. Compared to a single-task baseline, multi-behavior training yields a subnetwork that remains stable across behaviors (“persistent self”), while other components vary. Representative results from the first hidden layer of each network shown.
Figure 2: Visualization of the persistent self in static and variable conditions. Each policy is shown on its own plane, with hidden-layer units ordered by co-activation-based subnetworks, showing how the size and structure of the subnetworks change across learning. Alluvial flows connect matched neuron families across successive policies, grouped by source subnetwork → target subnetwork; dark purple denotes the self subnetwork. Flow width indicates how many matched families pass between subnetworks, and opacity encodes mean persistence score. Left: three consecutively trained walk policies produce a more fragmented organization, with flows distributed across many groupings. Right: walk → wiggle → bob reveals one dominant subnetwork that remains continuous across policies, while smaller groupings split, merge, and reroute more strongly. Results shown for the quadruped.
Figure 3: Quantitative evidence for a persistent self-like subnetwork. Shown here is one trained policy from each condition (constant-task and continual). Although both policies successfully perform the same behavior, they exhibit markedly different internal structure. The top panel shows the reordered neuron–neuron co-activation matrix with inferred subnetwork boundaries, and the bottom panel shows per-neuron persistence score in the same ordering. For a run-level overlay of this view across many plateaued snapshots and additional examples, see Sec. S5 . Results shown for the quadruped.
Figure 4: Subnetwork persistence and size: constant-task baseline vs. continual-learning. Mean persistence score (top) and self subnetwork size (bottom) across 50 cycles for both hidden layers. The continual-learning (multi-behavior) agent shows a clear separation between the self subnetwork (largest subnetwork) and the pooled task subnetwork (all remaining units), while the walk-only baseline exhibits weaker separation and comparatively small self-subnetwork sizes. Error bars indicate inter-quartile range (IQR); dashed lines denote across-cycle means. Results shown for the quadruped.
Figure 5: Reorganization concentrates in task-like regions at behavior switches. Overlay heatmaps show how much each subnetwork changes when learning a new behavior. Rows show the three robot systems; for each, the first two hidden layers are displayed. The self-like subnetwork exhibits consistently smaller change than the pooled task-like region in every system. The arm’s network is deeper, so only its first two layers are shown for comparability. (See Sec S4 for all four layers)
Figure 6: Overview of the training and comparison pipeline in the multi-behavior experiment. A single agent morphology was trained sequentially on walk, wiggle, and bob using SAC, with actor weights transferred between phases and plateau-based phase switching. After training, we compared the resulting policies on a shared set of reference states to identify neural groups that remained stable across behaviors and neural groups that reorganized more strongly. Shown for the quadruped; the same pipeline was applied to the hexapod and, with task families in place of locomotion behaviors, to the manipulation arm.
Figure 7: Sensitivity to τ in subnetwork sizes. Late-cycle mean subnetwork sizes (alive units) for the self and task subnetworks in Layers 1–2. We selected τ=0.70 as a representative operating point.
Figure 8: Sensitivity to τ in self–task separation. Late-cycle mean separation Δ (self minus task persistence score, in percentage points) for Layers 1–2 with confidence bands. We selected τ=0.70 as a representative operating point.
Figure 9: Pooled transfer overlays for freeze and lesion interventions. Left: per-transition normalized reward curves aggregated across matched self-freeze and task-freeze transfer runs. The self-freeze condition remains slightly above task-freeze across much of the evaluation window. Right: per-transition normalized reward curves aggregated across matched self-lesion, task-lesion, and original transfer runs. The task-lesion condition remains above self-lesion over much of the evaluation window. In both panels, faint traces show individual runs and bold curves show the pooled mean trends.
Figure 10: Hexapod: subnetwork persistence and size across cycles. Mean persistence score (left) and subnetwork size (right) across 30 training cycles, mirroring the format of Figure 4 in the main text. The self subnetwork consistently exhibits higher mean persistence than the task subnetwork in both layers throughout training.
Figure 11: Hexapod: sensitivity to τ . Top: Late-cycle mean separation Δ for Layers 1–2, averaged over cycles 10–30. Solid lines indicate valid decompositions; dashed lines indicate degenerate regimes. Bottom: Late-cycle mean subnetwork sizes for Layers 1–2. Shaded invalid zones mark degenerate regimes.
Figure 12: Manipulation tasks used in the Continual World replication. The six Meta-World manipulation tasks used across the four post-validation task sets, spanning tool use, pushing, pressing, sliding, closing, and alignment-sensitive extraction.
Figure 13: triC subnetwork change heatmaps. The self-like subnetwork changes less than the task-like remainder, with lower relative change in earlier layers.
Figure 14: triA. Manipulation results for hammer-v1 , faucet-close-v1 , and peg-unplug-side-v1 .
Figure 15: triC. Manipulation results for hammer-v1 , push-wall-v1 , and window-close-v1 .
Figure 16: quadA. Manipulation results for faucet-close-v1 , handle-press-side-v1 , window-close-v1 , and peg-unplug-side-v1 .
Figure 17: pentaA. Manipulation results for hammer-v1 , push-wall-v1 , faucet-close-v1 , stick-pull-v1 , and peg-unplug-side-v1 .
Figure 18: Overlay view of activation matrices and persistence-scores. Each composite is formed by alpha-blending activation-matrix visualizations from all successful plateaued snapshots within a continual-learning run (cycles 16–50), so consistent structure accumulates while inconsistent correlations fade. Top: single-behavior (walk-only) controls remain diffuse and “spaghetti-like” with no equally clean core. Bottom: multi-behavior training yields a clear, recurring self-like block in the first hidden layer. In the second hidden layer, both settings show substantial structure, but multi-behavior policies retain a clearer self–task distinction in persistence-scores, while single-behavior controls show a weaker separation.
Figure 19: Examples of Fig. 3 quantitative outputs (1/3). Run 2 Cycle 022 (Walk/Wiggle).
Figure 20: Examples of Fig. 3 quantitative outputs (2/3). Run 2 Cycle 022 (Bob) + Run 7 Cycle 036 (Walk).
Figure 21: Examples of Fig. 3 quantitative outputs (3/3). Run 7 Cycle 036 (Wiggle/Bob).
Figure 22: Single-behavior baselines: bob-only (top row) and wiggle-only (bottom row). Left column: Persistence score of the self and task subnetworks across phases. Right column: Size of the self and task subnetworks across phases.
Figure 23: Example rollout of the training schedule. An example training trace illustrating phase switching across two full walk → wiggle → bob cycles with the plateau-stop logic described above.
Figure 24: Quadruped in simulation and hardware: (left) CAD model showing the disk-shaped torso and four 2-DOF legs, and (right) the physical robot used for trajectory replay.
Quantity
Ant (sim)
Our quadruped
Torso radius
0.25 m
0.08 m
Torso diameter
0.50 m
0.16 m
Hip radius
0.283 m
0.09 m
Hip/torso radius
≈1.13
≈1.13
Upper leg length
≈0.28 m
0.08 m
Lower leg length
≈0.57 m
0.16 m
Table 2: Key dimensions of the MuJoCo/Gym Ant and our Ant-like quadruped.
Autonomous robots continuously encounter objects, changes, and situations, and every event admitted into cognition should receive an appropriate cognitive treatment rather than remain untreated until an explicit task requires attention. However, existing task-driven, reactive, or fixed-reasoning approaches generally process only selected events or apply predefined reasoning procedures, making it difficult to provide continuous cognitive coverage with differentiated treatment. This paper proposes a continuous cognitive coverage framework in which every cognitively admitted event is assigned an event-dependent cognitive treatment according to its state, context, and history. Different events may therefore invoke description, memory, risk prediction, planning, diagnosis, analogy, or other learned treatments. Familiar events can be processed automatically by learned mechanisms, whereas unfamiliar or uncertain events invoke explicit deliberation or fallback reasoning. Multiple cognitive processes can be suspended, resumed, and interleaved so that cognitive processing continues as new events arrive or existing events await evidence. Validated experiences are continuously learned to automate, refine, and revise event-specific treatments. Experiments achieve 96.76% structured treatment accuracy with 93.66% automatic processing, 92.64% cognitive coverage under bursty-delayed workloads, and 79.53% continual-learning joint accuracy, with novel-event reuse reaching 100% automatic processing.
Hong Su
School of Computer Science, Chengdu University of Information Technology, Chengdu, China
As learning-based robotic controllers are typically trained offline and deployed with fixed parameters, their ability to cope with unforeseen changes during operation is limited. Biologically inspired, this work presents a framework for online Continual Reinforcement Learning that enables automated adaptation during deployment. Building on DreamerV3, a model-based Reinforcement Learning algorithm, the proposed method leverages world model prediction residuals to detect out-of-distribution events and automatically trigger finetuning. Adaptation progress is monitored using both task-level performance signals and internal training metrics, allowing convergence to be assessed without external supervision and domain knowledge. The approach is validated on a variety of contemporary continuous control problems, including a quadruped robot in high-fidelity simulation, and a real-world model vehicle. Relevant metrics and their interpretation are presented and discussed, as well as resulting trade-offs described. The results sketch out how autonomous robotic agents could once move beyond static training regimes toward adaptive systems capable of self-reflection and -improvement during operation, just like their biological counterparts.
Fabian Domberg, Georg Schildbach
Autonomous Systems Lab (ASL), Institute for Electrical Engineering in Medicine, University of Lübeck, 23562 Lübeck, Germany
Recent advances in robot learning have produced increasingly capable embodied agents. Yet comparatively less attention has been given to a more basic form of competence that animals exhibit continuously: the ability to remain situated, responsive, and behaviorally coherent as physical, environmental, and social demands change over time. We propose the ethological behavioral substrate as a conceptual lens for studying this form of competence in artificial agents. Rather than treating these behaviors that animals exhibit as a set of isolated skills, we argue that their continual coordination under competing demands constitutes an important and underexplored target for modern robot learning. We further propose robotic animal companions as a useful research setting for studying sustained interaction and adaptation in human-centered environments. Such systems provide an opportunity to investigate how social behavior, memory, and continual learning develop over long periods of interaction. This perspective motivates further investigation of how such persistent behavioral competence may complement higher-level capabilities in embodied agents.
Samiyuru Menik, Hemadri Jayalath
School of Computing University of Georgia Athens, GA, USA