cs.ROMar 25, 2026
SaveEvidence of an Emergent "Self" in Continual Robot Learning
Organizations: Creative Machines Lab, Department of Mechanical Engineering, Columbia University, New York, NY · Creative Machines Lab, Department of Computer Science, Columbia University, New York, NY
Abstract
A key challenge to understanding self-awareness has been a principled way of quantifying whether an intelligent system has a concept of a "self", and if so how to differentiate the "self" from other cognitive structures. We propose that the "self" can be isolated by seeking the invariant portion of cognitive process that changes relatively little compared to more rapidly acquired cognitive skills - because our self is the most persistent aspect of our experiences. We used this principle to analyze the cognitive structure of robots under two conditions: One robot learns a constant task, while a second undergoes continual learning under variable tasks. We find that robots subjected to continual learning develop an invariant subnetwork that is significantly more stable (p < 0.001) compared to the control, and that this subnetwork is also functionally important: preserving it aids adaptation while damaging it impairs performance. We validate this pattern across three different robots spanning locomotion and manipulation.
Figures & tables
Figure 1: Continual learning produces a stable self-like core. Compared to a single-task baseline, multi-behavior training yields a subnetwork that remains stable across behaviors (“persistent self”), while other components vary. Representative results from the first hidden layer of each network shown.
Figure 2: Visualization of the persistent self in static and variable conditions. Each policy is shown on its own plane, with hidden-layer units ordered by co-activation-based subnetworks, showing how the size and structure of the subnetworks change across learning. Alluvial flows connect matched neuron families across successive policies, grouped by source subnetwork target subnetwork; dark purple denotes the self subnetwork. Flow width indicates how many matched families pass between subnetworks, and opacity encodes mean persistence score. Left: three consecutively trained walk policies produce a more fragmented organization, with flows distributed across many groupings. Right: walk wiggle bob reveals one dominant subnetwork that remains continuous across policies, while smaller groupings split, merge, and reroute more strongly. Results shown for the quadruped.
Figure 3: Quantitative evidence for a persistent self-like subnetwork. Shown here is one trained policy from each condition (constant-task and continual). Although both policies successfully perform the same behavior, they exhibit markedly different internal structure. The top panel shows the reordered neuron–neuron co-activation matrix with inferred subnetwork boundaries, and the bottom panel shows per-neuron persistence score in the same ordering. For a run-level overlay of this view across many plateaued snapshots and additional examples, see Sec. S5 . Results shown for the quadruped.
Figure 4: Subnetwork persistence and size: constant-task baseline vs. continual-learning. Mean persistence score (top) and self subnetwork size (bottom) across 50 cycles for both hidden layers. The continual-learning (multi-behavior) agent shows a clear separation between the self subnetwork (largest subnetwork) and the pooled task subnetwork (all remaining units), while the walk-only baseline exhibits weaker separation and comparatively small self-subnetwork sizes. Error bars indicate inter-quartile range (IQR); dashed lines denote across-cycle means. Results shown for the quadruped.
Figure 5: Reorganization concentrates in task-like regions at behavior switches. Overlay heatmaps show how much each subnetwork changes when learning a new behavior. Rows show the three robot systems; for each, the first two hidden layers are displayed. The self-like subnetwork exhibits consistently smaller change than the pooled task-like region in every system. The arm’s network is deeper, so only its first two layers are shown for comparability. (See Sec S4 for all four layers)
Figure 6: Overview of the training and comparison pipeline in the multi-behavior experiment. A single agent morphology was trained sequentially on walk, wiggle, and bob using SAC, with actor weights transferred between phases and plateau-based phase switching. After training, we compared the resulting policies on a shared set of reference states to identify neural groups that remained stable across behaviors and neural groups that reorganized more strongly. Shown for the quadruped; the same pipeline was applied to the hexapod and, with task families in place of locomotion behaviors, to the manipulation arm.
Figure 7: Sensitivity to in subnetwork sizes. Late-cycle mean subnetwork sizes (alive units) for the self and task subnetworks in Layers 1–2. We selected as a representative operating point.
Figure 8: Sensitivity to in self–task separation. Late-cycle mean separation (self minus task persistence score, in percentage points) for Layers 1–2 with confidence bands. We selected as a representative operating point.
Figure 9: Pooled transfer overlays for freeze and lesion interventions. Left: per-transition normalized reward curves aggregated across matched self-freeze and task-freeze transfer runs. The self-freeze condition remains slightly above task-freeze across much of the evaluation window. Right: per-transition normalized reward curves aggregated across matched self-lesion, task-lesion, and original transfer runs. The task-lesion condition remains above self-lesion over much of the evaluation window. In both panels, faint traces show individual runs and bold curves show the pooled mean trends.
Figure 10: Hexapod: subnetwork persistence and size across cycles. Mean persistence score (left) and subnetwork size (right) across 30 training cycles, mirroring the format of Figure 4 in the main text. The self subnetwork consistently exhibits higher mean persistence than the task subnetwork in both layers throughout training.
Figure 11: Hexapod: sensitivity to . Top: Late-cycle mean separation for Layers 1–2, averaged over cycles 10–30. Solid lines indicate valid decompositions; dashed lines indicate degenerate regimes. Bottom: Late-cycle mean subnetwork sizes for Layers 1–2. Shaded invalid zones mark degenerate regimes.
Figure 12: Manipulation tasks used in the Continual World replication. The six Meta-World manipulation tasks used across the four post-validation task sets, spanning tool use, pushing, pressing, sliding, closing, and alignment-sensitive extraction.
Figure 13: triC subnetwork change heatmaps. The self-like subnetwork changes less than the task-like remainder, with lower relative change in earlier layers.
Figure 14: triA. Manipulation results for hammer-v1 , faucet-close-v1 , and peg-unplug-side-v1 .
Figure 15: triC. Manipulation results for hammer-v1 , push-wall-v1 , and window-close-v1 .
Figure 16: quadA. Manipulation results for faucet-close-v1 , handle-press-side-v1 , window-close-v1 , and peg-unplug-side-v1 .
Figure 17: pentaA. Manipulation results for hammer-v1 , push-wall-v1 , faucet-close-v1 , stick-pull-v1 , and peg-unplug-side-v1 .
Figure 18: Overlay view of activation matrices and persistence-scores. Each composite is formed by alpha-blending activation-matrix visualizations from all successful plateaued snapshots within a continual-learning run (cycles 16–50), so consistent structure accumulates while inconsistent correlations fade. Top: single-behavior (walk-only) controls remain diffuse and “spaghetti-like” with no equally clean core. Bottom: multi-behavior training yields a clear, recurring self-like block in the first hidden layer. In the second hidden layer, both settings show substantial structure, but multi-behavior policies retain a clearer self–task distinction in persistence-scores, while single-behavior controls show a weaker separation.
Figure 19: Examples of Fig. 3 quantitative outputs (1/3). Run 2 Cycle 022 (Walk/Wiggle).
Figure 20: Examples of Fig. 3 quantitative outputs (2/3). Run 2 Cycle 022 (Bob) + Run 7 Cycle 036 (Walk).
Figure 21: Examples of Fig. 3 quantitative outputs (3/3). Run 7 Cycle 036 (Wiggle/Bob).
Figure 22: Single-behavior baselines: bob-only (top row) and wiggle-only (bottom row). Left column: Persistence score of the self and task subnetworks across phases. Right column: Size of the self and task subnetworks across phases.
Figure 23: Example rollout of the training schedule. An example training trace illustrating phase switching across two full walk wiggle bob cycles with the plateau-stop logic described above.
Figure 24: Quadruped in simulation and hardware: (left) CAD model showing the disk-shaped torso and four 2-DOF legs, and (right) the physical robot used for trajectory replay.
| Quantity | Ant (sim) | Our quadruped |
|---|---|---|
| Torso radius | m | m |
| Torso diameter | m | m |
| Hip radius | m | m |
| Hip/torso radius | ||
| Upper leg length | m | m |
| Lower leg length | m | m |
Table 2: Key dimensions of the MuJoCo/Gym Ant and our Ant-like quadruped.
More in this topic
Continual Robot Learning
Continual robot learning develops robots that acquire new skills and adapt to changing conditions while retaining previously learned behavior.
See all 89 papers