Same World, Differently Given: History-Dependent Perceptual Reorganization in Artificial Agents

Hongju Pae

cs.AIApr 6, 2026

Same World, Differently Given: History-Dependent Perceptual Reorganization in Artificial Agents

Authors: Hongju Pae

Organizations: Active Inference Institute, Crescent City, CA, USA

Abstract

What kind of internal organization would allow an artificial agent not only to adapt its behavior, but to sustain a history-sensitive perspective on its world? I present a minimal architecture in which a slow perspective latent $g$ feeds back into perception and is itself updated through perceptual processing. This allows identical observations to be encoded differently depending on the agent's accumulated stance. The model is evaluated in a minimal gridworld with a fixed spatial scaffold and sensory perturbations. Across analyses, three results emerge. First, the perspective latent reorganizes perceptual encoding: identical observations are represented differently depending on prior experience, and this reorganization of salience gating replicates across runs, with five of 16 gating dimensions changing direction consistently across 30 independent runs after correction for multiple comparisons. Second, only adaptive self-modulation yields the characteristic growth-then-stabilization dynamic of the perspective latent, unlike rigid or always-open update regimes. Third, perturbation history is followed by reduced adaptive plasticity after nominal conditions are restored, showing a directionally consistent trend across seeds. Gross behavior remains stable throughout the analysis, suggesting that the dominant reorganization is perceptual rather than behavioral. Together, these findings identify a minimal mechanism for history-dependent perspectival organization in artificial agents.

Figures & tables

Var.	Dim.	Role	Timescale	Stage 2
$x_{t}$	8	local observation patch	step	(input only)
$z_{\mathrm{raw}}$	16	pre-gating percept	step	frozen
$z_{t}$	16	gated perceptual latent	step	FiLM trainable
$p_{t}$	8	action-trace embedding	step	frozen
$g_{t}$	12	perspective latent	cross-episode	trainable
$\alpha_{t}$	1	plasticity rate $\in[0.03,0.30]$	step (state-dep.)	trainable

Table 1: Notation, dimensionality, functional role, characteristic timescale, and second-stage training status of all model variables. “Frozen” components are loaded from first-stage checkpoints and not updated in stage two.

Panel of Figure 1 — Figure 1: Schematic overview of the extended agent architecture. Gray, green, purple, and orange denote the environment, perceptual encoding, perspective dynamics, and action pathway. The main additions are the feedback links between perception and perspective: salience gating (purple arrow), in which $g_{t-1}$ modulates perceptual encoding through the FiLM gate, and self-modulating plasticity (green arrow), in which gated perception influences the plasticity rate $\alpha_{t}$ . Together with the update path $g_{t-1}\rightarrow\alpha_{t}\rightarrow g_{t}$ , these implement the central loop of the model: perspective shapes perception, and perception regulates perspective. The updated $g_{t}$ then biases the fast action pathway through $s_{t}$ . Thin black arrows indicate the main forward flow; gray arrows indicate auxiliary or implicit connections. For clarity, the observation decoder, prediction-error features, and explicit proprioceptive pathway are omitted.

Figure 2: Gridworld environment visualization. This Pygame-based simulator renders a fixed $23\times 7$ gridworld with a left-to-right observation noise gradient: left-side cells are noisier and right-side cells are more reliable. For analysis, the 23 columns are divided into 5 reporting zones (vertical white lines). Background color indicates noise level, from red (high) to teal (low). The agent (white circle) starts at the grid center. The top overlay shows timestep, position, reporting zone, action, perturbation status and trace, number of scheduled perturbations, and local observation noise $\sigma$ . The right panel shows the current 8-neighbor local observation patch $x_{t}$ . This visualization is illustrative only; all simulations are run headless.

Figure 3: Perspective reorganizes perception of the same input. The comparison perspective $g$ was obtained from a single illustrative mixed perturbation run ( $n_{P}=0\!\to\!4\!\to\!0$ ); cross-run statistics are reported in the text. (a) PCA projection of probe encodings under the late post-perturbation perspective state $g_{2}$ and the null condition $g\!=\!0$ , shown with covariance ellipses. Cluster centroids are marked with crosses. Encoding-geometry statistics (variance ratio, participation ratio, principal-axis rotation) are computed across all 30 runs and reported in the text. (b) Signed per-dimension difference $z_{t}(g_{2})-z_{t}(g\!=\!0)$ averaged across probe observations. (c) Per-dimension FiLM salience-gating coefficients $\gamma$ in Block 0 (pre-perturbation, $n_{P}\!=\!0$ ) and Block 2 (post-perturbation, $n_{P}\!=\!0$ ) of the mixed-history run.

Figure 4: Ablation analysis of self-modulating plasticity. (a) Episode-wise adaptive plasticity $\alpha$ with perturbation ( $n_{P}=4$ ) and without perturbation ( $n_{P}=0$ ). In both conditions, $\alpha$ rises rapidly early in training, but only the perturbed condition later declines below baseline. (b) Perspective magnitude $\|g\|$ across four update regimes: Adaptive with perturbation, Adaptive baseline, Rigid ( $\alpha=0.05$ ), and Open ( $\alpha=0.80$ ). The rigid regime shows continued growth, the open regime remains relatively flat, and the adaptive regimes lie between these extremes, with slightly larger late-stage $\|g\|$ under perturbation.

Figure 5: Plasticity under matched three-block histories. Mean adaptive plasticity $\alpha$ is compared between the first and final blocks across three schedules matched in total duration: Baseline ( $n_{P}=0\!\to\!0\!\to\!0$ ), Mixed perturbation ( $n_{P}=0\!\to\!4\!\to\!0$ ), and Persistent perturbation ( $n_{P}=4\!\to\!4\!\to\!4$ ). Bars show hierarchical medians, points show individual seed run values, and error bars indicate IQR. The block-wise decline is smallest under Baseline ( $0.285$ to $0.280$ ), largest under Mixed ( $0.279$ to $0.229$ ), and comparable under Persistent ( $0.293$ to $0.261$ ); see text for inferential statistics.

Cognitive Architectures for AI Agents

Cognitive architectures for AI agents organize perception, memory, goals, reasoning, and action into systems for persistent, adaptive behavior.

See all 86 papers

Body-Grounded Perspective Formation and Conative Attunement in Artificial Agents
May 16, 2026Hongju PaeEmbodied AI Embodied Cognition
Active Inference Institute, CA, USA
Artifacts as Memory Beyond the Agent Boundary
Apr 9, 2026John D. Martin, Fraser Mince, Esra'a Saleh +1Memory-Augmented Agents Reinforcement Learning
Openmind Research Institute · University of Alberta · Cohere Labs Community +3
Agents in the Large: Perception-Centered Architecture for Persistent Agents
Aug 31, 2026Shihan Dou, Haoxiang Jia, Shichun Liu +14Cognitive Architectures for AI Agents Lifelong Learning Agents
NLP Lab, Fudan University · PL Lab, Peking University · CCDS, Nanyang Technological University +1

May 16, 2026J/K move · Enter open · S save

Body-Grounded Perspective Formation and Conative Attunement in Artificial Agents

Hongju Pae

Active Inference Institute, CA, USA

Embodied AI Embodied Cognition

Open paper AskPDF arXiv

This paper proposes a minimal architecture for body-grounded perspective formation in artificial agents. Extending prior work, the model introduces an interoceptive viability signal, a Fisher-style metric over fused exteroceptive-interoceptive states, and a conative alignment mechanism linking bodily tendency to action readiness. In a reward-free gridworld, conation converts learned bodily tendency into stable body-directed behavior, while body-to-perspective routing allows bodily perturbations to leave a recoverable geometric residue in the perspective latent. This study shows how minimal structural conditions for artificial subjectivity can be operationalized in the phenomenological sense, through the embodied organization of how a world is given to an agent.

Abstract

Cognitive Architectures for AI Agents

Explore similar work

Body-Grounded Perspective Formation and Conative Attunement in Artificial Agents

Artifacts as Memory Beyond the Agent Boundary

Agents in the Large: Perception-Centered Architecture for Persistent Agents