Goal-conditioned systems assume goals are provided externally. We ask whether attentional priorities can emerge endogenously from an agent's internal cognitive state. We propose a priority function that generates observation targets from three epistemic gaps: ignorance (posterior variance), surprise (prediction error), and staleness (temporal decay of confidence in unobserved variables). We validate this in two systems: a minimal attention-allocation environment (2,000 runs) and a modular, partially observable world (500 runs). Ablation shows each component is necessary. A key finding is metric-dependent reversal: under global prediction error, coverage-based rotation wins; under change detection latency, priority-guided allocation wins, with advantage growing monotonically with dimensionality (d = -0.95 at N=48, p < 10^-6). Detection latency follows a power law in attention budget, with a steeper exponent for priority-guided allocation (0.55 vs. 0.40). When the decay rate is made learnable per variable, the system spontaneously recovers environmental volatility structure without supervision (t = 22.5, p < 10^-6). We demonstrate that epistemic gaps alone, without external reward, suffice to generate adaptive priorities that outperform fixed strategies and recover latent environmental structure.
Figures & tables
Strategy
Error ( ± SD)
vs. Pri. ( p )
d
Attn. (sw.)
Random
0.152±0.032
<10−6
−0.15
0.500
Priority
0.147±0.030
—
—
0.534
Var-only
0.148±0.031
<0.001
−0.11
0.509
Rotation
0.132±0.024
<10−8
+0.54
0.500
Error
0.134±0.025
<10−8
+0.48
0.533
Table 1: Minimal system ( N=6 , 2000 runs). Global prediction error, last 50%. Negative d = Priority is better.
Figure 1: Detection latency vs. number of variables ( N ), budget =1 . Rotation degrades linearly with N ; priority-guided allocation remains approximately constant. All comparisons p<10−6 .
N
Rand.
Rot.
Pri.
Gap
d
8
4.01
3.90
3.25
0.65
−0.27
16
4.15
5.93
3.90
2.02
−0.58
24
4.10
6.79
4.10
2.70
−0.72
32
3.98
6.99
4.23
2.76
−0.71
48
4.11
7.66
4.03
3.63
−0.95
Table 3: Detection latency (ticks) by strategy and dimensionality. Budget =1 , 500 runs. Priority vs. Rotation : all p<10−6 .
Figure 2: Detection latency vs. attention budget ( N=48 , log-log). Priority-guided allocation exhibits a steeper power law exponent (0.55 vs. 0.40), extracting greater marginal benefit from additional budget.
Var
Mod
Vol.
λi
Var
Mod
Vol.
λi
0
0
Hi
0.296
8
2
Lo
0.198
1
0
Hi
0.292
9
2
Lo
0.200
2
0
Hi
0.291
10
2
Lo
0.186
3
0
Hi
0.300
11
2
Lo
0.199
4
1
Hi
0.284
12
3
Lo
0.213
5
1
Hi
0.283
13
3
Lo
0.206
Table 4: Learned per-variable λi (50-run mean). Variables 0–7: high volatility ( ptrans=0.15 ); variables 8–15: low volatility ( ptrans=0.02 ). Paired t -test on per-run group means: t(49)=22.5 , p<10−6 .
Figure 3: Learned λi per variable (50-run mean). Red: high-volatility modules ( ptrans=0.15 ); blue: low-volatility modules ( ptrans=0.02 ). The system spontaneously differentiates volatility structure without supervision ( t=22.5 , p<10−6 ).
Paying attention improves performance, but attention is metabolically costly, so how should a resource-efficient agent allocate it? We study optimal allocation strategies using a normative model of a signal detection task in which attention comes at a cost. The model reveals that optimal attention is temporally structured in one of two patterns depending on task conditions: when attention costs penalize intense focus, optimal attention ramps up as evidence for the signal accumulates, but with less prohibitive attention costs, optimal attention fluctuates rhythmically. In cases with rhythmic attention, allocation frequency increases with task parameters such as reward magnitude for successful detection, signal brevity, and signal frequency. We argue that rhythmic attention emerges naturally in an agent whose belief updates are dominated by a stable temporal prior and not by the likelihood of new sensory observations. Our results characterize the conditions under which rhythmic vs. ramping attention is optimal and offer a normative account of attentional fluctuations observed in sustained attention tasks.
Neuroscience Institute Carnegie Mellon University, Pittsburgh, PA 15213 · Neuroscience Baylor College of Medicine, Houston, TX 77030 · Neuroscience Institute and Department of Machine Learning Carnegie Mellon University, Pittsburgh, PA 15213
Biological systems must regulate competing needs under limited perceptual bandwidth, where sharpening one estimate costs the capacity to sharpen the others. Any fixed-budget system therefore has to decide where to allocate its perceptual precision. We study this in a foraging agent that must keep several bodily needs satisfied to survive, modelled with active inference. At each step it reads its own body-state beliefs, identifies the most-needed channel, and reallocates a fixed budget of interoceptive precision toward it, so that the same precision-shaped likelihood feeds both belief update and planning. In AffectWorld, a four-channel foraging gridworld, this selective allocation more than doubles learning-phase survival at matched budget against a uniform-precision agent (0.414 vs 0.199 across 11 layouts, n=32 seeds each, paired cluster-bootstrap p≤10−4). Two further results sharpen the mechanism. The benefit runs through planning as well as perception, since denying the shaped likelihood to the planner alone removes about half of it. It is also need-aligned, since aiming precision at the least-needed channel does worse than spreading it evenly. The attended channel additionally learns its own dynamics about twice as fast, and stays ahead even at matched observation count, a behavioural trace of the same precision routing, visible in learning speed, not survival.
St John Grimbly, Nicolas Kuske, Evert A. Boonstra +7
Dept. of Mathematics & Applied Mathematics, Univ. of Cape Town, South Africa · Neuroscience Institute, Univ. of Cape Town, South Africa · MIND Institute and CSAM, University of the Witwatersrand, South Africa +3
A core question in decision-making for agents is whether the relative priorities of competing lower-level objectives can be determined by emotional preferences autonomously generated by higher-level goals, rather than being externally prespecified. Under changing external environments and evolving internal states, emotions play an important functional role in regulating the relative priorities of competing goals. Inspired by the goal-directed theory of emotion, this paper studies how such preference regulation can be computationally realized through reinforcement learning. We first propose a conception of emergent emotional preference: a high-level goal autonomously induces state-dependent preferences over competing lower-level objectives. This conception is built upon a framework consisting of a multi-objective reinforcement learning inner controller and an outer preference generator. The inner controller provides a repertoire of preference-conditioned goal-directed behaviors, while the outer preference generator learns a mapping from the current state to objective preferences through reinforcement learning on a high-level goal. We operationalize emotional preference as a state-dependent regulation of relative goal priorities that emerges through optimization. Furthermore, we characterize the policy space induced by preference regulation and derive an upper bound on the optimality gap in terms of the representation error of the inner behavioral repertoire. We show that the gap vanishes when the optimal policy can be represented by the available preference-conditioned policies. Experiments in self-constructed multi-objective exploration environments show that the learned preference function exhibits contextual priority switching, graded trade-offs, and temporal persistence, and outperforms the evaluated fixed-preference and handcrafted-preference strategies.
Shiqi Liu, Yihua Tan, Hu Fu +1
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Luoyu Street, Wuhan, 430074, Hubei, China