Organizations: Neuroscience Institute Carnegie Mellon University, Pittsburgh, PA 15213 · Neuroscience Baylor College of Medicine, Houston, TX 77030 · Neuroscience Institute and Department of Machine Learning Carnegie Mellon University, Pittsburgh, PA 15213
Paying attention improves performance, but attention is metabolically costly, so how should a resource-efficient agent allocate it? We study optimal allocation strategies using a normative model of a signal detection task in which attention comes at a cost. The model reveals that optimal attention is temporally structured in one of two patterns depending on task conditions: when attention costs penalize intense focus, optimal attention ramps up as evidence for the signal accumulates, but with less prohibitive attention costs, optimal attention fluctuates rhythmically. In cases with rhythmic attention, allocation frequency increases with task parameters such as reward magnitude for successful detection, signal brevity, and signal frequency. We argue that rhythmic attention emerges naturally in an agent whose belief updates are dominated by a stable temporal prior and not by the likelihood of new sensory observations. Our results characterize the conditions under which rhythmic vs. ramping attention is optimal and offer a normative account of attentional fluctuations observed in sustained attention tasks.
Figures & tables
Figure 1: Formal task structure. ( A ) In this POMDP, the agent constructs beliefs bt that reflect inferences about the latent world state st using observations ot controlled by the “attend” action at . The agent uses these beliefs to plan “commit” actions that may obtain rewards. ( B ) Specific world states and transitions in the finite state machine modeling this task. The noise state has a geometric duration. The signal lasts throughout a sequence of m signal states, with a tighter distribution the higher m is. Commit actions produce rewards when taken during the signal period and otherwise lead to a false alarm and penalty.
Figure 2: Attention policies over lower-dimension belief spaces. Attention over signal belief is shown in maroon; the background color shows if this signal belief is a level where the agent will not commit (light grey), will commit only if it observes signal (light blue), or will always commit (darker blue). ( A ) When we model a geometric distribution for signal duration using a single signal state ( m=1 ), belief is one-dimensional, so we may visualize full policies as a level of attention over total signal belief. We show a single-jump (left), 2-jump (middle) and many-jump (right) example here. ( B ) We can visualize a slice of policies with m>1 by fixing all but one signal state at 0. Here, we show an example of many-jumps policies where signal belief is concentrated in the final signal state for, from left to right, 1, 2, 3, and 4 signal states. We show and discuss more examples in the Appendix A.4 .
Figure 3: Illustration of agents’ interactions with simulated trials. Trial phase (noise: white, signal: black), observations (amber: no attention; white/black: noise/signal observation), belief, and commit action (green triangle). ( A ) agent with a rhythmic policy over a single trial of approximately 100 steps. ( B ) Same as A but with a ramping policy.
Figure 4: Rhythmicity and policy over attention cost exponent c2 . ( A ) Autocorrelation of attention allocations within a session for optimal policies under the attention cost function C(a)=0.2ac2 , for different c2 . Rhythmic policies show periodic spikes in autocorrelation; ramping policies are flat. ( B ) The corresponding policies producing the autocorrelations in A . The lines below show commit behavior (thin: never; medium: when observing signal; thick: always). ( C ) The rhythmicity metric plotted over c2 for 1, 2, 3, and 4 signal states. Rhythmicity drops sharply at approximately c2=1.7 : below this value, policies are rhythmic, and above it, policies ramp.
Figure 5: Comparison of optimal vs. constant-attention policies. ( A ) The ratio, over c2 , of mean net reward per step for optimal policies to that of a constant-attention control with the same mean attention. To keep the rest of the plot to scale, we show points where the optimal policy is ten times better, or where the constant policy has a negative net reward, on a grey bar above. ( B ) The inverse comparison: a histogram, over all solved policies, of how much of an optimal policy’s net reward a constant policy with the same mean attention captures. For visual clarity, we show only comparisons where the constant policy achieved non-negative mean reward.
Figure 6: Autocorrelations and policies over model parameters. All models shown here used a single signal state, allowing us to fully visualize their policies. Autocorrelations for models with more signal states are included in Appendix A.5 . ( A ) Autocorrelation of attention allocations within a session for optimal policies under different hit reward values. ( B ) The corresponding policies producing the autocorrelations in A . ( C ) and ( D ): the same for mean signal duration. ( E ) and ( F ): the same for mean noise duration.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: We may also visualize policies over two dimensions, allowing a look at how attention should be allocated over belief distributed in two signal states. For m=2 , this covers the full policy space. For m>2 , this enables visualizations of the policy over belief in two of the signal states while the others are fixed at 0. The same “jumps” in optimal attention can be observed here in the diagonal bands.
Figure 8: Visualization of policies and policy slices, as in Fig. 2 , except that signal belief in the slices is ( A ) concentrated in the first signal state or ( B ) distributed uniformly across all modeled signal states.
Figure 9: Autocorrelations over model parameters. The columns show models with m=1 , 2, 3, and 4, respectively. The rows show autocorrelations over different values of the following parameters, from top to bottom: hit reward, mean signal duration, mean noise duration, temporal discount factor, and attention cost exponent.
Figure 10: Comparison of optimal vs. cost-matched constant-attention policies. ( A ) The ratio, over c2 , of mean net reward per step for optimal policies to that of a constant-attention control with the same mean attention cost. A broken axis shows negative ratios and ratios above 10. ( B ) The inverse comparison: a histogram, over all solved policies, of the percentage of an optimal policy’s net reward captured by a constant-attention control with the same mean attention cost. For visual clarity, we show only comparisons where the control achieved non-negative mean reward.
Goal-conditioned systems assume goals are provided externally. We ask whether attentional priorities can emerge endogenously from an agent's internal cognitive state. We propose a priority function that generates observation targets from three epistemic gaps: ignorance (posterior variance), surprise (prediction error), and staleness (temporal decay of confidence in unobserved variables). We validate this in two systems: a minimal attention-allocation environment (2,000 runs) and a modular, partially observable world (500 runs). Ablation shows each component is necessary. A key finding is metric-dependent reversal: under global prediction error, coverage-based rotation wins; under change detection latency, priority-guided allocation wins, with advantage growing monotonically with dimensionality (d = -0.95 at N=48, p < 10^-6). Detection latency follows a power law in attention budget, with a steeper exponent for priority-guided allocation (0.55 vs. 0.40). When the decay rate is made learnable per variable, the system spontaneously recovers environmental volatility structure without supervision (t = 22.5, p < 10^-6). We demonstrate that epistemic gaps alone, without external reward, suffice to generate adaptive priorities that outperform fixed strategies and recover latent environmental structure.
Understanding how people allocate visual attention is central to Human-Computer Interaction (HCI), yet existing computational models of attention are often either descriptive, task-specific, or difficult to interpret. My dissertation develops a resource-rational, simulation-based framework for modeling visual attention as a sequential decision-making process under perceptual, memory, and time constraints. I formalize visual tasks, such as reading and multitasking, as bounded-optimal control problems using Partially Observable Markov Decision Processes, enabling eye-movement behaviors such as fixation and attention switching to emerge from rational adaptation rather than being hand-coded or purely data-driven. These models are instantiated in simulation environments spanning traditional text reading and reading-while-walking with smart glasses, where they reproduce classic empirical effects, explain observed trade-offs between comprehension and safety, and generate novel predictions under time pressure and interface variation. Collectively, this work contributes a unified computational account of visual attention, offering new tools for theory-driven and resource-efficient HCI design.
Biological systems must regulate competing needs under limited perceptual bandwidth, where sharpening one estimate costs the capacity to sharpen the others. Any fixed-budget system therefore has to decide where to allocate its perceptual precision. We study this in a foraging agent that must keep several bodily needs satisfied to survive, modelled with active inference. At each step it reads its own body-state beliefs, identifies the most-needed channel, and reallocates a fixed budget of interoceptive precision toward it, so that the same precision-shaped likelihood feeds both belief update and planning. In AffectWorld, a four-channel foraging gridworld, this selective allocation more than doubles learning-phase survival at matched budget against a uniform-precision agent (0.414 vs 0.199 across 11 layouts, n=32 seeds each, paired cluster-bootstrap p≤10−4). Two further results sharpen the mechanism. The benefit runs through planning as well as perception, since denying the shaped likelihood to the planner alone removes about half of it. It is also need-aligned, since aiming precision at the least-needed channel does worse than spreading it evenly. The attended channel additionally learns its own dynamics about twice as fast, and stays ahead even at matched observation count, a behavioural trace of the same precision routing, visible in learning speed, not survival.
St John Grimbly, Nicolas Kuske, Evert A. Boonstra +7
Dept. of Mathematics & Applied Mathematics, Univ. of Cape Town, South Africa · Neuroscience Institute, Univ. of Cape Town, South Africa · MIND Institute and CSAM, University of the Witwatersrand, South Africa +3