Organizations: Neuroscience Institute Carnegie Mellon University, Pittsburgh, PA 15213 · Neuroscience Baylor College of Medicine, Houston, TX 77030 · Neuroscience Institute and Department of Machine Learning Carnegie Mellon University, Pittsburgh, PA 15213
Paying attention improves performance, but attention is metabolically costly, so how should a resource-efficient agent allocate it? We study optimal allocation strategies using a normative model of a signal detection task in which attention comes at a cost. The model reveals that optimal attention is temporally structured in one of two patterns depending on task conditions: when attention costs penalize intense focus, optimal attention ramps up as evidence for the signal accumulates, but with less prohibitive attention costs, optimal attention fluctuates rhythmically. In cases with rhythmic attention, allocation frequency increases with task parameters such as reward magnitude for successful detection, signal brevity, and signal frequency. We argue that rhythmic attention emerges naturally in an agent whose belief updates are dominated by a stable temporal prior and not by the likelihood of new sensory observations. Our results characterize the conditions under which rhythmic vs. ramping attention is optimal and offer a normative account of attentional fluctuations observed in sustained attention tasks.
Figures & tables
Figure 1: Formal task structure. ( A ) In this POMDP, the agent constructs beliefs bt that reflect inferences about the latent world state st using observations ot controlled by the “attend” action at . The agent uses these beliefs to plan “commit” actions that may obtain rewards. ( B ) Specific world states and transitions in the finite state machine modeling this task. The noise state has a geometric duration. The signal lasts throughout a sequence of m signal states, with a tighter distribution the higher m is. Commit actions produce rewards when taken during the signal period and otherwise lead to a false alarm and penalty.
Figure 2: Attention policies over lower-dimension belief spaces. Attention over signal belief is shown in maroon; the background color shows if this signal belief is a level where the agent will not commit (light grey), will commit only if it observes signal (light blue), or will always commit (darker blue). ( A ) When we model a geometric distribution for signal duration using a single signal state ( m=1 ), belief is one-dimensional, so we may visualize full policies as a level of attention over total signal belief. We show a single-jump (left), 2-jump (middle) and many-jump (right) example here. ( B ) We can visualize a slice of policies with m>1 by fixing all but one signal state at 0. Here, we show an example of many-jumps policies where signal belief is concentrated in the final signal state for, from left to right, 1, 2, 3, and 4 signal states. We show and discuss more examples in the Appendix A.4 .
Figure 3: Illustration of agents’ interactions with simulated trials. Trial phase (noise: white, signal: black), observations (amber: no attention; white/black: noise/signal observation), belief, and commit action (green triangle). ( A ) agent with a rhythmic policy over a single trial of approximately 100 steps. ( B ) Same as A but with a ramping policy.
Figure 4: Rhythmicity and policy over attention cost exponent c2 . ( A ) Autocorrelation of attention allocations within a session for optimal policies under the attention cost function C(a)=0.2ac2 , for different c2 . Rhythmic policies show periodic spikes in autocorrelation; ramping policies are flat. ( B ) The corresponding policies producing the autocorrelations in A . The lines below show commit behavior (thin: never; medium: when observing signal; thick: always). ( C ) The rhythmicity metric plotted over c2 for 1, 2, 3, and 4 signal states. Rhythmicity drops sharply at approximately c2=1.7 : below this value, policies are rhythmic, and above it, policies ramp.
Figure 5: Comparison of optimal vs. constant-attention policies. ( A ) The ratio, over c2 , of mean net reward per step for optimal policies to that of a constant-attention control with the same mean attention. To keep the rest of the plot to scale, we show points where the optimal policy is ten times better, or where the constant policy has a negative net reward, on a grey bar above. ( B ) The inverse comparison: a histogram, over all solved policies, of how much of an optimal policy’s net reward a constant policy with the same mean attention captures. For visual clarity, we show only comparisons where the constant policy achieved non-negative mean reward.
Figure 6: Autocorrelations and policies over model parameters. All models shown here used a single signal state, allowing us to fully visualize their policies. Autocorrelations for models with more signal states are included in Appendix A.5 . ( A ) Autocorrelation of attention allocations within a session for optimal policies under different hit reward values. ( B ) The corresponding policies producing the autocorrelations in A . ( C ) and ( D ): the same for mean signal duration. ( E ) and ( F ): the same for mean noise duration.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: We may also visualize policies over two dimensions, allowing a look at how attention should be allocated over belief distributed in two signal states. For m=2 , this covers the full policy space. For m>2 , this enables visualizations of the policy over belief in two of the signal states while the others are fixed at 0. The same “jumps” in optimal attention can be observed here in the diagonal bands.
Figure 8: Visualization of policies and policy slices, as in Fig. 2 , except that signal belief in the slices is ( A ) concentrated in the first signal state or ( B ) distributed uniformly across all modeled signal states.
Figure 9: Autocorrelations over model parameters. The columns show models with m=1 , 2, 3, and 4, respectively. The rows show autocorrelations over different values of the following parameters, from top to bottom: hit reward, mean signal duration, mean noise duration, temporal discount factor, and attention cost exponent.
Figure 10: Comparison of optimal vs. cost-matched constant-attention policies. ( A ) The ratio, over c2 , of mean net reward per step for optimal policies to that of a constant-attention control with the same mean attention cost. A broken axis shows negative ratios and ratios above 10. ( B ) The inverse comparison: a histogram, over all solved policies, of the percentage of an optimal policy’s net reward captured by a constant-attention control with the same mean attention cost. For visual clarity, we show only comparisons where the control achieved non-negative mean reward.
Dept. of Mathematics & Applied Mathematics, Univ. of Cape Town, South Africa · Neuroscience Institute, Univ. of Cape Town, South Africa · MIND Institute and CSAM, University of the Witwatersrand, South Africa +3