cs.LGNov 19, 2026

Grid Cells as Spectral Successor Features: A Reusable Basis for Continual Reinforcement Learning

Authors: Élodie Côté-Gauthier, Doina Precup, Isabeau Prémont-Schwarz

Organizations: McGill University and Mila – Quebec AI Institute · Université Laval and Mila – Quebec AI Institute

Abstract

Continual reinforcement learning requires agents to adapt when the task changes while reusing what they know about the environment. Successor features separate dynamics from task-specific reward, but their effectiveness depends on the choice of base features. Taking the identification of entorhinal grid cells with eigenvectors of the successor representation as a design principle, we build this basis from eigenvectors of the transition matrix of a fixed reference policy. With such a basis, the successor features at a state-action pair are a linear combination of the base features at that same pair, which is not true in general, and the coefficients are known in closed form: each successor feature is its base feature multiplied by θi=1/(1−γλi)θ_i = 1/(1-γλ_i), where λiλ_i is its eigenvalue. The long-term return is thus predicted from the same features as the immediate reward, no successor-feature network or replay buffer is needed, and adapting to a new task reduces to a linear regression on the reward. The agent acts greedily on the action values of the reference policy, which is one step of policy improvement over it and suffices to reach a single goal. In a two-goal gridworld, with the uniform random walk as reference policy, whose eigenvectors are the low-frequency Laplacian eigenvectors, agents built this way maintain near-maximal return across every change of goal, whereas SimpleSF and DQN lose most of their return at each change.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Balancing Plasticity and Stability with Fast and Slow Successor Features

    May 25, 2026Raymond Chua, Doina Precup, Blake RichardsSequential Decision MakingNon-Stationary RL

  2. Robust Successor Features

    Sep 25, 2026Erik Nikulski, Yamen Habib, Vicenç Gomez +3Reinforcement LearningGeneralization in RL

  3. Beyond Single-Model Optimization: Preserving Plasticity in Continual Reinforcement Learning

    Apr 16, 2026Lute Lillo, Nick CheneyNon-Stationary RLReinforcement Learning