cs.CLNov 12, 2026

Goal Encoding in LLMs

Authors: Arnab Sen Sharma, Eric Todd, Sheridan Feucht, Andy Arditi, Rohit Gandikota, Natalie Shapira, Gabriele Sarti, Constanza Fierro, 

Organizations: Northeastern University · Parallax · University of Copenhagen · University of Oxford

Abstract

We investigate whether and how LLMs encode goals: internal neural representations that precede and guide action, and that are flexible enough to be realized through many different execution paths. We construct a coding setting with multiple sub-tasks that allows us to evaluate goal-seeking behavior and the selection of sub-goals. In this setting, we localize an encoding of the next sub-goal in the outputs of a sparse set of attention heads. The same heads can be used to decode the LM's future actions in natural language (e.g., "I'm checking if a word is a palindrome"). We verify that this encoding is causal for next action and is plastic: transplanting it into a different environment, e.g., another programming language, triggers the same operation realized with a different sequence of tokens. The LM also sustains the encoding throughout the action's execution, which is puzzling because a transformer LM without recurrence cannot carry a state forward for more tokens than it has layers left. We propose that it does not need to. Under a non-recurrent goal-seeking (NRGS) account, the LM frequently re-reads the goal from the task specification, compares it with the progress visible in the growing context, and pursues what remains. We test the predictions of this account with targeted causal interventions.

Figures & tables

Appendix figures & tables47 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Language Model Goal Selection Differs from Humans' in a Self-Directed Learning Task

    Feb 6, 2026Gaia Molinaro, Dave August, Danielle Perszyk +1Human-AI Decision Making

  2. Zero-Shot Goal Recognition with Large Language Models

    May 14, 2026Kin Max Piamolini Gusmão, Nathan Gavenski, Nir Oren +1Zero-Shot LearningLLM Evaluation