State Abstraction

Follow this topic Ask about this topic

State abstraction groups detailed states into simpler representations so agents and models can plan, learn, or act with less memory and data. Approaches include Markov and bisimulation quotients, decision- and task-conditioned representations, causal or topology-aware grouping, learned clustering, and hierarchical abstractions; they are tested in settings such as tabular MDPs, mazes, offline goal-conditioned learning, delayed control, and robotic planning. The central challenge is choosing what to preserve: abstractions that retain full predictive detail may be unnecessarily large, while aggressive compression can hide distinctions that matter for rewards, transitions, or transfer. Current work combines guarantees on value or plan preservation with adaptive and data-driven methods, but reliably balancing compression, decision quality, and generalization remains open.

Latest papers

27 papers0 papers in the last four weeks, down 100% on the four weeks before. 0.0% of all new papers.

Prediction Limits and Koopman Closure of Geometry-Induced Soft State Abstractions
Sep 26, 2026Mohit Kumar, Somayeh KargaranKoopman Operator Learning State Abstraction
University of Rostock, Rostock, Germany · Software Competence Center Hagenberg, Hagenberg, Austria
Certified Task-Conditioned Active Observability
Sep 22, 2026Linzhe Zhang, Changming XuActive Sensing State Abstraction
Graduate School, Northeastern University.
What Can a Recurrent State Safely Forget?
Sep 20, 2026Linzhe Zhang, Changming XuRecurrent Neural Networks Certified Robustness
Graduate School, Northeastern University.
An Information-Space Perspective to Scene Graph Sufficiency for Robotic Task Planning
Sep 14, 2026Başak Sakçak, Francesco VerdojaState Abstraction Robot Task Planning
Dept. of Advanced Computing Sciences, Maastricht University, the Netherlands · School of Electrical Engineering, Aalto University, Finland
Windowed A-K-MDP
Sep 12, 2026Xiangwen Yang, Frankie Cho, Iadine ChadesMarkov Decision Processes Markov Models
Monash University Melbourne, Australia
Pooling and Drift in Delayed Bandits
Sep 1, 2026Melika BaghiMulti-Armed Bandits Non-Stationary Bandits
Georgia Institute of Technology
A Data-Driven Approach to State Construction in Markov Models
Aug 21, 2026Linde Van Gestel, Marie-Anne Guerry, Evy RombautMarkov Models State Abstraction
Department of Business Technology and Operations, Data Analytics Laboratory, Vrije Universiteit Brussel, Pleinlaan 2, 1050 Brussels, Belgium · Mobility and Automotive Technology, Vrije Universiteit Brussel, Pleinlaan 2, 1050 Brussels, Belgium
$S^3$ : A Smooth Simulation Surrogate for Optimizing Discrete Abstractions of Dynamical Systems
Aug 16, 2026Jordan Peper, James Mathias Gast, Vignesh Nanduri +3State Abstraction Surrogate Modeling
University of Florida
RoMeRL: Balancing Feedback Coverage and the Memory-Reward Trap in Self-Evolving Agent Memory via Reduced-Order Utility States
Aug 3, 2026Yi Yang, Zhennan Chen, Yihong Zhuang +5State Abstraction State Representation Learning
1Nanjing University · 2Xiamen University · 3Zhejiang University
Tycho: Active Abstraction with Programmatic World Models for ARC-AGI-3
Jul 30, 2026Jens Lehmann, Andrei Aioanei, Sahar VahdatiGame-Playing Agents AI Coding Agents
Dresden University of Technology · Amazon · TIB – Leibniz Information Centre for Science and Technology +1
Minimal Markovization via Stable Quotients in Holonomy-Cover Decision Processes
Jul 29, 2026Zuyuan Zhang, Yongshan Chen, Mahdi Imani +1Markov Decision Processes Partially Observable RL
Department of Electrical and Computer Engineering, The George Washington University · Department of Electrical and Computer Engineering, Northeastern University
Property-driven Causal Abstractions for Markov Decision Processes
Jul 29, 2026Jule Schmidt, Maximilian Weininger, Clemens Dubslaff +2Markov Decision Processes Markov Models
Ruhr-University Bochum, Germany · Eindhoven University of Technology, The Netherlands · Oxford University, UK +1
A Behavioral State Vocabulary in Sony ERS-111 R-CODE
Jul 13, 2026Christopher A. TuckerRobotic Control State Abstraction
Dr. Christopher A. Tucker
Compositional Behavioral Semantics for State Abstraction in Reinforcement Learning
Jun 24, 2026Yivan Zhang, Ziyan Luo, Manuel BaltieriReinforcement Learning State Abstraction
The University of Tokyo · RIKEN AIP · Mila - Quebec Artificial Intelligence Institute +3
Performance-Driven Environment Abstraction with Multi-Timescale Learning
Jun 16, 2026Yue Guan, Dipankar Maity, Panagiotis TsiotrasMarkov Decision Processes Reinforcement Learning
School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA, USA · Department of Electrical and Computer Engineering, University of North Carolina at Charlotte, Charlotte, NC, USA
Adaptive state-action abstractions via rate-distortion
Jun 4, 2026Fernando E. RosasState Abstraction State Representation Learning
Department of Informatics, University of Sussex · Department of Brain Science, Imperial College London · Centre for Eudaimonia and Human Flourishing, University of Oxford
Topology-Aware State Abstraction with Tangle Cores for Markov Decision Processes
May 29, 2026Ibne Farabi Shihab, Sanjeda Akter, Anuj SharmaMarkov Decision Processes State Abstraction
Department of Computer Science, Iowa State University · Department of Civil, Construction & Environmental Engineering, Iowa State University
Answer-Set-Programming-based Abstractions for Reinforcement Learning
May 29, 2026Rafael Bankosegger, Thomas Eiter, Johannes OetschReinforcement Learning State Abstraction
Siemens AG Österreich, Vienna, Austria · TU Wien (Vienna University of Technology), Austria · Jönköping University, Sweden
Abstraction for Offline Goal-Conditioned Reinforcement Learning
May 21, 2026Clarisse Wibault, Alexander Goldie, Antonio Villares +2Hierarchical RL State Abstraction
FLAIR, MLRG · University of Oxford · FLAIR +1
Markovian Circuit Tracing for Transformer State Dynamic
May 20, 2026Abdullah XTransformer Interpretability Transformer
Project AWARE and Zephara AI
Smaller Abstract State Spaces Enable Cross-Scale Generalization in Reinforcement Learning
May 19, 2026Nasehatul Mustakim, Lucas LehnertPartially Observable RL OOD Generalization
Department of Computer Science University of Saskatchewan Saskatoon, Saskatchewan, Canada
Abstract Sim2Real through Approximate Information States
Apr 16, 2026Yunfu Deng, Yuhao Li, Josiah P. HannaSim-to-Real Transfer State Abstraction
Department of Computer Sciences, University of Wisconsin–Madison, Madison, WI 53706 USA · Manning College of Information and Computer Sciences, University of Massachusetts Amherst, Amherst, MA 01003 USA
Delayed homomorphic reinforcement learning for environments with delayed feedback
Apr 4, 2026Jongsoo Lee, Jangwon Kim, Soohee HanDelayed RL Reinforcement Learning
Department of Convergence IT Engineering, POSTECH, Pohang, 37673, South Korea · Department of Electrical Engineering, POSTECH, Pohang, 37673, South Korea
Multi-level meta-reinforcement learning with skill-based curriculum
Mar 9, 2026Sichen Yang, Mauro MaggioniReinforcement Learning Hierarchical RL
Johns Hopkins University · Department of Applied Mathematics & Statistics Johns Hopkins University, 3400 N. Charles Street, Baltimore, MD 21218, USA · Department of Mathematics, Department of Applied Mathematics & Statistics Johns Hopkins University, 3400 N. Charles Street, Baltimore, MD 21218, USA
Verification of Unknown Dynamical Systems via Autoencoder Latent Space
Dec 15, 2025Robert Reed, Luca Laurenti, Morteza LahijanianFormal Verification State Abstraction
Dept. of Aerospace Engineering University of Colorado Boulder Boulder, Colorado, USA · Delft University of Technology Delft, Netherlands · AI4I, Turin, Italy
Poincaré Meets Bellman: Revisable Memory, Operational Quotients, and Evidence-Supported Learning in Changing Environments
Nov 28, 2025Xin LiMarkov Decision Processes Dynamic Programming
Department of Computer Science University at Albany
Decision-Centered Abstractions via Orthogonal Estimation of Difference-of-Q Functions
Jun 12, 2024Defu Cao, Angela ZhouValue Function Estimation State Abstraction
University of Southern California

Sep 26, 2026J/K move · Enter open · S save

Prediction Limits and Koopman Closure of Geometry-Induced Soft State Abstractions

Mohit Kumar, Somayeh Kargaran

University of Rostock, Rostock, Germany · Software Competence Center Hagenberg, Hagenberg, Austria

Koopman Operator Learning State Abstraction State Representation Learning

Open paper AskPDF arXiv

Soft state representations assign states nonnegative class weights that sum to one, but agreement with reference labels does not guarantee accurate prediction of the next weight vector. We derive lower confidence bounds on the minimum population root-mean-square prediction error over linear predictors with a prescribed spectral-norm limit. The bounds compare within-class successor variation with retained within-class input variation, with an alternative based on reference-label deviation. They are estimated from independent transition pairs without fitting a prediction matrix to the evaluation data. A bound above a chosen error tolerance excludes that accuracy for every admissible predictor at the stated confidence level; a bound of zero does not establish whether an accurate predictor exists. For coordinates constructed from Kernel Affine Hull Machine reconstruction scores, we relate the reference class's affinity advantage over competitors to label agreement. The class-centered population bound equals the optimum in a four-state example. Using 117,000 previously generated evaluation datasets from fixed four-state designs, we compare the finite-sample certificates with an empirical-risk-minimization baseline. For one Van der Pol representation selected before evaluation, the label-based bound is positive in all 64 independent evaluation datasets at the fitted matrix's spectral norm. Separately, exact matrix propagation under deterministic dynamics yields a Koopman interpretation of the coordinate-function span. The certificates help identify forecasting targets that cannot be reached by refitting a fixed representation within a specified predictor class and evaluation distribution.