Optimal Control

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 96

All topics

Hidden States as Value Gradients: The Pontryagin Structure of Recurrent Policies
May 6, 2026David Leeftink, Max Hinne, Marcel van GervenRL Control Reinforcement Learning
Department of Machine Learning and Neural Computating Donders Institute for Brain, Cognition and Behaviour Radboud University, Nijmegen, the Netherlands.
Bilinear Mamba-Koopman Neural MPC for Varying Dynamics
May 6, 2026Matan Pagi, Zohar SorekDynamical Systems Koopman Operator Learning
Psistar AI
Right Model, Right Time: Real-Time Cascaded-Fidelity MPC for Bipedal Walking
May 6, 2026Franek Stark, Felix Wiebe, Shubham Vyas +2Nonlinear MPC Whole-Body Control
Robotics Innovation Center at the German Research Center for Artificial Intelligence (DFKI), Bremen, Germany · University Bremen, Germany
Dream-MPC: Gradient-Based Model Predictive Control with Latent Imagination
May 6, 2026Jonathan Spieler, Sven BehnkeModel Predictive Control Optimal Control
Autonomous Intelligent Systems, Computer Science Institute VI - Intelligent Systems and Robotics, Center for Robotics and the Lamarr Institute for Machine Learning and Artificial Intelligence, University of Bonn, Germany.
Globally Solving Unbalanced Optimal Transport and Density Control for Gaussian Distributions
May 5, 2026Haruto Nakashima, Siddhartha Ganguly, Kenji KashimaSemidefinite Programming Stochastic Optimal Control
The Applied Mathematics and Physics Department, Graduate School of Informatics, Kyoto University, Kyoto, Japan · Daniel Guggenheim School of Aerospace, Georgia Institute of Technology, Atlanta, USA
Adaptive Estimation and Optimal Control in Offline Contextual MDPs without Stationarity
May 5, 2026Riddhiman Bhattacharyya, Sayak Chakrabarty, Imon BanerjeeNon-Stationary RL Markov Decision Processes
Department of Statistics, University of California, Santa Cruz Santa Cruz, California, USA · Department of Computer Science Northwestern University Evanston, IL, USA · Department of Industrial Engineering and Management Sciences Northwestern University Evanston, IL, USA
Sampling-Based Control via Entropy-Regularized Optimal Transport
May 4, 2026Vincent Pacelli, Akash Ratheesh, Evangelos A. TheodorouRobotic Control Nonlinear MPC
Autonomous Control and Decision Systems Laboratory Georgia Institute of Technology
Stability of Control Lyapunov Function Guided Reinforcement Learning
May 3, 2026Zachary Olkin, William D. Compton, Aaron D. AmesRL Control Lyapunov Stability
The authors are with the Department of Control and Dynamical Systems at the California Institute of Technology.
Optimizing Trajectory-Trees in Belief Space: An Application from Model Predictive Control to Task and Motion Planning
May 3, 2026Camille Phiquepal, Marc ToussaintBelief-Space Planning Model Predictive Control
Learning and Intelligent Systems Lab (LIS), TU Berlin, Germany
Analytic Bridge Diffusions for Controlled Path Generation
May 3, 2026Michael ChertkovStochastic Optimal Control Schrödinger Bridges
Program in Applied Mathematics & Department of Mathematics University of Arizona, AZ 85721, USA
Continuous-time q-learning for mean-field control with common noise, part-II: q-learning algorithms
Apr 30, 2026Zhenjie Ren, Xiaoli Wei, Xiang Yu +1Reinforcement Learning Stochastic Optimal Control
LaMME, Universit´e ´Evry Paris-Saclay, ´Evry, France · Department of Applied Mathematics, The Hong Kong Polytechnic University, Kowloon, Hong Kong · Department of Industrial Engineering and Operations Research, Columbia University, New York, USA
Continuous-time q-learning for mean-field control with common noise, part-I: Theoretical foundations
Apr 30, 2026Zhenjie Ren, Xiaoli Wei, Xiang Yu +1Stochastic Optimal Control Q-Learning
LaMME, Universit´e ´Evry Paris-Saclay, ´Evry, France · Department of Applied Mathematics, The Hong Kong Polytechnic University, Kowloon, Hong Kong · Department of Industrial Engineering and Operations Research, Columbia University, New York, USA
Learning to Spend: Model Predictive Control for Budgeting under Non-Stationary Returns
Apr 29, 2026Nilavra Pathak, Smriti Shyamal, Prasant Mhasker +1Stochastic Optimal Control Model Predictive Control
Marketing Data Science, Expedia Group · Department of Chemical Engineering, McMaster University
Nonlinear Non-Gaussian Density Steering with Input and Noise Channel Mismatch: Sinkhorn with Memory for Solving the Control-affine Schrödinger Bridge Problem
Apr 25, 2026Georgiy A. Bondar, Asmaa Eldesoukey, Yongxin Chen +1Stochastic Optimal Control Schrödinger Bridges
Department of Aerospace Engineering, Iowa State University, Ames, IA 50011, USA · School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA
Rate-Optimal Regret for the Safe Learning-based Control of the Constrained Linear Quadratic Regulator
Apr 24, 2026Spencer Hutchinson, Nanfei Jiang, Mahnoosh AlizadehStochastic Linear Dynamical Systems Stochastic Optimal Control
Department of Electrical and Computer Engineering, University of California Santa Barbara
Model Predictive Control of Hybrid Dynamical Systems
Apr 23, 2026Ricardo G. Sanfelice, Berk AltinDynamical Systems Model Predictive Control
Department of Electrical and Computer Engineering, University of California, Santa Cruz, CA, 94107 USA
SLAM as a Stochastic Control Problem with Partial Information: Optimal Solutions and Rigorous Approximations
Apr 23, 2026Ilir Gusija, Fady Alajaji, Serdar YükselSimultaneous Localization and Mapping Stochastic Optimal Control
Department of Mathematics and Statistics, Queen’s University, Kingston, ON, Canada
Efficient Reinforcement Learning using Linear Koopman Dynamics for Nonlinear Robotic Systems
Apr 21, 2026Wenjian Hao, Yuxuan Fang, Zehui Lu +1Robotic Control Reinforcement Learning
School of Aeronautics and Astronautics, Purdue University, West Lafayette, IN 47907, USA · independent researcher
Accelerating trajectory optimization with Sobolev-trained diffusion policies
Apr 21, 2026Théotime Le Hellard, Franki Nguimatsia Tiofack, Quentin Le Lidec +1Trajectory Optimization Trajectory Generation
Inria - Département d’Informatique de l’École normale supérieure, PSL Research University, France · Courant Institute, New York University, USA
Learning the Riccati solution operator for time-varying LQR via Deep Operator Networks
Apr 20, 2026Jun Chen, Umberto Biccari, Junmin WangFeedback Control Deep Operator Networks
Model Predictive Path Integral PID Control for Learning-Based Path Following
Mar 31, 2026Teruki Kato, Koshi Oishi, Seigo ItoNonlinear MPC Residual Dynamics Learning
Toyota Central R&D Labs., Inc., Nagakute 480-1192, Japan
Separation is Optimal for LQR under Intermittent Feedback
Mar 29, 2026Abdullah Y. Etcibasi, C. Emre Koksal, Eylem EkiciSequential Decision Making Stochastic Optimal Control
The Ohio State University
Adjoint Matching through the Lens of the Stochastic Maximum Principle in Optimal Control
Mar 28, 2026Carles Domingo-Enrich, Jiequn HanStochastic Optimal Control Optimal Control
Microsoft Research New England · Flatiron Institute
Fundamental Limits for Sensor-Based Control via the Gibbs Variational Principle
Mar 19, 2026Vincent Pacelli, Evangelos A. TheodorouFeedback Control Partially Observable Markov Decision Processes
Daniel Guggenheim School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA, USA, 30308
State-space models through the lens of ensemble control
Mar 13, 2026Ye Feng, Jianfeng LuMirror Descent Optimal Control
Accelerating Sampling-Based Control via Learned Linear Koopman Dynamics
Mar 5, 2026Wenjian Hao, Yuxuan Fang, Zehui Lu +1Robotic Control Robot Trajectory Tracking
independent researcher
Soft Swing-up: Benchmarking Model-Based Optimal Control for Rigid-Soft Underactuated Systems
Feb 3, 2026Daniele Caradonna, Nikhil Nair, Anup Teejo Mathew +5Robotic Control Model Predictive Control
The BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy · Department of Excellence in Robotics and AI, Scuola Superiore Sant’Anna, Pisa, Italy · Department of Cognitive Robotics, Delft University of Technology, 2628 CN Delft, The Netherlands +3
Soft Fitted Q-Iteration without Bellman Completeness: Occupancy Reweighting and Temperature Annealing
Dec 30, 2025Lars van der Laan, Nathan KallusReinforcement Learning KL-Regularized RL
Department of Statistics, University of Washington · Netflix · Cornell University
Modeling and Control of Deep Sign-Definite Dynamics with Application to Hybrid Powertrain Control
Sep 24, 2025Teruki Kato, Ryotaro Shima, Kenji KashimaData-Driven Control Model Predictive Control
Toyota Central R&D Labs., Inc., 41-1 Yokomichi, Nagakute, Aichi, Japan. · Graduate School of Informatics, Kyoto University, Yoshida-honmachi, Sakyo-ku, Kyoto, Japan
Observed Control - Linearly Scalable Nonlinear Model Predictive Control with Adaptive Horizons
Aug 18, 2025Eugene T. Hamzezadeh, Andrew J. PetruskaNonlinear MPC Model Predictive Control
Colorado School of Mines, Golden, CO 80401, USA

May 6, 2026, cs.LGJ/K move · Enter open · S save

Hidden States as Value Gradients: The Pontryagin Structure of Recurrent Policies

David Leeftink, Max Hinne, Marcel van Gerven

Department of Machine Learning and Neural Computating Donders Institute for Brain, Cognition and Behaviour Radboud University, Nijmegen, the Netherlands.

RL Control Reinforcement Learning Partially Observable RL Policy Learning Optimal Control

Open paper AskPDF arXiv

A key capability of intelligent agents is to act effectively under incomplete state observations. Recurrent policies address this by compressing observation histories into a hidden state. In this work, we show that the hidden state of a recurrent policy admits a control-theoretic interpretation: it plays the role of the co-state in Pontryagin's minimum principle, and the readout that maps it to actions implements control-Hamiltonian minimization. The hidden state thus tracks the gradient of the value function, encoding the optimality structure of the underlying control problem. We formalize this correspondence through a class of policies we refer to as co-state policies (CPs) and show that several modern recurrent cells implicitly realize this structure. The correspondence also allows for a co-state loss for actor-critic training, in which the critic's gradient serves as a target for the actor's hidden state. Empirically, we find that hidden states trained with the co-state loss encode co-state information beyond what is linearly decodable from the environment state alone, and that the loss improves policies both within and outside this class on challenging locomotion tasks, including the H1 and Berkeley humanoids. By connecting the minimum principle to recurrent memory, we provide a control-theoretic account of what hidden states compute in continuous control and a mechanism for shaping them toward optimality.

Momentum

Related topics

Latest papers 96