Bellman Equation

Follow this topic Ask about this topic

Momentum

16 papers in the last four weeks, against 1 the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 81

All topics

Cost-augmented Schrödinger bridges on graphs are exactly solvable: a Feynman-Kac tilt replaces learned control
Oct 1, 2026Akshay BalsubramaniSchrödinger Bridges Missing Bridge
Robust and Learned Online Matching in Growing Trees
Sep 30, 2026Marek Gałązka, Hanna WdowickaDistribution Matching Bellman Equation
Faculty of Mathematics and Computer Science, Adam Mickiewicz University, Poznań, Poland · Department of Statistics, Poznań University of Economics and Business, Poznań, Poland
Fast Regularized Policy Mirror Descent with One-Step TD Updates
Sep 30, 2026Qipei Chen, Wenye Li, Yule Sun +1Mirror Descent Temporal Difference
School of Mathematical Sciences, Fudan University · School of Data Science, Fudan University
A First Glance at Jev for Network Traffic Classification: Accuracy, Processing Time, and Cost
Sep 30, 2026Shenghe Xu, Lifan MeiModel Evaluation Cloud
Amazon.com, Inc. · Xi’an Jiaotong-Liverpool University
Cross-Rollout Bellman Closure for Long-Horizon Agentic Reinforcement Learning
Sep 28, 2026Yangyang Ren, Haodong Zhu, Linlin Yang +3Group-Based Reinforcement Learning Bellman Equation
Beihang University · Zhongguancun Academy · Communication University of China +2
Deep Weighted Bellman Residual Minimization for $Q^*$ Estimation
Sep 28, 2026Lican Kang, Jerry Zhijian Yang, Cheng Yuan +1Offline Reinforcement Learning Bellman Equation
Institute for Math and AI, Wuhan University, Wuhan, 430072, China · Hubei Key Laboratory of Computational Science, Wuhan University, Wuhan, 430072, China · School of Artificial Intelligence, Wuhan University, Wuhan, 430072, China +1
Vector Bellman Theory for Multichain Robust Average-Reward Markov Decision Processes
Sep 23, 2026Yue Wang, George AtiaMarkov Decision Processes Bellman Equation
Department of Electrical and Computer Engineering University of Central Florida
Lifted Bellman Linear Programming for Offline Reinforcement Learning
Sep 21, 2026Hyukjun Yang, Jongchan Park, Narim Jeong +1Model-Based Reinforcement Learning Offline Reinforcement Learning
Korea Advanced Institute of Science and Technology(KAIST) · NAVER LABS
A Convergence Framework for Deep $V$ -Learning: Error Propagation and Sharp Action-Gap Bounds
Sep 16, 2026Yury KolomeytsevValue Functions Optimal Learner
Faculty of Computational Mathematics and Cybernetics Lomonosov Moscow State University
Symmetric solution of the Bellman optimality equation for repeated harmony game
Sep 14, 2026Hisato KomatsuBellman Equation Symmetry
Department of Physics, Kindai University, 577-8502, Higashi-Osaka, Osaka, Japan · Data Science and AI Innovation Research Promotion Center, Shiga University, 522-8522, Hikone, Shiga, Japan
Bellman Policy Optimization
Sep 14, 2026Zhuoqing Song, Haotian Xu, Xikun Zhang +1Reinforcement Learning With Verifiable Reward Frictive Policy Optimization
1Apodex US, Inc. · 2Princeton University
Adaptive Agent Design
Sep 14, 2026Raj Kiriti Velicheti, Subhonmesh Bose, Tamer BaşarQ-Learning Markov Decision Processes
Department of Electrical and Computer Engineering and the Coordinated Science Laboratory at the University of Illinois, Urbana-Champaign, Urbana, IL 61801
A Bellman Optimality Equation for Plasticity
Sep 11, 2026Jeremy Lucas, Doina PrecupPlasticity Bellman Equation
Mila – Quebec Artificial Intelligence Institute Montreal, Quebec, Canada
Support Discovery With Iteratively Reweighted Least Squares for Fixed-Charge Network Flow
Sep 10, 2026Sindura Saraswathi, Christian KümmerleBringing Network Coding Optimal Power Flow
University of Central Florida
Optimal Value Inference for Reinforcement Learning
Sep 9, 2026Nan Lu, Ethan Lee, James M. Robins +2Value Functions Bellman Equation
BRACE: Anchored Bellman-Residual Correction for Stale Critics in Asynchronous RL
Sep 9, 2026Guanqun Zhao, Zijun Xie, Binbin Zheng +3Gradient Staleness Large Language Model Training
Beijing University of Posts and Telecommunications · Baidu Inc. · Peking University +1
Online Estimation of Dynamic Origin-Destination Matrices Using Reinforcement Learning with Link-Flow Propagation Guidance
Aug 31, 2026Donggyu Min, Dong-Kyu KimOrigin Destination Flows Flow Policies
Seoul National University · Seoul, 08826, South Korea
CODS: Iterative Bellman-Residual Data Selection for Reusable Offline Reinforcement Learning
Aug 7, 2026Ibne Farabi Shihab, Sanjeda Akter, Abu Sa-Adat Mohamed Moon-Im Al Ahsan +2Model-Based Reinforcement Learning Offline Reinforcement Learning
Department of Computer Science, Iowa State University · Department of Computer Science & Engineering, BRAC University · Department of Civil, Construction & Environmental Engineering, Iowa State University
Adaptive Finite-Budget Training for CVaR Risk-Aware Q-Learning
Aug 5, 2026Yifan Wu, Junjie Lei, Wenjie HuangConditional-Value-At-Risk Q-Learning
Department of Data and Systems Engineering The University of Hong Kong Hong Kong, Hong Kong SAR
Revisiting TD Target Aggregation under Uncertainty in Q-Learning
Aug 4, 2026Lipeng Zu, Xiaonan ZhangDeep Q-Networks Q-Learning
Computer Science Department, Florida State University, Tallahassee, Florida, USA
Search as Computation Allocation
Jul 30, 2026Alexander TuisovComputational Cost Bellman Equation
Technion - Israel Institute of Technology
Micro-Segmentation Anomaly Detection in Zero-Trust Software-Defined Network Fabrics
Jul 24, 2026Ashly JosephIntrusion Detection Optical Networks
Reliability-Contagion Feasibility in LLM Multi-Agent Networks
Jul 24, 2026Ruiwu Niu, Xincheng Shu, Ying ZhaoMulti-Agent Communication Large Language Model Reliability
Department of Applied Data Science, Hong Kong Shue Yan University, Hong Kong SAR, China · Computational Communication Research Center, Beijing Normal University, Zhuhai 519087, China · School of Journalism and Communication, Beijing Normal University, Beijing 100875, China +1
Relative Value Learning
Jul 23, 2026Marc Höftmann, Jan Robine, Stefan HarmelingValue Functions Bellman Equation
Department of Computer Science, Technical University of Dortmund, Germany · Lamarr Institute for Machine Learning and Artificial Intelligence
Long-Term Sequential Decision Making under Risk
Jul 22, 2026Irmaan, Mirzanejad, Nadjet Bourdache +1Target Risk Sequential Decision Making
Mohammad · Université de Caen Normandie, ENSICAEN, CNRS, Normandie Univ, GREYC UMR 6072, France
Generalised Bellman recurrence and three dualities in sequential decision-making
Jul 20, 2026Fernando E. Rosas, David Hyland, Daniel PolaniBellman Equation Value Functions
Department of Informatics, University of Sussex · Department of Computer Science, University of Oxford · Department of Computer Science, University of Hertfordshire
Distributional Soft Bellman Operator under the Cramér Geometry
Jul 20, 2026Keru Wang, Yixin Deng, Yao Lyu +2Distributional Reinforcement Learning Bellman Equation
School of Electrical and Electronic Engineering, University College Dublin, Dublin, Ireland · School of Vehicle and Mobility, Tsinghua University, Beijing, China · College of Artificial Intelligence, Tsinghua University, Beijing, China
Rationalizing Boltzmann Rationality: An Axiomatic Characterization of Entropy-Regularized Policies
Jul 19, 2026Silviu PitisEntropy Regularized Reinforcement Learning Bellman Equation
University of Michigan
Statistical Efficiency and Inference of Quantile Distributional Reinforcement Learning
Jul 9, 2026Zijie Cheng, Yang Peng, Zhihua ZhangDistributional Reinforcement Learning Distributions
School of Mathematical Sciences, Peking University · Yau Mathematical Sciences Center, Tsinghua University
Fitted Occupancy-Ratio Evaluation without Bellman Completeness
Jul 6, 2026Lars van der Laan, Nathan KallusBellman Equation Occupancy
Stanford University · Netflix · Cornell University
Tight Lower Bounds for the Multi-Secretary Problem via Bellman Certificates
Jul 2, 2026Jiawei Zhang\Widetilde{\Mathcal{O}}(\Sqrt{T})$ Regret Lower Bounds
Department of Technologies, Operations, and Statistics Stern School of Business, New York University
Learning the Supports for Categorical Critic in Reinforcement Learning
Jul 2, 2026Jen-Yen Chang, Takayuki Osa, Tatsuya HaradaSoft Actor-Critic Value Functions
Graduate School of Information Science and Technology, The University of Tokyo · RIKEN AIP
PedNStream: Scalable Network Flow Simulation for Pedestrian Traffic Management
Jul 1, 2026Weiming Mai, Dorine Duives, Serge HoogendoornMicroscopic Traffic Simulations Flow Models
Department of Transportation and Planning, Delft University of Technology, Delft, Netherlands
Regularized Reward-Punishment Reinforcement Learning
Jun 26, 2026Jiexin Wang, Eiji UchibeDeep Q-Networks Bellman Equation
Dept. of Brain Robot Interface, ATR Computational Neuroscience Laboratories, 2-2-2, Hikaridai, Seikacho, Soraku-gun, Kyoto, 619-0288, Japan
Mean-Field PhiBE: Continuous-Time Mean-Field Reinforcement Learning from Discrete-Time Data
Jun 25, 2026Erhan Bayraktar, Martin Hernandez, Qinxin Yan +1Stochastic Optimal Control Model-Free
Department of Mathematics, University of Michigan, Ann Arbor, MI, USA. · Department of Statistics and Data Science, University of California, Los Angeles, CA, USA. · Program in Applied and Computational Mathematics, Princeton University, Princeton, NJ, USA.
Deterministic Pareto-Optimal Policy Synthesis for Multi-Objective Reinforcement Learning
Jun 24, 2026Aniruddha Joshi, Niklas Lauffer, Sanjit SeshiaMulti-Objective Reinforcement Learning Markov Decision Processes
University of California Berkeley, USA.
A Transport-Based Geometry of Belief-Cost
Jun 19, 2026Laurent CaraffaBellman Equation
Université Gustave Eiffel, LASTIG, IGN-ENSG
Inverting the Bellman Equation: From $Q$ -Values to World Models
Jun 19, 2026Alistair Letcher, Mattie Fellows, Alexander D. Goldie +3Model-Free Reinforcement Learning Goal-Conditioned Reinforcement Learning
FLAIR, University of Oxford · Google DeepMind · Mila, University of Montreal
Robust $Q$ -learning for mean-field control under Wasserstein uncertainty in common noise
Jun 18, 2026Mathieu Laurière, Ariel Neufeld, Kyunghyun ParkStochastic Optimal Control Q-Learning
NYU Shanghai HPC · MOE AcRF Tier 1 Grant RG109/25 · National Research Foundation of Korea
A Diffusion Approximation for Temporal-Difference Learning with Linear Features under Markovian Noise
Jun 16, 2026M. Forzo, E. Monzio Compagnoni, A. Russo +1Temporal Difference Stochastic Differential Equations
Technical University of Munich (TUM), Munich, Germany · University of Basel, Basel, Switzerland · Boston University, Boston, USA.
Deep Q-Learning on Hölder Spaces
Jun 15, 2026Qian QiQ-Learning Deep Q-Networks
Peking University
Maximum Entropy Inverse Reinforcement Learning for Mean-Field Games with Average Reward
Jun 15, 2026Şevket Kaan Alkır, Naci Saldı, Berkay Anahtarcı +1Mean Field Games Entropy Regularized Reinforcement Learning
Department of Mathematics Bilkent University 06800 Ankara, Turkey · Department of Mathematical Engineering Özyeğin University 34794 Istanbul, Turkey
Bellman-sufficient Information Complexity
Jun 9, 2026Yunbei XuInformation-Theoretic Limits Bellman Equation
National University of Singapore
Bellman-Taylor Score Decoding for Markov Decision Processes with State-Dependent Feasible Action Sets
Jun 9, 2026Yi Chen, Rushuai Yang, Qiang Chen +2Markov Decision Processes Bellman Equation
Lucy · Department of Industrial Engineering and Decision Analytics Hong Kong University of Science and Technology
FLOWREADER: Min-Cost Flow Optimization for Multi-Modal Long Document Q&A
Jun 5, 2026Ambuj Mehrish, Sebastiano VasconMultimodal Documents Top-K
Ca’ Foscari University of Venice
Prospect-Theory Behavior from Bellman Optimality in MDPs with Catastrophic States
May 31, 2026Yujiao ChenConditional-Value-At-Risk Markov Decision Processes
Massachusetts Institute of Technology · Cambridge, MA 02139, USA
Multivariate Distributional Reinforcement Learning Using Sliced Divergences
May 29, 2026Baptiste Debes, Tinne TuytelaarsDistributional Reinforcement Learning Bellman Equation
Department of Electrical Engineering (ESAT), KU Leuven, Leuven, Belgium.
On Distributional Reinforcement Learning in Chaotic Dynamical Systems
May 28, 2026James Rudd-Jones, Mirco Musolesi, María Pérez-OrtizDistributional Reinforcement Learning Chaos
Centre for Artificial Intelligence Department of Computer Science University College London London, UK · Department of Computer Science and Engineering, University of Bologna Bologna, Italy
Reward Transfer from Inverse Reinforcement Learning: A Coupled Minimax Approach
May 27, 2026Guang-Yuan Hao, Lars van der Laan, Aurélien Bibaut +1Offline Reinforcement Learning Value Functions
Cornell Tech, Cornell University · Netflix Research · Department of Statistics, University of Washington
A Contractive Feedback Semantics for Reinforcement Learning
May 23, 2026Zuyuan ZhangBellman Equation Feedback
The George Washington University
Target-Aligned Bellman Backup for Cross-domain Offline Reinforcement Learning
May 21, 2026Wei Liu, Ting LongModel-Based Reinforcement Learning Offline Reinforcement Learning
School of Artificial Intelligence Jilin University China
Beyond the Bellman Recursion: A Pontryagin-Guided Framework for Non-Exponential Discounting
May 20, 2026Hojin Ko, Jeonggyu HuhFrictive Policy Optimization Bellman Equation
Department of Mathematics, Sungkyunkwan University, Suwon, Republic of Korea
On Gaussian approximation for entropy-regularized Q-learning with function approximation
May 17, 2026Artemy Rubtsov, Rahul Singh, Eric Moulines +2Entropy Regularized Reinforcement Learning Q-Learning
HSE University, Moscow, Russia · Mohamed Bin Zayed University of AI, UAE · EPITA, France +1
BAPR: Bayesian amnesic piecewise-robust reinforcement learning for non-stationary continuous control
May 15, 2026Yifan Zhang, Liang ZhengOffline Reinforcement Learning Bellman Equation
Central South University
Adaptive Multi-Round Allocation with Stochastic Arrivals
May 12, 2026Yuqi Pan, Davin Choo, Haichuan Wang +3Queue Control Bellman Equation
Harvard University · University of Witwatersrand · Wits Health Consortium +1
Stochastic Minimum-Cost Reach-Avoid Reinforcement Learning
May 12, 2026Jingduo Pan, Taoran Wu, Yiling Xue +1Offline Reinforcement Learning Bellman Equation
Key Laboratory of System Software (Chinese Academy of Sciences), Institute of Software, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences, Beijing, China
Offline Policy Evaluation for Manipulation Policies via Discounted Liveness Formulation
May 12, 2026Hao Wang, Joshua Bowden, Colton Crosby +1Robot Policies Policy Evaluation
University of Southern California · Stanford University
Natural Policy Gradient as Doubly Smoothed Policy Iteration: A Bellman-Operator Framework
May 11, 2026Phalguni Nanda, Zaiwei ChenPolicy Gradient Boltzmann Policies
Edwardson School of Industrial Engineering, Purdue University
Quantile-Coupled Flow Matching for Distributional Reinforcement Learning
May 8, 2026Michael Groom, Victor-Alexandru Darvariu, Lars Kunze +2Distributional Reinforcement Learning Conditional Flow Matching
Oxford Robotics Institute, University of Oxford, Oxford, United Kingdom · Bristol Robotics Laboratory, University of the West of England, Bristol, United Kingdom · Dyson Institute of Engineering & Technology, Malmesbury, United Kingdom
Central Limit Theorem for Two-Time-Scale Approximate Distributionally Robust RL
May 8, 2026Shengbo Wang, Zexi ZhangDistributional Reinforcement Learning Large Deviation Principle
Daniel J. Epstein Department of Industrial and Systems Engineering, University of Southern California

Oct 1, 2026, cs.LGJ/K move · Enter open · S save

Cost-augmented Schrödinger bridges on graphs are exactly solvable: a Feynman-Kac tilt replaces learned control

Akshay Balsubramani

Schrödinger Bridges Missing Bridge Bellman Equation Free Energy Principle Markov

Open paper AskPDF arXiv

The generalized Schrödinger bridge on a graph moves mass between two distributions while charging a cost for the states visited. It has been approached by learning the rates of a controlled continuous-time Markov chain, with a temporal-difference penalty that restores the cost. A state cost folds into the reference process as a Feynman-Kac tilt. The cost-augmented bridge is then a plain bridge against the tilted reference, and the penalty is unnecessary. The bridge is computed exactly by alternating two endpoint rescalings, each one sparse matrix-exponential application; nothing is discretized in time or learned. The alternation converges at a rate set by the endpoint coupling alone. For a quadratic congestion cost on time-averaged occupancies, damped best response around the exact bridge is gradient descent on a strongly convex function, and its residual bounds its error. On a protein-folding model, a free-energy cost lowers the expected barrier of the folding paths. On the learned approach's road network, roll-outs of the exact bridge match the target within sampling error, and on networks with millions of intersections its memory grows linearly.

Momentum

Related topics

Latest papers 81