Markov Decision Processes

Follow this field Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

16 new papers

A weekly snapshot of new work published in Markov Decision Processes.

Period ending 2026-09-14

11 new papers

A weekly snapshot of new work published in Markov Decision Processes.

Period ending 2026-09-07

5 new papers

A weekly snapshot of new work published in Markov Decision Processes.

Inside this field

Focused directions

Focused topic

Markov Decision Processes

Within Markov Decision Processes

174 papersExplore

Focused topic

Optimal Policies

Within Markov Decision Processes

105 papersExplore

Focused topic

Value Functions

Within Markov Decision Processes

71 papersExplore

Focused topic

Bellman Equation

Within Markov Decision Processes

67 papersExplore

Focused topic

Partially Observable Markov Decision Process

Within Markov Decision Processes

66 papersExplore

419 papers

Latest in Markov Decision Processes

Browse all topics →

Beyond the Bellman Recursion: A Pontryagin-Guided Framework for Non-Exponential Discounting
May 20, 2026Hojin Ko, Jeonggyu HuhMulti-Objective Policy Optimization Bellman Equation
Dynamic Shapley Computation
May 20, 2026Xuan Yang, Hsi-Wen Chen, Ming-Syan Chen +1Shapley Value Value Functions
ReversedQ: Opportunities for Faster Q-Learning in Episodic Online Reinforcement Learning
May 20, 2026Sofia R. Miskala-Dinc, Aviva PrinsQ-Learning Model-Free Reinforcement Learning
Stitched Value Model for Diffusion Alignment
May 19, 2026Hyojun Go, Hyungjin Chung, Prune Truong +8Generative Flow Networks Fabric
Minimax Optimal Variance-Aware Regret Bounds for Multinomial Logistic MDPs
May 19, 2026Pierre Boudart, Pierre Gaillard, Alessandro RudiNear-Optimal Regret Guarantees Markov Decision Processes
Smaller Abstract State Spaces Enable Cross-Scale Generalization in Reinforcement Learning
May 19, 2026Nasehatul Mustakim, Lucas LehnertState Space Partially Observable Markov Decision Process
Learning to Hand Off: Provably Convergent Workflow Learning under Interface Constraints
May 18, 2026Jiayu Li, Enpei Zhang, Dawei Zhou +2Q-Learning Multi-Agent Workflows
Learning Interpretable Point-Based Clinical Risk Scores via Direct Optimization
May 18, 2026Ying Cui, Albert M Li, Vivek Charu +3Quadratic Slack Penalty Machine Learning
On Gaussian approximation for entropy-regularized Q-learning with function approximation
May 17, 2026Artemy Rubtsov, Rahul Singh, Eric Moulines +2Q-Learning Central Limit Theorem
MATE: Solving Contextual Markov Decision Processes with Memory of Accumulated Transition Embeddings
May 17, 2026Himchan Hwang, Hyeokju Jeong, Gene Chung +3Markov Decision Processes Markov
Scalable Bi-causal Optimal Transport via KL Relaxation and Policy Gradients
May 17, 2026Haoyang Cao, Jesse Hoekstra, Renyuan Xu +2Optimal Transport Stochastic Optimization
Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP
May 15, 2026Igor Bogdanov, Chung-Horng Lung, Thomas Kunz +3Large Language Model Agents Partially Observable Markov Decision Process
BAPR: Bayesian amnesic piecewise-robust reinforcement learning for non-stationary continuous control
May 15, 2026Yifan Zhang, Liang ZhengBellman Equation Change-Point Detection
Sign-Separated Asymmetric Finite-Time Error Analysis of Q-Learning
May 15, 2026Donghwan LeeQ-Learning Finite Time Analysis
Tighter Regret Bounds for Contextual Action-Set Reinforcement Learning
May 15, 2026Zijun Chen, Zihan ZhangLinear Regret Value Functions
Harnessing Unimodality in Semiparametric Contextual Pricing via Oracle Price Map Learning
May 14, 2026Yingying Fan, Yuxuan Han, Jinchi Lv +2Prices Linear Contextual Bandits
Second-Order Actor-Critic Methods for Discounted MDPs via Policy Hessian Decomposition
May 14, 2026Sanjeev Manivannan, Shuban VSoft Actor-Critic Markov Decision Processes
Synthesizing POMDP Policies: Sampling Meets Model-checking via Learning
May 14, 2026Debraj Chakraborty, Anirban Majumdar, Prince Mathew +2Partially Observable Markov Decision Process Markov Decision Processes
Dynamic Latent Routing
May 14, 2026Fangyuan Yu, Xin Su, Amir AbdullahEfficient Latent Reasoning Markov Decision Processes
Heuristic Pathologies and Further Variance Reduction via Uncertainty Propagation in the AIVAT Family of Techniques
May 14, 2026Juho Kim, Tuomas SandholmVariance Reduction Value Functions
Action-Conditioned Risk Gating for Safety-Critical Control under Partial Observability
May 14, 2026Yushen Liu, Yin-Jen Chen, Ziyi Chen +4Belief Space Safety-Critical Scenarios
Learning POMDP World Models from Observations with Language-Model Priors
May 13, 2026Valentin Six, Frederik Panse, Mathis Fajeau +7Partially Observable Markov Decision Process Latent World Models
Tight Sample Complexity Bounds for Entropic Best Policy Identification
May 13, 2026Amer Essakine, Claire VernadeEntropy Regularized Reinforcement Learning Optimal Policies
Adaptive mine planning under geological uncertainty: A POMDP framework for sequential decision-making
May 13, 2026Hamza Khalifi, Jef Caers, Yassine Taha +2Partially Observable Markov Decision Process Markov Decision Processes
Achieving $ε^{-2}$ Sample Complexity for Single-Loop Actor-Critic under Minimal Assumptions
May 13, 2026Ishaq Hamza, Zaiwei ChenSoft Actor-Critic Optimal Policies
Differentiable Learning of Lifted Action Schemas for Classical Planning
May 13, 2026Jonas Reiter, Jakob Elias Gebler, Hector GeffnerClassical Planning Planning Methods
State-Centric Decision Process
May 12, 2026Sungheon Jeong, Ryozo Masukawa, Sanggeon Yun +2Markov Decision Processes Agent Decisions
What Do You Think I Think? Accounting for Human Beliefs Using Second-Order Theory of Mind
May 12, 2026Patrick Callaghan, Reid Simmons, Henny AdmoniTheory-Of-Mind Reasoning Belief
BSO: Safety Alignment Is Density Ratio Matching
May 12, 2026Tien-Phat Nguyen, Truong Nguyen, Thin Nguyen +3Safety Alignment Large Language Model Alignment
Ensuring Logic in the Fog: Sound POMDP Synthesis with LTL Objectives
May 12, 2026Can Zhou, Yulong Gao, Pian YuLinear Temporal Logics Partially Observable Markov Decision Process
Missingness-MDPs: Bridging the Theory of Missing Data and POMDPs
May 12, 2026Joshua Wendland, Markel Zubia, Roman Andriushchenko +6Partially Observable Markov Decision Process Missing Data
Optimal Policy Learning under Budget and Coverage Constraints
May 12, 2026Giovanni CerulliOptimal Policies Knapsack Constraint
On the Importance of Multistability for Horizon Generalization in Reinforcement Learning
May 12, 2026Asad Bakija, Florent De Geeter, Julien Brandoit +2Deep Reinforcement Learning Partially Observable Markov Decision Process
Adaptive Multi-Round Allocation with Stochastic Arrivals
May 12, 2026Yuqi Pan, Davin Choo, Haichuan Wang +3Stochastic Delays Wireless Resource Allocation
Split the Differences, Pool the Rest: Provably Efficient Multi-Objective Imitation
May 12, 2026Ziyad Sheebaelhamd, Luca Viano, Volkan Cevher +1Imitation Learning Behavior Cloning
Stochastic Minimum-Cost Reach-Avoid Reinforcement Learning
May 12, 2026Jingduo Pan, Taoran Wu, Yiling Xue +1Constrained Reinforcement Learning Bellman Equation
When Simulation Lies: A Sim-to-Real Benchmark and Domain-Randomized RL Recipe for Tool-Use Agents
May 12, 2026Xiaolin Zhou, Aojie Yuan, Zheng Luo +12Deep Reinforcement Learning Sim-To-Real Gap
Delightful Gradients Accelerate Corner Escape
May 12, 2026Jincheng Mei, Ian OsbandPolicy Gradient Optimal Policies
Federated Client Selection under Partial Visibility: A POMDP Approach with Spatio-Temporal Attention
May 12, 2026Qijun Hou, Yuchen Shi, Pingyi Fan +1Federated Learning Partially Observable Markov Decision Process
Augmented Lagrangian Method for Last-Iterate Convergence for Constrained MDPs
May 12, 2026Michael Lu, Max Qiushi Lin, Mo Chen +1Markov Decision Processes Exponential Horizon Dependence
Learning Weakly Communicating Average-Reward CMDPs: Strong Duality and Improved Regret
May 12, 2026Kihyun Yu, Beomhan Baek, Dabeen LeeMarkov Decision Processes Linear Regret
Primal-Dual Policy Optimization for Linear CMDPs with Adversarial Losses
May 12, 2026Kihyun Yu, Seoungbin Bae, Dabeen LeePrimal-Dual Methods Markov Decision Processes
Engagement Process: Rethinking the Temporal Interface of Action and Observation
May 12, 2026Jialian Li, Yuchen Cao, Junhong Liu +5Partially Observable Markov Decision Process Toward Interaction Dynamics
Offline Policy Evaluation for Manipulation Policies via Discounted Liveness Formulation
May 12, 2026Hao Wang, Joshua Bowden, Colton Crosby +1Robot Policies Exponential Horizon Dependence
Adaptive Policy Learning Under Unknown Network Interference
May 11, 2026Aidan Gleich, Eric Laber, Alexander VolfovskyThompson Sampling Treatment Effect
Revisiting Policy Gradients for Restricted Policy Classes: Escaping Myopic Local Optima with $k$ -step Policy Gradients
May 11, 2026Alex DeWeese, Guannan QuPolicy Gradient Optimal Policies
Shields to Guarantee Probabilistic Safety in MDPs
May 11, 2026Linus Heck, Filip Macák, Roman Andriushchenko +2Shielding Safety Constraints
Policy Gradient Methods for Non-Markovian Reinforcement Learning
May 11, 2026Avik Kar, Siddharth Chandak, Rahul Singh +4Markov Decision Processes Policy Gradient
Switching-Geometry Analysis of Deflated Q-Value Iteration
May 11, 2026Donghwan LeeQ-Learning Convergence Analysis
Natural Policy Gradient as Doubly Smoothed Policy Iteration: A Bellman-Operator Framework
May 11, 2026Phalguni Nanda, Zaiwei ChenPolicy Gradient Bellman Equation
Teacher-Aware Evolution of Heuristic Programs from Learned Optimization Policies
May 11, 2026Minyu Chen, Song Qin, Ling-I Wu +2Heuristics Evolutionary Search
Towards Model-Free Learning in Dynamic Population Games: An Application to Karma Economies
May 11, 2026Matteo Cederle, Saverio Bolognani, Gian Antonio SustoDeep Q-Networks Model-Free Reinforcement Learning
Attribution-based Explanations for Markov Decision Processes
May 10, 2026Paul Kobialka, Andrea Pferscher, Francesco Leofante +3Markov Decision Processes Artificial Intelligence Models
Minimizing Worst-Case Weighted Latency for Multi-Robot Persistent Monitoring: Theory and RL-Based Solutions
May 10, 2026Weizhen Wang, Ziheng Wang, Jianping He +2Multi-Robot Systems Constrained Reinforcement Learning
CLR-voyance: Reinforcing Open-Ended Reasoning for Inpatient Clinical Decision Support with Outcome-Aware Rubrics
May 10, 2026Aishik Nagar, Arun-Kumar Kaliya-Perumal, Yu-Hsuan Han +5Clinical Reasoning Rubrics
ACSAC: Adaptive Chunk Size Actor-Critic with Causal Transformer Q-Network
May 10, 2026Qian Chen, Junqiao Zhao, Hongtu Zhou +4Soft Actor-Critic Sparse Rewards
Learning the Preferences of a Learning Agent
May 9, 2026Karim Abdel Sadek, Mark Bedaywi, Rhys Gould +1Preference Learning Entropy Regularized Reinforcement Learning
Large Language Models for Sequential Decision-Making: Improving In-Context Learning via Supervised Fine-Tuning
May 9, 2026Minmin Zhang, Sina Aghaei, Soroush SaghafianIn-Context Learning Large Language Model Fine-Tuning
A Single Deep Preference-Conditioned Policy for Learning Pareto Coverage Sets
May 9, 2026Akihiro Kubo, Kosuke Nakanishi, Shin IshiiOptimal Policies Constrained Reinforcement Learning
Quantile-Coupled Flow Matching for Distributional Reinforcement Learning
May 8, 2026Michael Groom, Victor-Alexandru Darvariu, Lars Kunze +2Bellman Equation

May 20, 2026 · cs.LGJ/K move · Enter open · S save

Beyond the Bellman Recursion: A Pontryagin-Guided Framework for Non-Exponential Discounting

Hojin Ko, Jeonggyu Huh

Most value-based and actor--critic reinforcement learning methods rely on Bellman-style recursions, yet these recursions collapse under non-exponential discounting common in human preferences and survival processes. We show the breakdown is structural: exponential discounting sits at a fragile intersection of multiplicativity and time homogeneity, and violating either property breaks standard dynamic programming. To overcome this, we propose Pontryagin-Guided Direct Policy Optimization (PG-DPO), a variational framework that abandons recursion and couples the Pontryagin Maximum Principle with Monte Carlo rollouts via an Adjoint-MC projection enforcing pointwise Hamiltonian maximization. Across multi-dimensional hyperbolic and survival-discount benchmarks, PG-DPO improves accuracy and stability where equation-driven solvers and critic-based baselines diverge.

Multi-Objective Policy Optimization Bellman Equation Recursion

Open paper Ask about it PDF