Temporal-Difference Learning

Follow this topic Ask about this topic

Momentum

9 papers in the last four weeks, up 200% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 57

All topics

$λ$ -Reachability: Geometric-Horizon Safety Bellman Equations for Humanoid Safety
Jun 14, 2026Rui Chen, Shangtao Li, Yifan Sun +1Reachability Analysis Temporal-Difference Learning
Robotics Institute Carnegie Mellon University United States · Mechanical Engineering Carnegie Mellon University United States
Fast and Robust Convergence Rate for TD(0) with Linear Function Approximation, Universal Learning Steps and I.I.D. Samples
Jun 4, 2026Ziad Kobeissi, Éloïse BerthierStochastic Approximation Temporal-Difference Learning
L2S · L2S, INRIA Université Paris-Saclay, CentraleSupelec, Gif-sur-Yvette, France · U2IS +1
Trace-Mediated Peak Bias: Bridging Temporal Credit Assignment and Cognitive Heuristics in Deep Reinforcement Learning
Jun 3, 2026Viktor Veselý, Aleksandar Todorov, Erwan Escudie +1Temporal Credit Assignment Temporal-Difference Learning
Department of AI, University of Groningen, The Netherlands
Non-Asymptotic Convergence of Stochastic Iterative Algorithms: A Lyapunov Framework
May 29, 2026Zaiwei Chen, Siva Theja MaguluriLyapunov Stability Stochastic Approximation
Purdue IE · Georgia Tech IE
Convergence of Two-Timescale Markovian Stochastic Approximations with Applications in Reinforcement Learning
May 29, 2026Vagul Mahadevan, Claire Chen, Shuze Daniel Liu +1Stochastic Approximation Temporal-Difference Learning
Metron, Reston, VA, USA · Division Office Physics, Math and Astronomy, California Institute of Technology, Pasadena, CA, USA · Data Science Lab, MIT, Cambridge, MA, USA +2
Behavior-Aware Auxiliary Corrections for Off-Policy Temporal-Difference Prediction
May 17, 2026Xingguo Chen, Zhiang He, Yuchen Shen +4Temporal-Difference Learning Linear Function Approximation
Nanjing University of Posts and Telecommunications, Nanjing, China · Department of Computer Science and Technology, Nanjing University, Nanjing, China · College of Electronic Countermeasure, National University of Defense Technology, Hefei, China
Behavior-Induced Mirror-Prox Temporal-Difference Learning for Faster Off-Policy Prediction
May 16, 2026Xingguo Chen, Yuchen Shen, Shangdong Yang +3Off-Policy Evaluation Temporal-Difference Learning
Nanjing University of Posts and Telecommunications, Nanjing, China · Department of Computer Science and Technology, Nanjing University, Nanjing, China · College of Electronic Countermeasure, National University of Defense Technology, Hefei, China
Adaptive TD-Lambda for Cooperative Multi-agent Reinforcement Learning
May 12, 2026Yue Deng, Zirui Wang, Yin ZhangTemporal-Difference Learning Density Ratio Estimation
1Zhongguancun Academy, Beijing, China · College of Computer Science and Technology, Zhejiang University
Quotient-Categorical Representations for Bellman-Compatible Average-Reward Distributional Reinforcement Learning
May 11, 2026Ege C. Kaya, Aliasghar Pourghani, Vijay Gupta +1Distributional RL Reinforcement Learning
Elmore Family School of Electrical and Computer Engineering Purdue University West Lafayette, IN 47906
One for All: A Non-Linear Transformer can Enable Cross-Domain Generalization for In-Context Reinforcement Learning
May 10, 2026Bowen He, Juncheng Dong, Lin Lin +1Reinforcement Learning Transformer
Duke University, Durham, U.S.A.
MemQ: Integrating Q-Learning into Self-Evolving Memory Agents over Provenance DAGs
May 8, 2026Junwei Liao, Haoting Shi, Ruiwen Zhou +9LLM Agent Memory Q-Learning
Shanghai Jiao Tong University · Shanghai Innovation Institute · National University of Singapore +3
Beyond Linear Attention: Softmax Transformers Implement In-Context Reinforcement Learning
May 8, 2026Zixuan Xie, Xinyu Liu, Claire Chen +3Reinforcement Learning Transformer-Based RL
University of Virginia · California Institute of Technology · Purdue University
Convergence and Emergence of In-Context Reinforcement Learning with Chain of Thought
May 8, 2026Zixuan Xie, Xinyu Liu, Rohan Chandra +1Reinforcement Learning Transformer
University of Virginia
On the Divergence of Differential Temporal Difference Learning without Local Clocks
May 7, 2026David Antrobius, Shangtong ZhangReinforcement Learning Average-Reward RL
Department of Computer Science University of Virginia
A Sharp Finite-Iteration Theory for Asynchronous Categorical Distributional Temporal-Difference Learning
May 7, 2026Ege C. Kaya, Abolfazl HashemiDistributional RL Reinforcement Learning
Elmore Family School of Electrical and Computer Engineering Purdue University West Lafayette, IN 47906
Beyond the Independence Assumption: Finite-Sample Guarantees for Deep Q-Learning under $τ$ -Mixing
May 7, 2026Leon Halgryn, Sophie Langer, Janusz M. Meylahn +1Reinforcement Learning Deep Q-Learning
University of Twente · Department of Applied Mathematics University of Twente · Ruhr-Universität Bochum +1
AdaGamma: State-Dependent Discounting for Temporal Adaptation in Reinforcement Learning
May 7, 2026Yaomin Wang, Jianting Pan, Ran Tian +4Reinforcement Learning Temporal-Difference Learning
School of Data Science, The Chinese University of Hong Kong, Shenzhen
Long-Horizon Q-Learning: Accurate Value Learning via n-Step Inequalities
May 7, 2026Armaan A. Abraham, Lucy Xiaoyang Shi, Chelsea FinnQ-Learning Temporal-Difference Learning
Stanford University
Extending Differential Temporal Difference Methods for Episodic Problems
May 6, 2026Kris De Asis, Mohamed Elsayed, Jiamin HeReinforcement Learning Temporal-Difference Learning
1Openmind Research Institute · Department of Computing Science, University of Alberta, Canada · 3Alberta Machine Intelligence Institute (Amii)
Bridging the Gap Between Average and Discounted TD Learning
May 3, 2026Haoxing Tian, Zaiwei Chen, Ioannis Ch. Paschalidis +1Reinforcement Learning Average-Reward RL
Department of Electrical and Computer Engineering Boston University · Edwardson School of Industrial Engineering Purdue University
AdamO: A Collapse-Suppressed Optimizer for Offline RL
May 3, 2026Nan Qiao, Sheng Yue, Shuning Wang +1Reinforcement Learning Temporal-Difference Learning
1Tsinghua University, China · 2Central South University, China · 3Sun Yat-sen University, China
Regularized Centered Emphatic Temporal Difference Learning
May 2, 2026Xingguo Chen, Chaohui Wu, Jinguo Ye +5Temporal-Difference Learning
Nanjing University of Posts and Telecommunications · Nanjing University · Microsoft +1
Bayesian policy gradient and actor-critic algorithms
Apr 30, 2026Mohammad Ghavamzadeh, Yaakov Engel, Michal ValkoBayesian RL Temporal-Difference Learning
Adobe Research & Inria · Rafael Advanced Defence System, Israel · Inria Lille — SequeL team, France
Intentional Updates for Streaming Reinforcement Learning
Apr 21, 2026Arsalan Sharifnassab, Mohamed Elsayed, Kris De Asis +2Reinforcement Learning Temporal-Difference Learning
Openmind Research Institute · Department of Computing Science, University of Alberta.
Ensemble Elastic DQN: A Step Dependent Ensemble Approach for Reducing Overestimation in Deep Value-Based Reinforcement Learning
Jun 6, 2025Adrian Ly, Richard Dazeley, Peter Vamplew +2Reinforcement Learning Deep Q-Learning
Reinforcement Learning in Switching Non-Stationary Markov Decision Processes: Algorithms and Convergence Analysis
Mar 24, 2025Mohsen Amiri, Sindri MagnússonNon-Stationary RL Markov Decision Processes
Department of Computer and System Science, Stockholm University, SE-164 25 Stockholm, Sweden
Statistical Inference for Policy Evaluation with Temporal Difference Learning
Oct 21, 2024Weichen Wu, Gen Li, Yuting Wei +1Parameter Estimation Confidence Region Estimation
The Voleon Group, New York, NY, USA · Department of Statistics, The Chinese University of Hong Kong, Hong Kong SAR. · Department of Statistics and Data Science, The Wharton School, University of Pennsylvania, Philadelphia, PA, USA. +1

Jun 14, 2026, cs.ROJ/K move · Enter open · S save

$λ$ -Reachability: Geometric-Horizon Safety Bellman Equations for Humanoid Safety

Rui Chen, Shangtao Li, Yifan Sun, Changliu Liu

Robotics Institute Carnegie Mellon University United States · Mechanical Engineering Carnegie Mellon University United States

Reachability Analysis Temporal-Difference Learning Robot Safety Humanoid Robotics

Open paper AskPDF arXiv

We introduce

λ

-Reachability, a scalable approach to Hamilton--Jacobi safety analysis for high-dimensional robotic systems. Unlike prior discounted formulations that rely on fixed one-step Bellman updates,

λ

-Reachability employs a stochastic multi-step estimator of the safety value, using a geometrically distributed rollout horizon together with a randomly absorbed terminal. Conceptually analogous to TD(

λ

λ

-Reachability interpolates between local self-consistency updates and long-horizon max-over-trajectory safety targets via an interpretable horizon-control parameter. Unlike TD(

λ

), where the terminal value is always incorporated in learning targets, the terminal safety value in

λ

-Reachability is only used at a probability controlled by parameter

δ

. We formally show that for

δ<1

, the update induces a contraction mapping that allows temporal-difference learning; as

λ\to 1

, the estimator recovers the undiscounted reachability objective. We apply

λ

-Reachability to high-dimensional safety learning problems with both simulated and real humanoid robots under balance and collision avoidance constraints. Experimental results demonstrate that

λ

-Reachability significantly improves both safe-set boundary classification and safety margin estimation compared to single-step temporal-difference baselines.

Momentum

Related topics

Latest papers 57