Temporal Difference

Follow this topic Ask about this topic

Recent momentum

-100%

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-07

1 new paper

A weekly snapshot of new work published in Temporal Difference.

31 papers

Latest in Temporal Difference

Open your feed →

Online Inference for Quantile Temporal Difference Learning in Distributional Reinforcement Learning
Aug 13, 2026Zijie Cheng, Yang Peng, Zhihua ZhangDistributional Reinforcement Learning Temporal Difference
Self-Normalized Inference for Constant-Stepsize Temporal-Difference Learning under Markovian Sampling
Aug 11, 2026Min Zeng, Yichen Zhang, Xiaofeng ShaoTemporal Difference Large Deviations Theory
Gated-BEPO: Confidence-Gated Bellman Credit Assignment for Large Language Model Agents
Aug 7, 2026Hongxi Yan, Ziyue Huang, Shichao Fan +1Goal-Conditioned Reinforcement Learning Large Language Model Agents
Visual Representation Matters: Exploiting Temporal Differences in Video-to-Audio Generation
Aug 5, 2026Zehua Chen, Junyou Wang, Yuxuan Jiang +5Joint Audio-Video Generation Diffusion Models
Revisiting TD Target Aggregation under Uncertainty in Q-Learning
Aug 4, 2026Lipeng Zu, Xiaonan ZhangDeep Q-Networks Q-Learning
Generalized Kalman filter based temporal difference reinforcement learning
Jul 22, 2026Vasos Arnaoutis, Eric Lutters, Bojana RosićTemporal Difference Extended Kalman Filter
Difference-Based Relational Learning for Zero-Shot Object-Goal Visual Navigation With Direct Sim-to-Real Transfer
Jul 17, 2026Guolei Qi, Feitian ZhangZero-Shot Sim-To-Real Transfer Vision-And-Language Navigation
Non-Convex Sparse Reinforcement Learning via Non-Monotone Inclusions
Jul 6, 2026Kyohei Suzuki, Konstantinos SlavakisLipschitz Constant Bellman Equation
Mesh-RL: Coupled subgrid reinforcement learning
Jun 24, 2026Behnam Gheshlaghi, Bahador Rashidi, Shahin AtakishiyevTemporal Difference Q-Learning
A Single Stepsize Suffices for Unprojected Linear TD(0): Simultaneous Robust and Fast Rates via Polyak--Ruppert Averaging
Jun 23, 2026Wei-Cheng Lee, Francesco OrabonaLarge Step Sizes Temporal Difference
On the Variance of Temporal Difference Learning and its Reduction Using Control Variates
Jun 18, 2026Hsiao-Ru Pan, Bernhard SchölkopfTemporal Difference Variance Reduction
pdSTL: Probabilistic Differentiable Signal Temporal Logic for Stochastic Systems
Jun 17, 2026Bennett Dogbey, Hemanth ManjunathaSignal Temporal Logic Gradient-Based Trajectory Optimization
A Diffusion Approximation for Temporal-Difference Learning with Linear Features under Markovian Noise
Jun 16, 2026M. Forzo, E. Monzio Compagnoni, A. Russo +1Temporal Difference Stochastic Differential Equations
You Don't Need Strong Assumptions: Visual Representation Learning via Temporal Differences
Jun 14, 2026Ninad Daithankar, Alexi Gladstone, Yann LeCun +1Visual Representation Learning Self-Supervised Learning
Temporal Difference Learning for Diffusion Models
Jun 13, 2026Qizhen Ying, Yangchen Pan, Victor Adrian Prisacariu +1Denoising Trajectory Generative Models
Fast and Robust Convergence Rate for TD(0) with Linear Function Approximation, Universal Learning Steps and I.I.D. Samples
Jun 4, 2026Ziad Kobeissi, Éloïse BerthierTemporal Difference Finite Time Analysis
Convergence of Two-Timescale Markovian Stochastic Approximations with Applications in Reinforcement Learning
May 29, 2026Vagul Mahadevan, Claire Chen, Shuze Daniel Liu +1Stochastic Approximation Temporal Difference
ST-SimDiff: Balancing Spatiotemporal Similarity and Difference for Efficient Video Understanding with MLLMs
May 21, 2026Bingjun Luo, Tony Wang, Chaoqi Chen +1Video Understanding Visual Tokens
TIDE: Asymmetric Neural Circuits for Stabilized Temporal Inhibitory-Excitatory Dynamics
May 19, 2026Alexander Kyuroson, Denis Kleyko, Marcus LiwickiNeural Dynamics Bio-Inspired Framework
Behavior-Aware Auxiliary Corrections for Off-Policy Temporal-Difference Prediction
May 17, 2026Xingguo Chen, Zhiang He, Yuchen Shen +4Temporal Difference Off-Policy Reinforcement Learning
Behavior-Induced Mirror-Prox Temporal-Difference Learning for Faster Off-Policy Prediction
May 16, 2026Xingguo Chen, Yuchen Shen, Shangdong Yang +3Temporal Difference Mirror Descent
The Benefits of Temporal Correlations: SGD Learns k-Juntas from Random Walks Efficiently
May 11, 2026Elisabetta Cornacchia, Dan Mikulincer, Elchanan MosselStochastic Gradient Descent Optimal Sample Complexity
One for All: A Non-Linear Transformer can Enable Cross-Domain Generalization for In-Context Reinforcement Learning
May 10, 2026Bowen He, Juncheng Dong, Lin Lin +1Safe Reinforcement Learning In-Context Learning
Equivalence of Coarse and Fine-Grained Models for Learning with Distribution Shift
May 7, 2026Adam R. Klivans, Shyamal Patel, Konstantinos Stavropoulos +1Distributional Learning Distribution Shifts
On the Divergence of Differential Temporal Difference Learning without Local Clocks
May 7, 2026David Antrobius, Shangtong ZhangTemporal Difference Clocks
A Finite-Iteration Theory for Asynchronous Categorical Distributional Temporal-Difference Learning
May 7, 2026Ege C. Kaya, Abolfazl HashemiTemporal Difference Distributional Learning
Extending Differential Temporal Difference Methods for Episodic Problems
May 6, 2026Kris De Asis, Mohamed Elsayed, Jiamin HeTemporal Difference Finite Planning Horizon
Structural Equivalence and Learning Dynamics in Delayed MARL
May 5, 2026Jules Sintes, Ana Bušić, Jiamin ZhuPartially Observable Markov Decision Process Multi-Agent Reinforcement Learning
Bridging the Gap Between Average and Discounted TD Learning
May 3, 2026Haoxing Tian, Zaiwei Chen, Ioannis Ch. Paschalidis +1Temporal Difference Bellman Equation
Regularized Centered Emphatic Temporal Difference Learning
May 2, 2026Xingguo Chen, Chaohui Wu, Jinguo Ye +5Temporal Difference Regularization Methods
Statistical Inference for Policy Evaluation with Temporal Difference Learning
Oct 21, 2024Weichen Wu, Gen Li, Yuting Wei +1Temporal Difference Value Functions

Aug 13, 2026 · stat.MLJ/K move · Enter open · S save

Online Inference for Quantile Temporal Difference Learning in Distributional Reinforcement Learning

Zijie Cheng, Yang Peng, Zhihua Zhang

School of Mathematical Sciences, Peking University · Yau Mathematical Sciences Center, Tsinghua University

In this paper, we study how to perform statistical inference for quantile temporal difference learning (QTD) in distributional reinforcement learning. Assuming access to a generative model, we first establish functional central limit theorems for both synchronous and asynchronous QTD, which show that the averaged iterates of QTD converge weakly to a rescaled Brownian motion. We next provide online inference methods. Based on random scaling, the inference procedure constructs an asymptotically pivotal statistic for inference by using the information along the whole QTD path. Meanwhile, the proposed statistic can be computed online without storing the entire trajectory of QTD iterates. This substantially reduces the memory requirement and enables efficient statistical inference in distributional reinforcement learning.

Distributional Reinforcement Learning Temporal Difference Distributional Learning Large Deviations Theory

Open paper Ask about it PDF