Value Functions

Follow this topic Ask about this topic

Recent momentum

-60%

4 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-14

2 new papers

A weekly snapshot of new work published in Value Functions.

67 papers

Latest in Value Functions

Open your feed →

Optimal Value Inference for Reinforcement Learning
Sep 9, 2026Nan Lu, Ethan Lee, James M. Robins +2Value Functions Bellman Equation
The Surprising Effectiveness of Approximate Value Iteration in Self-Play
Sep 8, 2026Raphael Boige, Amine Boumaza, Bruno ScherrerSelf-Play Value Functions
Locally-Guided Actor-Critic: Training a Goal-conditioned Actor with a Subgoal-aware Critic
Aug 31, 2026Olivier Serris, Stéphane Doncieux, Olivier SigaudGoal-Conditioned Reinforcement Learning Soft Actor-Critic
Game-Agnostic Value Functions through Automatic JSON Feature Extraction
Aug 30, 2026Dien Nguyen, Diego Perez-LiebanaValue Functions Value
Thermodynamics of Learning: A Typed Four-Component Accounting of Memory, Fit, and Value
Aug 13, 2026Akihito SudoValue Functions Value
Aftab: A Comprehensive Benchmark of CNN Encoders and Advanced Value Functions in Parallelized Q-Networks
Aug 7, 2026Taha Shieenavaz, Shabnam Zareshahraki, Loris NanniDeep Q-Networks Value Functions
Differentiating Through Dual Prices: End-to-End Policy Learning Under Capacity Constraints
Aug 5, 2026Mohammadsaeed Haghi, Mahdi Salmani, Nima KelidariQueueing Prices
Analytic Planning under Uncertainty with Moment Closure
Aug 3, 2026Shishir Sharma, Doina PrecupPredictive Uncertainty Model-Free Reinforcement Learning
Upper-Expectile Multi-Step Q-Learning for Off-Policy Reinforcement Learning
Aug 3, 2026Abdelghani Ghanem, Mounir GhoghoQ-Learning Off-Policy Reinforcement Learning
General Value Functions for Remaining Useful Life and Failure-Mode Prediction
Jul 24, 2026Hao Yan, Ali Sarabi, Qing Zou +1Early Failure Prediction Useful Life Prediction
Variance-Reduced Q-Learning over Static and Time-Varying Networks
Jul 24, 2026Sreejeet Maity, Feng Zhu, Aritra Mitra +1Multi-Agent Reinforcement Learning Value Functions
Relative Value Learning
Jul 23, 2026Marc Höftmann, Jan Robine, Stefan HarmelingValue Functions Bellman Equation
Asymptotically Optimal Regret for Reinforcement Learning without Horizon Dependence
Jul 22, 2026Runlong Zhou, Zihan Zhang, Maryam Fazel +1Finite Planning Horizon Finite-Horizon Episodic Markov Decision Processes
Generalised Bellman recurrence and three dualities in sequential decision-making
Jul 20, 2026Fernando E. Rosas, David Hyland, Daniel PolaniBellman Equation Value Functions
A Causal Markov Condition for Value
Jul 18, 2026Olav Benjamin VassendConditional Independence Value Functions
Reverse to Advance: Teleoperation-Cost Effective Hard Policy Learning from Reversed Easy Tasks
Jul 15, 2026Qiyuan Qiao, Ge Yuan, Can Wang +1Teleoperation Robotic Manipulation Policies
Environment Parameter Gradient Theorem for Policy-Environment Co-Design in Reinforcement Learning
Jul 14, 2026Amber SrivastavaValue Functions Gradcam
Robo-ValueRL: Reliable Value Estimation for Offline-to-Online Reinforcement Learning
Jul 10, 2026Wenke Xia, Pei Ren, Wenbo Yu +10Offline Safe Reinforcement Learning Value Functions
Principled Analysis of Deep Reinforcement Learning Evaluation and Design Paradigms
Jul 8, 2026Ezgi KorkmazSafe Reinforcement Learning Value Functions
Deep Learning for Dynamic Programming with Recursive Utility
Jul 5, 2026Xianhua Peng, Wu GuoValue Functions Optimal Learner
Generalization in offline RL: The structure is more important than the amount of pessimism
Jul 2, 2026Max Weltevrede, Matthijs T. J. Spaan, Wendelin BöhmerOffline Safe Reinforcement Learning Pessimism
Learning the Supports for Categorical Critic in Reinforcement Learning
Jul 2, 2026Jen-Yen Chang, Takayuki Osa, Tatsuya HaradaSoft Actor-Critic Value Functions
ShardNet: Training Neural Controllers with Hard, Non-Convex Constraints
Jun 29, 2026Long Kiu Chung, Shreyas KousikNeural Control Safety Constraints
World Value Models for Robotic Manipulation
Jun 23, 2026Zhihao Wang, Jianxiong Li, Yu Cui +4Robotic Manipulation Value Functions
Beyond Monotonic Progress: Retry-Supervised Value Learning for Robot Imitation
Jun 23, 2026Xinyao Qin, Junjie Lu, Kaixin Wang +7Robot Imitation Learning Imitation
Hessian-augmented Supervised Learning for Hamilton-Jacobi-Bellman PDEs
Jun 22, 2026Matías Gómez-Aedo, Behzad Azmi, Yuyang Huang +2Optimal Control Hessian Matrix
VIMPO: Value-Implicit Policy Optimization for LLMs
Jun 18, 2026Zhewei Kang, Aosong Feng, Sergey Levine +2Verifiable Rewards Relative Policy Optimization
UniIntervene: Agentic Intervention for Efficient Real-World Reinforcement Learning
Jun 10, 2026Haoyuan Deng, Yitong Gao, Yudong Lin +3Value Functions Agentic AI Systems
A Mathematical Theory of Value: a synthesis on goal-directed agency under resource constraints
Jun 10, 2026Cheng QianValue Value Functions
Bellman-Taylor Score Decoding for Markov Decision Processes with State-Dependent Feasible Action Sets
Jun 9, 2026Yi Chen, Rushuai Yang, Qiang Chen +2Markov Decision Processes Bellman Equation
Bayesian learning for the stochastic shortest path problem
Jun 3, 2026Chon Wai Ho, Sumeetpal S. Singh, Jiaqi GuoMarkov Decision Processes Bayesian Learning
A Goal-Set Characterization of Task Composition in the Boolean Task Algebra
Jun 2, 2026Eduardo Terrés-Caballero, Herke van HoofValue Functions Task Synthesis
ShaplEIG: Bayesian Experimental Design for Shapley Value Estimation
Jun 1, 2026David Rundel, Fabian Fumagalli, Maximilian Muschalik +2Shapley Value Bayesian Experimental Design
Shape Your Body: Value Gradients for Multi-Embodiment Robot Design
May 30, 2026Nico Bohlinger, Jan PetersReal-Robot Experiments Value Functions
Value Functions as Supermartingale Certificates
May 29, 2026Alessandro Abate, Daniel Contro, Mirco Giacobbe +2Value Functions Markov Decision Processes
BASIS: Batchwise Advantage Estimation from Single-Rollout Information Sharing for LLM Reasoning
May 26, 2026Shijin Gong, Erhan Xu, Kai Ye +3Reinforcement Learning Base-To-Thinking Gap Verifiable Rewards
In LLM Reasoning, there is Irrationality on top of Value Misalignment
May 26, 2026Kejiang Qian, Fengxiang HeLLM Reasoning Methods Rationality
Global linear convergence of entropy-regularized softmax policy gradient beyond tabular MDPs
May 24, 2026Ziyue Chen, David Šiška, Lukasz SzpruchPolicy Gradient Kullback-Leibler Regularization
Generative Modeling by Value-Driven Transport
May 21, 2026Pablo Moreno-Muñoz, Adrian Müller, Gergely NeuGenerative Models Stochastic Optimal Control
On the Sample Complexity of Discounted Reinforcement Learning with Optimized Certainty Equivalents
May 20, 2026Oliver Mortensen, Mohammad Sadegh TalebiValue Functions Markov Decision Processes
Dynamic Shapley Computation
May 20, 2026Xuan Yang, Hsi-Wen Chen, Ming-Syan Chen +1Shapley Value Value Functions
Stitched Value Model for Diffusion Alignment
May 19, 2026Hyojun Go, Hyungjin Chung, Prune Truong +8Generative Flow Networks Sewing
Learning Interpretable Point-Based Clinical Risk Scores via Direct Optimization
May 18, 2026Ying Cui, Albert M Li, Vivek Charu +3Risk Scores Clinical Prediction
Sign-Separated Asymmetric Finite-Time Error Analysis of Q-Learning
May 15, 2026Donghwan LeeQ-Learning Finite Time Analysis
Heuristic Pathologies and Further Variance Reduction via Uncertainty Propagation in the AIVAT Family of Techniques
May 14, 2026Juho Kim, Tuomas SandholmVariance Uncertainty Propagation
ACSAC: Adaptive Chunk Size Actor-Critic with Causal Transformer Q-Network
May 10, 2026Qian Chen, Junqiao Zhao, Hongtu Zhou +4Soft Actor-Critic Sparse Rewards
Drifting Field Policy: A One-Step Generative Policy via Wasserstein Gradient Flow
May 8, 2026Juil Koo, Mingue Park, Jiwon Choi +2Diffusion Policies Wasserstein Gradient Flows
Q-MMR: Off-Policy Evaluation via Recursive Reweighting and Moment Matching
May 7, 2026Xiang Li, Nan JiangOff-Policy Reinforcement Learning Value Functions
Soft Deterministic Policy Gradient with Gaussian Smoothing
May 7, 2026Hyunjun Na, Donghwan LeePolicy Gradient Value Functions
QuadraSHAP: Stable and Scalable Shapley Values for Product Games via Gauss-Legendre Quadrature
May 7, 2026Majid Mohammadi, Grigory Reznikov, Pavel Sinitcyn +2Shapley Value Coalitions
Long-Horizon Q-Learning: Accurate Value Learning via n-Step Inequalities
May 7, 2026Armaan A. Abraham, Lucy Xiaoyang Shi, Chelsea FinnQ-Learning Value Functions
Exact Dual Geometry of SOC-ICNN Value Functions
May 6, 2026Kang Liu, Jianchen Hu, Wei PengGraph Neural Network Inference Convex Sets
Value Functions for Temporal Logic: Optimal Policies and Safety Filters
May 1, 2026Oswin So, William Sharpless, Sylvia Herbert +1Value Functions Signal Temporal Logic
SAVGO: Learning State-Action Value Geometry with Cosine Similarity for Continuous Control
May 1, 2026Stavros Orfanoudakis, Pedro P. VergaraSimulation-Based Reinforcement Learning Value Functions
Kernelized Advantage Estimation: From Nonparametric Statistics to LLM Reasoning
Apr 30, 2026Shijin Gong, Kai Ye, Jin Zhu +3LLM Reasoning Methods Policy Gradient
DGLight: DQN-Guided GRPO Fine-Tuning of Large Language Models for Traffic Signal Control
Apr 28, 2026Chenbo YuTraffic Signal Control Deep Q-Networks
Distributional Reinforcement Learning via the Cramér Distance
Apr 26, 2026Vanya Aziz, Ivo Nowak, E. M. T HendrixDistributional Reinforcement Learning Soft Actor-Critic
Reward Models Are Secretly Value Functions: Temporally Coherent Reward Modeling
Apr 24, 2026Alex NikulkovLarge Reasoning Models Value Functions
Beyond Bellman: High-Order Generator Regression for Continuous-Time Policy Evaluation
Apr 21, 2026Yaowei Zheng, Richong Zhang, Shenxi Wu +5Value Functions Bellman Equation
Scale-free adaptive planning for deterministic dynamics & discounted rewards
Apr 20, 2026Peter L. Bartlett, Victor Gabillon, Jennifer Healey +1Finite-Horizon Episodic Markov Decision Processes Classical Planning

Sep 9, 2026 · stat.MLJ/K move · Enter open · S save

Optimal Value Inference for Reinforcement Learning

Nan Lu, Ethan Lee, James M. Robins, David Simchi-Levi+1

We study offline inference for the optimal value in reinforcement learning. Two new nuisances are derived as fixed points of a self-induced Bellman equation, in which we approximate the maximum Bellman operator by its softmax correspondence. We propose a debiased estimator through the Neyman orthogonality and establish its asymptotic normality under diverging horizons even when the behavior policy changes with time, as long as the nuisances have the statistical rates that can be achieved by many machine learning methods. We provide a concrete estimating procedure for these nuisances and show they can lead to valid inference. Synthetic experiments validate the numerical performance of our inference method, and we implement it in real-life decision-making problems, including bike repositioning and AI agentic tool use.

Value Functions Bellman Equation Inference-Time Control Optimal

Open paper Ask about it PDF

Recent momentum

Neighbouring directions

Recent digests

2 new papers

Latest in Value Functions