Value Functions

Follow this topic Ask about this topic

Momentum

12 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 78

All topics

One-Step Generative Modeling via Training Dynamics Action
Sep 30, 2026Zhangyong Liang, Ying Huang, Haibin LingGenerative Models Student-Generated Trajectories
Department of Artificial Intelligence, Westlake University · Hangzhou Normal University
Loop-Free Inverse Reinforcement Learning via Sequential Value Recovery with Q-Score Matching
Sep 30, 2026Yang chen, Yitan Zhang, Michael Witbrock +1Offline Reinforcement Learning Value Functions
Shanghai Artificial Intelligence Laboratory · University of Auckland
A Reachability-based Safety Certificate for Dynamical System Motion Policies
Sep 29, 2026Aditya Vats, Tianyi Xia, Nadia FigueroaProbabilistic Safety Control Barrier Functions
The authors are with the University of Pennsylvania, Philadelphia, PA, USA.
Deep Epistemic Value Functions for Optimistic Exploration
Sep 28, 2026Leander Diaz-Bone, Marco Bagatella, Jonas Hübotter +1Stochastic Exploration Value Functions
ETH Zürich, Switzerland · Max Planck Institute for Intelligent Systems, Germany
Statistical Convergence of Transformer Encoder-Accelerated Robust Reinforcement Learning
Sep 20, 2026Suman Banerjee, Hiroyasu TsukamotoOffline Reinforcement Learning Value Functions
Department of Aerospace Engineering, The Grainger College of Engineering, University of Illinois Urbana-Champaign, Urbana, IL, USA
SeeQ: Training Generalist Value Functions for Long-Horizon Robotic Manipulation
Sep 18, 2026Saksham Singh, Zheyuan Hu, Max Sobol Mark +3Robot Policies Robotic Manipulation
Carnegie Mellon University
Winning a Won Game: Strict Reach-Avoid-Stay Control Barrier Functions for High-Dimensional Black-Box Systems
Sep 16, 2026Donggeon David Oh, Duy P. Nguyen, Gongkai Yuan +3Control Barrier Functions Value Functions
Department of Computer Science, Johns Hopkins University, Baltimore, MD 21218, USA
A Convergence Framework for Deep $V$ -Learning: Error Propagation and Sharp Action-Gap Bounds
Sep 16, 2026Yury KolomeytsevValue Functions Optimal Learner
Faculty of Computational Mathematics and Cybernetics Lomonosov Moscow State University
Certified Safety Curation: Distribution-Free Guarantees for Safe Offline Reinforcement Learning
Sep 14, 2026Adam Haroon, Cody FlemingModel-Based Reinforcement Learning Safety Claims
Iowa State University Ames, IA, USA
Learning to Solve Stochastic Controls with Unknown Drifts and Running Rewards: Theory, Algorithms and Convergence
Sep 14, 2026Jin Ma, Gaozhan Wang, Jianfeng Zhang +1Stochastic Optimal Control Value Functions
Department of Mathematics, University of Southern California, Los Angeles, CA 90089 · Department of Industrial Engineering and Operations Research, Columbia University, New York, NY 10027
Optimal Value Inference for Reinforcement Learning
Sep 9, 2026Nan Lu, Ethan Lee, James M. Robins +2Value Functions Bellman Equation
The Surprising Effectiveness of Approximate Value Iteration in Self-Play
Sep 8, 2026Raphael Boige, Amine Boumaza, Bruno ScherrerSelf-Play Monte Carlo Tree Search
Université de Lorraine, CNRS, Inria, LORIA, F-54000 Nancy, France
Game-Agnostic Value Functions through Automatic JSON Feature Extraction
Aug 30, 2026Dien Nguyen, Diego Perez-LiebanaValue Functions Autonomous Agents
Queen Mary University of London - Game AI Group
Aftab: A Progressive Design Study of Visual Encoders and Value Estimation for Replay-Free Parallelized Q-Learning
Aug 7, 2026Taha Shieenavaz, Shabnam Zareshahraki, Loris NanniDeep Q-Networks Value Functions
Department of Information Engineering, University of Padua, Italy
Analytic Planning under Uncertainty with Moment Closure
Aug 3, 2026Shishir Sharma, Doina PrecupPredictive Uncertainty Value Functions
1McGill University, Montreal, Canada · 2Mila – Quebec Artificial Intelligence Institute, Montreal, Canada
Upper-Expectile Multi-Step Q-Learning for Off-Policy Reinforcement Learning
Aug 3, 2026Abdelghani Ghanem, Mounir GhoghoQ-Learning Value Functions
College of Computing Mohammed VI Polytechnic University Rabat 11103, Morocco
Climate-Dyna Deep Hedging for XVAs: Model-Based Reinforcement Learning, Residual Climate HVA, and Hedge-Instrument Discovery
Aug 2, 2026Xiaozhen Wang, Francois Buet-GolfouseClimate Value Functions
CEREMADE Université Paris Dauphine–PSL, France · AIML Global Markets Barclays, United Kingdom
Do You Really Need to Pretrain Q-Functions for Online RL Fine-Tuning?
Jul 29, 2026Perry Dong, Ron Polonsky, Dorsa Sadigh +1Reinforcement Fine-Tuning Value Functions
Stanford University
Sparse Gaussian-Mixture-Model Q-Functions via Hadamard Overparametrization for Online Reinforcement Learning
Jul 26, 2026Minh Vu, Konstantinos SlavakisOffline Reinforcement Learning Value Functions
Institute of Science Tokyo, Department of Information and Communications Engineering, Yokohama, Japan.
General Value Functions for Remaining Useful Life and Failure-Mode Prediction
Jul 24, 2026Hao Yan, Ali Sarabi, Qing Zou +1Useful Life Prediction Early Failure Prediction
School of Computing and Augmented Intelligence, Arizona State University
Variance-Reduced Q-Learning over Static and Time-Varying Networks
Jul 24, 2026Sreejeet Maity, Feng Zhu, Aritra Mitra +1Multi-Agent Reinforcement Learning Decentralized Learning
Department of Electrical and Computer Engineering, North Carolina State University · Department of Electrical and Computer Engineering, University of California San Diego, San Diego, USA
Relative Value Learning
Jul 23, 2026Marc Höftmann, Jan Robine, Stefan HarmelingValue Functions Bellman Equation
Department of Computer Science, Technical University of Dortmund, Germany · Lamarr Institute for Machine Learning and Artificial Intelligence
Asymptotically Optimal Regret for Reinforcement Learning without Horizon Dependence
Jul 22, 2026Runlong Zhou, Zihan Zhang, Maryam Fazel +1Model-Free Reinforcement Learning Value Functions
University of Washington. · Hong Kong University of Science and Technology. · §University of Washington. +1
Generalised Bellman recurrence and three dualities in sequential decision-making
Jul 20, 2026Fernando E. Rosas, David Hyland, Daniel PolaniBellman Equation Value Functions
Department of Informatics, University of Sussex · Department of Computer Science, University of Oxford · Department of Computer Science, University of Hertfordshire
Environment Parameter Gradient Theorem for Policy-Environment Co-Design in Reinforcement Learning
Jul 14, 2026Amber SrivastavaPolicy Gradient Value Functions
Faculty of Mechanical Engineering, Indian Institute of Technology Delhi, India
Robo-ValueRL: Reliable Value Estimation for Offline-to-Online Reinforcement Learning
Jul 10, 2026Wenke Xia, Pei Ren, Wenbo Yu +10Model-Based Reinforcement Learning Value Functions
1Gaoling School of Artificial Intelligence, Renmin University of China · 2Beijing Innovation Center of Humanoid Robotics · 3Beijing Forestry University +4
Principled Analysis of Deep Reinforcement Learning Evaluation and Design Paradigms
Jul 8, 2026Ezgi KorkmazValue Functions Principles
Deep Learning for Dynamic Programming with Recursive Utility
Jul 5, 2026Xianhua Peng, Wu GuoValue Functions Optimal Learner
HSBC Business School, Peking University, University Town, Nanshan District, Shenzhen, 518055, China.
Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models
Jul 4, 2026Xinyi Xie, Zican Hu, Zhanyu Liu +7Frozen Vision-Language Models Action Generation
Nanjing University · Australian National University · Institute of Automation, Chinese Academy of Sciences +1
Generalization in offline RL: The structure is more important than the amount of pessimism
Jul 2, 2026Max Weltevrede, Matthijs T. J. Spaan, Wendelin BöhmerPessimism Model-Based Reinforcement Learning
Delft University of Technology, Delft, The Netherlands
Learning the Supports for Categorical Critic in Reinforcement Learning
Jul 2, 2026Jen-Yen Chang, Takayuki Osa, Tatsuya HaradaSoft Actor-Critic Value Functions
Graduate School of Information Science and Technology, The University of Tokyo · RIKEN AIP
ShardNet: Training Neural Controllers with Hard, Non-Convex Constraints
Jun 29, 2026Long Kiu Chung, Shreyas KousikSafety Constraints Learning-Based Control
Department of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA.
World Value Models for Robotic Manipulation
Jun 23, 2026Zhihao Wang, Jianxiong Li, Yu Cui +4Efficient World-Action Model Robotic Manipulation
ByteDance Seed · Tsinghua University · Peking University
Beyond Monotonic Progress: Retry-Supervised Value Learning for Robot Imitation
Jun 23, 2026Xinyao Qin, Junjie Lu, Kaixin Wang +7Imitation Value Functions
Tsinghua University · Work done at Microsoft Research Asia · University of Technology Sydney +3
Hessian-augmented Supervised Learning for Hamilton-Jacobi-Bellman PDEs
Jun 22, 2026Matías Gómez-Aedo, Behzad Azmi, Yuyang Huang +2Optimal Control Hamilton-Jacobi Reachability
Department of Mathematics, Imperial College London, South Kensington Campus, SW7 2AZ London, UK · Department of Mathematics and Statistics, University of Konstanz, Germany · RICAM and Institute of Mathematics and Scientific Computing, University of Graz, Austria
Horizon Adaptive Offline Policy Learning via Value Stitching
Jun 19, 2026Kexin Zheng, Xianyuan Zhan, Xintao YanValue Functions Horizon
The University of Hong Kong · Tsinghua University
VIMPO: Value-Implicit Policy Optimization for LLMs
Jun 18, 2026Zhewei Kang, Aosong Feng, Sergey Levine +2Reinforcement Learning With Verifiable Reward Frictive Policy Optimization
UC Berkeley · Yale University
UniIntervene: Agentic Intervention for Efficient Real-World Reinforcement Learning
Jun 10, 2026Haoyuan Deng, Yitong Gao, Yudong Lin +3Value Functions Agentic
Nanyang Technological University · Beijing University of Posts and Telecommunications
A Mathematical Theory of Value: a synthesis on goal-directed agency under resource constraints
Jun 10, 2026Cheng QianValue Utility Maximization
Bellman-Taylor Score Decoding for Markov Decision Processes with State-Dependent Feasible Action Sets
Jun 9, 2026Yi Chen, Rushuai Yang, Qiang Chen +2Markov Decision Processes Bellman Equation
Lucy · Department of Industrial Engineering and Decision Analytics Hong Kong University of Science and Technology
Bayesian learning for the stochastic shortest path problem
Jun 3, 2026Chon Wai Ho, Sumeetpal S. Singh, Jiaqi GuoMarkov Decision Processes Bayesian Neural Networks
Department of Engineering, University of Cambridge, UK · School of Mathematics and Physics, University of Wollongong, Wollongong, Australia
ShaplEIG: Bayesian Experimental Design for Shapley Value Estimation
Jun 1, 2026David Rundel, Fabian Fumagalli, Maximilian Muschalik +2Shapley Value Bayesian Experimental Design
Department of Statistics, LMU Munich, Munich, Germany · Munich Center for Machine Learning, Munich, Germany · Institute for Informatics, LMU Munich, Munich, Germany +2
Shape Your Body: Value Gradients for Multi-Embodiment Robot Design
May 30, 2026Nico Bohlinger, Jan PetersValue Functions Body
Technical University of Darmstadt, Germany · Robotics Institute Germany (RIG); German Research Center for AI (DFKI); hessian.AI
Value Functions as Supermartingale Certificates
May 29, 2026Alessandro Abate, Daniel Contro, Mirco Giacobbe +2Value Functions Reinforcement Learning With Verifiable Reward
University of Oxford, UK · University of Birmingham, UK
Reward Transfer from Inverse Reinforcement Learning: A Coupled Minimax Approach
May 27, 2026Guang-Yuan Hao, Lars van der Laan, Aurélien Bibaut +1Offline Reinforcement Learning Value Functions
Cornell Tech, Cornell University · Netflix Research · Department of Statistics, University of Washington
BASIS: Batchwise Advantage Estimation from Single-Rollout Information Sharing for LLM Reasoning
May 26, 2026Shijin Gong, Erhan Xu, Kai Ye +3Value Functions Verifiable Rewards
School of Management, University of Science and Technology of China · Department of Statistics, London School of Economics and Political Science · Department of Engineering Science, University of Oxford
In LLM Reasoning, there is Irrationality on top of Value Misalignment
May 26, 2026Kejiang Qian, Fengxiang HeLarge Language Model Alignment LLM Reasoning Strategies
University of Edinburgh
Global Convergence of Wasserstein Policy Gradient for Entropy-Regularized Reinforcement Learning
May 25, 2026Zhaoyu Zhu, Rui Gao, Shuang LiEntropy Regularized Reinforcement Learning Policy Gradient
Shanghai Jiao Tong University · The University of Texas at Austin · The Chinese University of Hong Kong, Shenzhen
Global linear convergence of entropy-regularized softmax policy gradient beyond tabular MDPs
May 24, 2026Ziyue Chen, David Šiška, Lukasz SzpruchEntropy Regularized Reinforcement Learning Policy Gradient
On the Sample Complexity of Discounted Reinforcement Learning with Optimized Certainty Equivalents
May 20, 2026Oliver Mortensen, Mohammad Sadegh TalebiConditional-Value-At-Risk Value Functions
Department of Computer Science, University of Copenhagen.
Dynamic Shapley Computation
May 20, 2026Xuan Yang, Hsi-Wen Chen, Ming-Syan Chen +1Shapley Value Learning Dynamics
1Duke University · 2National Taiwan University
Stitched Value Model for Diffusion Alignment
May 19, 2026Hyojun Go, Hyungjin Chung, Prune Truong +8Diffusion Alignment Diffusion Models
1ETH Zurich · 2Google · University of Copenhagen
Learning Interpretable Point-Based Clinical Risk Scores via Direct Optimization
May 18, 2026Ying Cui, Albert M Li, Vivek Charu +3Risk Stratification Multimorbidity
Department of Biomedical Data Science, Stanford University · Decatur High School · Department of Pathology, Stanford University School of Medicine +1
TabQL: In-Context Q-Learning with Tabular Foundation Models
May 18, 2026Qisai Liu, Zhanhong Jiang, Timilehin Ayanlade +4Deep Q-Networks Q-Learning
Department of Mechanical Engineering Iowa State University Ames, IA 50011 · Translational AI Center Iowa State University Ames, IA 50011 · Department of Computer Science Iowa State University Ames, IA 50011 +1
Sign-Separated Asymmetric Finite-Time Error Analysis of Q-Learning
May 15, 2026Donghwan LeeQ-Learning Value Functions
Department of Electrical Engineering Korea Advanced Institute of Science and Technology (KAIST) Daejeon 34141, South Korea
Heuristic Pathologies and Further Variance Reduction via Uncertainty Propagation in the AIVAT Family of Techniques
May 14, 2026Juho Kim, Tuomas SandholmVariance Value Functions
Computer Science Department Carnegie Mellon University
Executable Agentic Memory for GUI Agent
May 12, 2026Zerui Qin, Sheng Yue, Xingyuan Hua +2Graphical User Interface Agents Agentic Memory
1Tsinghua University, China · 2Sun Yat-sen University, China
ACSAC: Adaptive Chunk Size Actor-Critic with Causal Transformer Q-Network
May 10, 2026Qian Chen, Junqiao Zhao, Hongtu Zhou +4Soft Actor-Critic Action Chunks
Tongji University
Drifting Field Policy: A One-Step Generative Policy via Wasserstein Gradient Flow
May 8, 2026Juil Koo, Mingue Park, Jiwon Choi +2Diffusion Policies Wasserstein Gradient Flows
KAIST
Q-MMR: Off-Policy Evaluation via Recursive Reweighting and Moment Matching
May 7, 2026Xiang Li, Nan JiangValue Functions Reweighting
Nanjing University · UIUC

Sep 30, 2026, cs.LGJ/K move · Enter open · S save

One-Step Generative Modeling via Training Dynamics Action

Zhangyong Liang, Ying Huang, Haibin Ling

Department of Artificial Intelligence, Westlake University · Hangzhou Normal University

Generative Models Student-Generated Trajectories Value Functions Data Generation

Open paper AskPDF arXiv

One-step generative models construct a static generator through iterative training-time transport. Existing transport objectives primarily assess distributional motion, although a neural generator needs to realize the requested sample displacements jointly through shared parameter updates. The training-time construction raises the question: \emph{once training becomes the iterative process that constructs the final one-step map, what to optimize: the next distributional move, or the route by which the finite generator learns the final map?} To address the question, we introduce \textbf{T}raining \textbf{D}ynamics \textbf{A}ction (\textbf{TDAction}), which selects transport targets according to local shared-parameter realization cost while retaining a prescribed level of distributional progress. We formulate the cost as a soft-terminal control problem and derive a closed-form Batch Tangent Action-to-Go value that accounts for parameter effort and terminal mismatch. The criterion captures cross-sample interactions omitted by independent pairwise costs; under isotropic mobility, the criterion agrees with quadratic Euclidean assignment for deterministic balanced couplings. Randomized tangent probes provide a low-rank implementation that constructs shared detached targets without adding an inference-time trajectory. Controlled studies examine the relationship between generator geometry, transport selection, and realized local action. On ImageNet

256\times256

, TDAction attains an FID below

1.1

without distillation.

Momentum

Related topics

Latest papers 78