Value Function Estimation

Follow this topic Ask about this topic

Momentum

8 papers in the last four weeks, up 167% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 45

All topics

CLOVER: Closed-Loop Value Estimation and Ranking for End-to-End Autonomous Driving Planning
May 14, 2026Sining Ang, Yuguang Yang, Canyu Chen +1End-to-End Autonomous Driving Value Function Estimation
Department of Automation, University of Science and Technology of China · Institute for AI Industry Research, Tsinghua University · School of Electronic Information Engineering, Beihang University +1
Peng's Q( $λ$ ) for Conservative Value Estimation in Offline Reinforcement Learning
May 14, 2026Byeongchan Kim, Min-hwan OhReinforcement Learning Value Function Estimation
Seoul National University
Your Language Model is Its Own Critic: Reinforcement Learning with Value Estimation from Actor's Internal States
May 8, 2026Yunho Choi, Jongwon Lim, Woojin Ahn +3Value Function Estimation RL for Language Model Reasoning
Graduate School of Data Science Seoul National University · Computer Science and Engineering Seoul National University
Q-MMR: Off-Policy Evaluation via Recursive Reweighting and Moment Matching
May 7, 2026Xiang Li, Nan JiangOff-Policy Evaluation Value Function Estimation
Nanjing University · UIUC
First-Order Efficiency for Probabilistic Value Estimation via A Statistical Viewpoint
May 4, 2026Ziqi Liu, Kiljae Lee, Yuan Zhang +1Value Function Estimation Shapley Value Attribution
Department of Statistics & Data Science, Carnegie Mellon University. · Department of Statistics, The Ohio State University.
Kernelized Advantage Estimation: From Nonparametric Statistics to LLM Reasoning
Apr 30, 2026Shijin Gong, Kai Ye, Jin Zhu +3Value Function Estimation RL for Language Model Reasoning
School of Management, University of Science and Technology of China · Department of Statistics, London School of Economics and Political Science · School of Mathematics, University of Birmingham +2
Reward Models Are Secretly Value Functions: Temporally Coherent Reward Modeling
Apr 24, 2026Alex NikulkovReward Modeling Value Function Estimation
AI at Meta
Planning in entropy-regularized Markov decision processes and games
Apr 21, 2026Jean-Bastien Grill, Omar Darwiche Domingues, Pierre Ménard +2Value Function Estimation Markov Models
DeepMind Paris · SequeL team, Inria Lille
Beyond Bellman: High-Order Generator Regression for Continuous-Time Policy Evaluation
Apr 21, 2026Yaowei Zheng, Richong Zhang, Shenxi Wu +5Reinforcement Learning Value Function Estimation
School of Computer Science and Engineering, Beihang University, China · Fudan University, China · Faculty of Humanities and Social Sciences, University of Nottingham, China +1
Provably Efficient Offline-to-Online Value Adaptation with General Function Approximation
Apr 15, 2026Shangzhe Li, Weitong ZhangOffline-To-Online RL Sample-Efficient RL
UNC Chapel Hill
Fitted Q-Evaluation without Bellman Completeness via Occupancy Weighting
Dec 29, 2025Lars van der Laan, Nathan KallusOff-Policy Evaluation Value Function Estimation
Department of Statistics, University of Washington · Netflix · Cornell University
A Robust Model-Based Approach for Continuous-Time Policy Evaluation with Unknown Lévy Process Dynamics
Apr 2, 2025Qihao Ye, Xiaochuan Tian, Yuhua ZhuReinforcement Learning Value Function Estimation
Statistical Inference for Policy Evaluation with Temporal Difference Learning
Oct 21, 2024Weichen Wu, Gen Li, Yuting Wei +1Parameter Estimation Confidence Region Estimation
The Voleon Group, New York, NY, USA · Department of Statistics, The Chinese University of Hong Kong, Hong Kong SAR. · Department of Statistics and Data Science, The Wharton School, University of Pennsylvania, Philadelphia, PA, USA. +1
Decision-Centered Abstractions via Orthogonal Estimation of Difference-of-Q Functions
Jun 12, 2024Defu Cao, Angela ZhouValue Function Estimation State Abstraction
University of Southern California
Bringing Value Models Back: Generative Critics for Value Modeling in LLM Reinforcement Learning
Date pendingZikang Shan, Han Zhong, Liwei Wang +1RL for Language Models Reinforcement Learning
Peking University · Microsoft Research Asia · Shanghai Jiao Tong University

May 14, 2026, cs.ROJ/K move · Enter open · S save

CLOVER: Closed-Loop Value Estimation and Ranking for End-to-End Autonomous Driving Planning

Sining Ang, Yuguang Yang, Canyu Chen, Yan Wang

Department of Automation, University of Science and Technology of China · Institute for AI Industry Research, Tsinghua University · School of Electronic Information Engineering, Beihang University+1

End-to-End Autonomous Driving Value Function Estimation Autonomous Driving Planning

Open paper AskPDF arXiv

End-to-end autonomous driving planners are commonly trained by imitating a single logged trajectory, yet evaluated by rule-based planning metrics that measure safety, feasibility, progress, and comfort. This creates a training--evaluation mismatch: trajectories close to the logged path may violate planning rules, while alternatives farther from the demonstration can remain valid and high-scoring. The mismatch is especially limiting for proposal-selection planners, whose performance depends on candidate-set coverage and scorer ranking quality. We propose CLOVER, a Closed-LOop Value Estimation and Ranking framework for end-to-end autonomous driving planning. CLOVER follows a lightweight generator--scorer formulation: a generator produces diverse candidate trajectories, and a scorer predicts planning-metric sub-scores to rank them at inference time. To expand proposal support beyond single-trajectory imitation, CLOVER constructs evaluator-filtered pseudo-expert trajectories and trains the generator with set-level coverage supervision. It then performs conservative closed-loop self-distillation: the scorer is fitted to true evaluator sub-scores on generated proposals, while the generator is refined toward teacher-selected top-

k

and vector-Pareto targets with stability regularization. We analyze when an imperfect scorer can improve the generator, showing that scorer-mediated refinement is reliable when scorer-selected targets are enriched under the true evaluator and updates remain conservative. On NAVSIM, CLOVER achieves 94.5 PDMS and 90.4 EPDMS, establishing a new state of the art. On the more challenging NavHard split, it obtains 48.3 EPDMS, matching the strongest reported result. On supplementary nuScenes open-loop evaluation, CLOVER achieves the lowest L2 error and collision rate among compared methods. Code data will be released at https://github.com/WilliamXuanYu/CLOVER.

Momentum

Related topics

Latest papers 45