Value Function Estimation

Follow this topic Ask about this topic

Momentum

8 papers in the last four weeks, up 167% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 45

All topics

Evaluation of Active Feature Acquisition Policies with Tabular Foundation Models
Oct 5, 2026Yuta Kobayashi, Divyam Madaan, Shalmali JoshiOff-Policy Evaluation Value Function Estimation
Department of Biomedical Informatics, Columbia University
Privy to the Foil: Recasting Value Estimation with a Self-Privileged Critic for RLVR
Sep 29, 2026Kun Liang, Chenming Tang, Clive Bai +5Value Function Estimation RL for Language Model Reasoning
School of Computer Science, Peking University · National Key Laboratory for Multimedia Information Processing, Peking University · Foundation Model Department, Tencent
Unlocking the Critic: Reward-Free Policy Optimization for LLM Post-Training
Sep 29, 2026Hongyang Li, Xiao Li, Caesar Wu +3Reinforcement Learning Value Function Estimation
University of Luxembourg · Seafill Open-Source Community · Université Paris-Saclay
Deep Epistemic Value Functions for Optimistic Exploration
Sep 28, 2026Leander Diaz-Bone, Marco Bagatella, Jonas Hübotter +1Value Function Estimation RL Exploration
ETH Zürich, Switzerland · Max Planck Institute for Intelligent Systems, Germany
Deep Weighted Bellman Residual Minimization for $Q^*$ Estimation
Sep 28, 2026Lican Kang, Jerry Zhijian Yang, Cheng Yuan +1Off-Policy Evaluation Value Function Estimation
Institute for Math and AI, Wuhan University, Wuhan, 430072, China · Hubei Key Laboratory of Computational Science, Wuhan University, Wuhan, 430072, China · School of Artificial Intelligence, Wuhan University, Wuhan, 430072, China +1
Limiting-Kernel Q( $λ$ ): Bridging Short and Long Horizons
Sep 23, 2026Tolga Ok, Arman Sharifi Kolarijani, Peyman Mohajerin Esfahani +1Value Function Estimation Policy Evaluation
Delft University of Technology, The Netherlands · Alpha Brain Technologies, The Netherlands · University of Toronto, Canada
Rethinking Critic Learning in PPO: Understanding and Mitigating Value Flattening
Sep 16, 2026Yizhuo Li, Jianhao Yan, Yun Luo +9RL for Language Models Value Function Estimation
Shanghai Jiao Tong University · Shanghai AI Laboratory · Westlake University +4
Optimal Value Inference for Reinforcement Learning
Sep 9, 2026Nan Lu, Ethan Lee, James M. Robins +2Reinforcement Learning Value Function Estimation
The Surprising Effectiveness of Approximate Value Iteration in Self-Play
Sep 8, 2026Raphael Boige, Amine Boumaza, Bruno ScherrerGame-Playing Agents Self-Play RL
Université de Lorraine, CNRS, Inria, LORIA, F-54000 Nancy, France
Recursive Value Learning for Long-Horizon Offline Goal-Conditioned RL
Sep 2, 2026Hyeonseong Jeon, Youngwoon LeeReinforcement Learning Offline RL
1Yonsei University · 2Seoul National University
Graph4BiLO: Graph Neural Network Approximation for Bilevel Mixed-Integer Linear Optimization
Aug 31, 2026Jessica D. Elrefaei, Kaixun Hua, Seungbae Kim +2Graph Neural Networks Value Function Estimation
Game-Agnostic Value Functions through Automatic JSON Feature Extraction
Aug 30, 2026Dien Nguyen, Diego Perez-LiebanaGame-Playing Agents Value Function Estimation
Queen Mary University of London - Game AI Group
Revisiting TD Target Aggregation under Uncertainty in Q-Learning
Aug 4, 2026Lipeng Zu, Xiaonan ZhangDeep Q-Learning Value Function Estimation
Computer Science Department, Florida State University, Tallahassee, Florida, USA
ValueFormer: A Causal Transformer Value Function with Stage-Aware Labels for Semi-Autonomous Vision-Language-Action Policies
Aug 3, 2026Inkyu Sa, Konstantin Stulov, Rajat BhageriaReward Shaping Value Function Estimation
Chef Robotics
Diffusion Policy with Behavioral Advantage Correction for Offline Reinforcement Learning
Aug 3, 2026Botao Dong, Longyang Huang, Ning Pang +1Distribution Shift Value Function Estimation
Botao Dong is with the School of Chemistry, Chemical Engineering and Biotechnology, Nanyang Technological University, Singapore 637459
General Value Functions for Remaining Useful Life and Failure-Mode Prediction
Jul 24, 2026Hao Yan, Ali Sarabi, Qing Zou +1RUL Estimation Value Function Estimation
School of Computing and Augmented Intelligence, Arizona State University
Relative Value Learning
Jul 23, 2026Marc Höftmann, Jan Robine, Stefan HarmelingValue Function Estimation Temporal-Difference Learning
Department of Computer Science, Technical University of Dortmund, Germany · Lamarr Institute for Machine Learning and Artificial Intelligence
Generalized Kalman filter based temporal difference reinforcement learning
Jul 22, 2026Vasos Arnaoutis, Eric Lutters, Bojana RosićRL Control Reinforcement Learning
Applied Mechanics and Data Analysis, Faculty of Engineering Technology, University of Twente, Enschede, the Netherlands · Department of Design, Production and Management, Faculty of Engineering Technology, University of Twente, Enschede, the Netherlands · Digital Engineering research group, Faculty of Mechanical and Industrial Engineering, Vienna University of Technology, Vienna, Austria
Robo-ValueRL: Reliable Value Estimation for Offline-to-Online Reinforcement Learning
Jul 10, 2026Wenke Xia, Pei Ren, Wenbo Yu +10Value Function Estimation Robotic Manipulation
1Gaoling School of Artificial Intelligence, Renmin University of China · 2Beijing Innovation Center of Humanoid Robotics · 3Beijing Forestry University +4
Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models
Jul 4, 2026Xinyi Xie, Zican Hu, Zhanyu Liu +7Value Function Estimation Test-Time Scaling
Nanjing University · Australian National University · Institute of Automation, Chinese Academy of Sciences +1
Accelerating Q-learning through Efficient Value-Sharing across Actions
Jun 29, 2026Prabhat Nagarajan, Brett Daley, Martha White +1Deep Q-Learning Value Function Estimation
Department of Computing Science, University of Alberta, Edmonton, AB, Canada. · Alberta Machine Intelligence Institute · Meta (independent work, not on Meta’s behalf). +1
Beyond Monotonic Progress: Retry-Supervised Value Learning for Robot Imitation
Jun 23, 2026Xinyao Qin, Junjie Lu, Kaixin Wang +7Value Function Estimation Imitation Learning
Tsinghua University · Work done at Microsoft Research Asia · University of Technology Sydney +3
Hessian-augmented Supervised Learning for Hamilton-Jacobi-Bellman PDEs
Jun 22, 2026Matías Gómez-Aedo, Behzad Azmi, Yuyang Huang +2Value Function Estimation PDE Solving
Department of Mathematics, Imperial College London, South Kensington Campus, SW7 2AZ London, UK · Department of Mathematics and Statistics, University of Konstanz, Germany · RICAM and Institute of Mathematics and Scientific Computing, University of Graz, Austria
Horizon Adaptive Offline Policy Learning via Value Stitching
Jun 19, 2026Kexin Zheng, Xianyuan Zhan, Xintao YanValue Function Estimation Offline RL
The University of Hong Kong · Tsinghua University
On the Variance of Temporal Difference Learning and its Reduction Using Control Variates
Jun 18, 2026Hsiao-Ru Pan, Bernhard SchölkopfValue Function Estimation Temporal-Difference Learning
Max Planck Institute for Intelligent Systems, Tübingen · ELLIS Institute Tübingen · ETH Zürich
Feat2Go: Visual Feature-Grounded Value Estimation for Embodied Reinforcement Learning
May 29, 2026Junyang Shu, Zhiwei Lin, Bingqing Wei +1Reward Shaping Value Function Estimation
Wangxuan Institute of Computer Technology, Peking University, China
Hista and Numca: Estimate State Value Effectively for LLM Reinforcement Learning
May 28, 2026Zizhe Chen, Jiqian Dong, Yizhou Tian +4RL for Language Models Value Function Estimation
Department of Computer Science and Engineering, The Chinese University of Hong Kong · Huawei Technologies Ltd.
What Frozen VLAs Already Know About Success: A Probing Study of Value-Like Structure in Foundation Robot Policies
May 27, 2026Jiachen Zhang, Junnan Nie, Junyi Lao +4Value Function Estimation Robotic Manipulation
Peking University · China Agricultural University · 1Peking University. +1
Generative Modeling by Value-Driven Transport
May 21, 2026Pablo Moreno-Muñoz, Adrian Müller, Gergely NeuValue Function Estimation Stochastic Optimal Control
Universitat Pompeu Fabra · Barcelona, Spain · ETH Zürich +2
OPPO: Bayesian Value Recursion for Token-Level Credit Assignment in LLM Reasoning
May 21, 2026Yu Li, Rui Miao, Tian Lan +1Value Function Estimation Token-Level Credit Assignment
1George Washington University · 2The University of Texas at Dallas

Oct 5, 2026, cs.LGJ/K move · Enter open · S save

Evaluation of Active Feature Acquisition Policies with Tabular Foundation Models

Yuta Kobayashi, Divyam Madaan, Shalmali Joshi

Department of Biomedical Informatics, Columbia University

Off-Policy Evaluation Value Function Estimation Active Learning Active Feature Acquisition Epistemic Uncertainty

Open paper AskPDF arXiv

Active feature acquisition learns policies that sequentially acquire features to maximize information about a target variable. We study how to learn and evaluate such policies from finite offline data using prior-data fitted networks (PFNs), which are off-the-shelf models that output posterior predictive distributions without task-specific training. We show that under the imbalanced coverage of offline data, using total predictive entropy as a reward creates an epistemic bias that penalizes acquiring sparsely observed features. Specifically, this reward conflates epistemic uncertainty (arising from lack of offline data) with aleatoric uncertainty (arising from uninformative features). To address this, we target the posterior expected (aleatoric) entropy instead of the total predictive entropy output by a PFN for evaluating feature acquisitions. Empirical evaluations on synthetic and real-world datasets demonstrate that our approach consistently reduces value estimation bias and yields credible intervals with strong empirical coverage, which can translate to improved downstream policy selection.

Momentum

Related topics

Latest papers 45