Conservative Value Estimation

Conservative value estimation focuses on mitigating overestimation in machine learning models, particularly in reinforcement learning and related areas like automatic speech recognition, by producing lower-bound estimates of values or likelihoods. Current research emphasizes techniques like conservative Q-learning, density estimation, and data filtering to address the challenges of out-of-distribution data and improve robustness, often leveraging neural networks and Bayesian methods. This work is crucial for enhancing the reliability and safety of AI systems in applications ranging from robotics and autonomous driving to natural language processing, where overconfidence can lead to undesirable or even dangerous outcomes.

Papers

July 18, 2024

Robust ASR Error Correction with Conservative Data Filtering
Takuma Udagawa, Masayuki Suzuki, Masayasu Muraoka, Gakuto Kurata
Automatic Speech Recognition Error Correction Automatic Speech Recognition Hypothesis Automatic Speech Recognition Error Correction Conservative Value Estimation

June 6, 2024

Strategically Conservative Q-Learning
Yutaka Shimizu, Joey Hong, Sergey Levine, Masayoshi Tomizuka
Offline Reinforcement Learning Distribution Data Conservative Q Learning Conservative Value Estimation

January 16, 2024

Learning from Sparse Offline Datasets via Conservative Density Estimation
Zhepeng Cen, Zuxin Liu, Zitong Wang, Yihang Yao, Henry Lam, Ding Zhao
LeArning Abstract Offline Reinforcement Learning Sparse Reward Importance Sampling DIstribution Estimation Conservative Value Estimation Offline Dataset

September 22, 2023

Counterfactual Conservative Q Learning for Offline Multi-agent Reinforcement Learning
Jianzhun Shao, Yun Qu, Chen Chen, Hongchang Zhang, Xiangyang Ji
Multi Agent Reinforcement Learning Offline Multi Agent Reinforcement Learning Conservative Q Learning Conservative Value Estimation

August 7, 2023

Exploiting Generalization in Offline Reinforcement Learning via Unseen State Augmentations
Nirbhay Modhe, Qiaozi Gao, Ashwin Kalyan, Dhruv Batra, Govind Thattai, Gaurav Sukhatme
Offline Reinforcement Learning State Augmentation Conservative Value Estimation Knowledge Invariant Perturbation

July 20, 2023

Conservative Estimation of Perception Relevance of Dynamic Objects for Safe Trajectories in Automotive Scenarios
Ken Mori, Kai Storms, Steven Peters
Automated Driving Perception Aware Safe Trajectory Collision Scenario Perception Module Dynamic Object Relevance Ranking Conservative Value Estimation Perceptual Concept

April 21, 2023

Balancing Simulation-based Inference for Conservative Posteriors
Arnaud Delaunoy, Benjamin Kurt Miller, Patrick Forré, Christoph Weniger, Gilles Louppe
Simulation Based Inference Balancing Strategy Posterior Approximation Conservative Value Estimation Neural Posterior Estimation Neural Ratio Estimation

February 26, 2023

Kernel Conditional Moment Constraints for Confounding Robust Inference
Kei Ishikawa, Niao He
Unobserved Confounders Robust Inference Conservative Value Estimation Offline Contextual Bandit Marginal Sensitivity Model

February 14, 2023

Conservative State Value Estimation for Offline Reinforcement Learning
Liting Chen, Jie Yan, Zhengdao Shao, Lu Wang, Qingwei Lin, Saravan Rajmohan, Thomas Moscibroda, Dongmei Zhang
Offline Reinforcement Learning Conservative Q Learning Conservative Value Estimation Function Estimation

December 8, 2022

Confidence-Conditioned Value Functions for Offline Reinforcement Learning
Joey Hong, Aviral Kumar, Sergey Levine
Offline Reinforcement Learning Value Function Conservative Value Estimation

October 28, 2022

Conservative Likelihood Ratio Estimator for Infrequent Data Slightly above a Frequency Threshold
Masato Kikuchi, Yuhi Kusakabe, Tadachika Ozono
Context Information Estimation Task Context Aware Low Quality Conservative Value Estimation

October 7, 2022

Conservative Bayesian Model-Based Value Expansion for Offline Policy Optimization
Jihwan Jeong, Xiaoyu Wang, Michael Gimelfarb, Hyunwoo Kim, Baher Abdulhai, Scott Sanner
Offline Reinforcement Learning Model Based Offline Policy Distribution Free Conservative Value Estimation Value Expansion

September 27, 2022

DCE: Offline Reinforcement Learning With Double Conservative Estimates
Chen Zhao, Kai Xing Huang, Chun yuan
Offline Reinforcement Learning Traditional Reinforcement Learning Conservative Value Estimation

June 6, 2022

RORL: Robust Offline Reinforcement Learning via Conservative Smoothing
Rui Yang, Chenjia Bai, Xiaoteng Ma, Zhaoran Wang, Chongjie Zhang, Lei Han
Reinforcement Learning Offline Reinforcement Learning Offline Reinforcement Learning Algorithm Linear MDPs Robust Offline Reinforcement Learning Conservative Value Estimation

March 14, 2022

Conservative Filtering for Heterogeneous Decentralized Data Fusion in Dynamic Robotic Systems
Ofer Dagan, Nisar R. Ahmed
Multi Robot Bayesian Perspective Data Fusion Decentralized Multi Robot Conservative Value Estimation Fusion Algorithm