Information Theoretic Reward

Information-theoretic reward design focuses on shaping reinforcement learning agents' behavior by maximizing the information gained about the environment or minimizing uncertainty. Current research emphasizes using mutual information as a reward signal, leading to the development of algorithms like adaptive particle filters and variational information bottleneck methods for improved efficiency and robustness, particularly in addressing reward hacking in RLHF and efficient decision-making under uncertainty. This approach offers significant potential for enhancing the performance and reliability of autonomous systems in various applications, including robotics, active learning, and resource-constrained environments, by enabling more informed and efficient decision-making.

Papers

March 4, 2024

ASPIRe: An Informative Trajectory Planner with Mutual Information Approximation for Target Search and Tracking
Kangjie Zhou, Pengying Wu, Yao Su, Han Gao, Ji Ma, Hangxin Liu, Chang Liu
Web Tracking Particle Filter Informative Path Planning Information Maximization Information Theoretic Reward

February 14, 2024

InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modeling
Yuchun Miao, Sen Zhang, Liang Ding, Rong Bao, Lefei Zhang, Dacheng Tao
Reward Model Reinforcement Learning From Human Feedback Reward Overoptimization Reward Hacking Reward Misspecification Information Theoretic Reward

December 8, 2023

Pruning Convolutional Filters via Reinforcement Learning with Entropy Minimization
Bogdan Musat, Razvan Andonie
Reinforcement Learning Neural Network Neural Network Pruning Structural Pruning Neural Network Layer Entropy Minimization Deep Convolutional Activation Feature Information Theoretic Reward

September 19, 2023

Measurement Simplification in \rho-POMDP with Performance Guarantees
Tom Yotam, Vadim Indelman
Decision Making Performance Guarantee Belief Distribution Active SLAM Information Theoretic Reward

November 30, 2022

Global Convergence of Localized Policy Iteration in Networked Multi-Agent Reinforcement Learning
Yizhou Zhang, Guannan Qu, Pan Xu, Yiheng Lin, Zaiwei Chen, Adam Wierman
Multi Agent Reinforcement Learning Optimal Policy Policy Iteration Global Convergence Information Theoretic Reward

January 14, 2022

Adaptive Information Belief Space Planning
Moran Barenboim, Vadim Indelman
Autonomous System Planning Algorithm Action Selection Informed Decision Belief Space Planning Information Theoretic Reward

Information Theoretic Reward

Papers

ASPIRe: An Informative Trajectory Planner with Mutual Information Approximation for Target Search and Tracking

InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modeling

Pruning Convolutional Filters via Reinforcement Learning with Entropy Minimization

Measurement Simplification in \rho-POMDP with Performance Guarantees

Global Convergence of Localized Policy Iteration in Networked Multi-Agent Reinforcement Learning

Adaptive Information Belief Space Planning