Policy Sampling

Policy sampling in reinforcement learning and related fields focuses on optimizing the selection of data used to train and evaluate policies, aiming to improve efficiency and robustness. Current research emphasizes the development of adaptive sampling strategies, including those driven by learned models or guided by theoretical bounds, to address challenges like distributional shift and high sample complexity. This work is crucial for advancing the performance and scalability of algorithms in diverse applications, from robotics and AI alignment to resource allocation and recommender systems, where efficient and reliable policy learning is paramount. The ultimate goal is to develop methods that minimize the amount of data needed while maximizing the accuracy and stability of learned policies.

Papers

May 21, 2024

Efficient Imitation Learning with Conservative World Models
Victor Kolev, Rafael Rafailov, Kyle Hatch, Jiajun Wu, Chelsea Finn
Imitation Learning Policy Learning Adversarial Imitation Policy Sampling

May 14, 2024

Understanding the performance gap between online and offline alignment algorithms
Yunhao Tang, Daniel Zhaohan Guo, Zeyu Zheng, Daniele Calandriello, Yuan Cao, Eugene Tarassov, Rémi Munos, Bernardo Ávila Pires, Michal Valko, Yong Cheng, Will Dabney
Reinforcement Learning AI Alignment Offline Policy Large Language Model Alignment Performance Gap Policy Sampling Online Alignment

April 22, 2024

Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Data
Fahim Tajwar, Anikait Singh, Archit Sharma, Rafael Rafailov, Jeff Schneider, Tengyang Xie, Stefano Ermon, Chelsea Finn, Aviral Kumar
Contrastive Learning Fine Tuning Medical LLM Preference Fine Tuning Policy Sampling Policy Data

January 8, 2024

Adaptive Experimental Design for Policy Learning
Masahiro Kato, Kyohei Okumura, Takuya Ishihara, Toru Kitagawa
Policy Learning Adaptive Sampling Adaptive Experimental Design Policy Sampling Target Identification

November 23, 2023

Learning Optimal and Fair Policies for Online Allocation of Scarce Societal Resources from Data Collected in Deployment
Bill Tang, Çağıl Koçyiğit, Eric Rice, Phebe Vayanos
Data Driven Data Collection Resource Constrained Capital Allocation Optimal Learning Online Allocation Policy Sampling

November 14, 2023

On-Policy Policy Gradient Reinforcement Learning Without On-Policy Sampling
Nicholas E. Corrado, Josiah P. Hanna
Policy Gradient Policy Reinforcement Learning Policy Sampling

February 24, 2023

Why Target Networks Stabilise Temporal Difference Methods
Mattie Fellows, Matthew J. A. Smith, Shimon Whiteson
Deep Reinforcement Learning Markov Decision Process Temporal Difference Learning Temporal Difference Policy Sampling Temporal Difference Algorithm

November 3, 2022

Active CT Reconstruction with a Learned Sampling Policy
Ce Wang, Kun Shang, Haimiao Zhang, Shang Zhao, Dong Liang, S. Kevin Zhou
Computed Tomography Computed Tomography Reconstruction High Quality Reconstruction Region of Interest Uniform Sampling Limited Angle Policy Sampling

August 5, 2022

An Approximate Policy Iteration Viewpoint of Actor-Critic Algorithms
Zaiwei Chen, Siva Theja Maguluri
Actor Critic Algorithm Policy Iteration Natural Policy Gradient Reinforcement Learning Problem Policy Sampling Approximate Policy Iteration Policy Based Algorithm

February 7, 2022

Bayesian Non-stationary Linear Bandits for Large-Scale Recommender Systems
Saeed Ghoorchian, Evgenii Kortukov, Setareh Maghsudi
Recommender System Linear Bandit Non Stationary Linear Contextual Bandit Large Scale Recommender System Policy Sampling

January 15, 2022

Block Policy Mirror Descent
Guanghui Lan, Yan Li, Tuo Zhao
Reinforcement Learning Policy Gradient Policy OpTimization Policy Sampling Policy Mirror

December 29, 2021

GPS: A Policy-driven Sampling Approach for Graph Representation Learning
Tiehua Zhang, Yuze Liu, Xin Chen, Xiaowei Huang, Feng Zhu, Xi Zheng
Graph Representation Learning Graph Classification Graph Based Graph Data Adaptive Graph Policy Sampling

November 29, 2021

Robust On-Policy Sampling for Data-Efficient Policy Evaluation in Reinforcement Learning
Rujie Zhong, Duohan Zhang, Lukas Schäfer, Stefano V. Albrecht, Josiah P. Hanna
Reinforcement Learning Policy Evaluation Policy Distribution Policy Sampling