Trajectory Preference

Trajectory preference research focuses on aligning reinforcement learning (RL) agents' behavior with human preferences, expressed through comparisons of entire agent trajectories rather than instantaneous rewards. Current research emphasizes efficient exploration and learning from limited human feedback, employing methods like dynamic policy fusion, preference-guided policy optimization, and inverse reinforcement learning to infer underlying reward functions from trajectory preferences. This field is crucial for developing safe and user-friendly RL agents in complex applications, such as autonomous driving and robotics, where designing explicit reward functions is difficult or impossible.

Papers

September 30, 2024

Personalisation via Dynamic Policy Fusion
Ajsal Shereef Palattuparambil, Thommen George Karimpanal, Santu Rana
Deep Reinforcement Learning Human Feedback Reward Function Personalization Performance Trajectory Preference Policy Blending

July 9, 2024

Preference-Guided Reinforcement Learning for Efficient Exploration
Guojian Wang, Faguo Wu, Xiao Zhang, Tianyuan Chen, Xuyang Chen, Lin Zhao
Reinforcement Learning Efficient Exploration Preference Based Reinforcement Learning Trust Region Policy Optimization Trajectory Preference

June 28, 2024

Safety through feedback in Constrained RL
Shashank Reddy Chirra, Pradeep Varakantham, Praveen Paruchuri
Human Feedback Human SAFETY Value Function Feedback System Trajectory Preference

May 30, 2024

LAGMA: LAtent Goal-guided Multi-Agent Reinforcement Learning
Hyungho Na, Il-chul Moon
Multi Agent Reinforcement Learning Trajectory Generation Intrinsic Reward Intermediate Latent Trajectory Preference

November 25, 2023

Automated Lane Merging via Game Theory and Branch Model Predictive Control
Luyao Zhang, Shaohang Han, Sergio Grammatico
Game Theory Motion Planning Framework Dense Traffic Trajectory Preference Lane Merging

July 29, 2023

Human-Like Implicit Intention Expression for Autonomous Driving Motion Planning: A Method Based on Learning Human Intention Priors
Jiaqi Liu, Xiao Qi, Ying Ni, Jian Sun, Peng Hang
Autonomous Vehicle Human Driven Vehicle Human Interaction Trajectory Preference Learning Based Trajectory Planning

January 25, 2023

An Incremental Inverse Reinforcement Learning Approach for Motion Planning with Separated Path and Velocity Preferences
Armin Avaei, Linda van der Spaa, Luka Peternel, Jens Kober
Motion Planning Trajectory Planning Robotic Manipulator Temporal Trajectory Trajectory Preference Object Transportation

May 23, 2022

Human-in-the-loop: Provably Efficient Preference-based Reinforcement Learning with General Function Approximation
Xiaoyu Chen, Han Zhong, Zhuoran Yang, Zhaoran Wang, Liwei Wang
Optimal Policy Human in the Loop Preference Based Reinforcement Learning Human in the Loop Reinforcement Trajectory Preference

December 7, 2021

PTR-PPO: Proximal Policy Optimization with Prioritized Trajectory Replay
Xingxing Liang, Yang Ma, Yanghe Feng, Zhong Liu
Proximal Policy Optimization PPO Algorithm Trajectory Wise Reward Trajectory Preference Trajectory Replay

November 8, 2021

Dueling RL: Reinforcement Learning with Trajectory Preferences
Aldo Pacchiano, Aadirupa Saha, Jonathan Lee
Reinforcement Learning Regret Guarantee Human Like RL Traditional Reinforcement Learning Non Markovian Reward Trajectory Preference