Trajectory Wise Reward

Trajectory-wise reward in reinforcement learning (RL) focuses on scenarios where an agent receives a single reward for an entire sequence of actions, rather than for individual steps. Current research emphasizes developing efficient algorithms that can effectively learn from this type of feedback, often employing model-based approaches, GFlowNet architectures, or methods that decompose the trajectory reward into per-step proxies. This research area is significant because it addresses the limitations of traditional RL in settings with delayed or episodic rewards, improving the applicability of RL to real-world problems where immediate feedback is unavailable or impractical.

Papers

October 19, 2024

Action abstractions for amortized sampling
Oussama Boussif, Léna Néhale Ezzine, Joseph D Viviano, Michał Koziarski, Moksh Jain, Nikolay Malkin, Emmanuel Bengio, Rim Assouel, Yoshua Bengio
Reinforcement Learning Open Sampling Generative Flow Network Entropy Regularized Trajectory Wise Reward Action Abstraction

October 1, 2024

Enhancing Solution Efficiency in Reinforcement Learning: Leveraging Sub-GFlowNet and Entropy Integration
Siyi He
Reinforcement Learning Markov Chain Monte Carlo Molecule Generation Traditional Reinforcement Learning Enhancing Efficiency Trajectory Wise Reward

August 16, 2024

Model-based RL as a Minimalist Approach to Horizon-Free and Second-Order Bounds
Zhiyong Wang, Dongruo Zhou, John C.S. Lui, Wen Sun
Markov Decision Process Model Based Reinforcement Learning Regret Analysis New Horizon Trajectory Wise Reward

June 30, 2024

Towards shutdownable agents via stochastic choice
Elliott Thornley, Alexander Roman, Christos Ziakas, Leyton Ho, Louis Thomson
Artificial Intelligence Agent Sophisticated Agent Trajectory Wise Reward Simple Agent Critical Agent Incomplete Preference Stochastic Decision

June 11, 2024

Reinforcement Learning from Human Feedback without Reward Inference: Model-Free Algorithm and Instance-Dependent Analysis
Qining Zhang, Honghao Wei, Lei Ying
Reinforcement Learning Human Feedback Episodic Markov Decision Process Model Free Algorithm Trajectory Wise Reward Policy Identification Reward Inference

May 13, 2024

Accelerating the Evolution of Personalized Automated Lane Change through Lesson Learning
Jia Hu, Mingyue Lei, Duo Li, Zhenning Li, Jaehyun, So, Haoran Wang
Specie Evolution Driver Assistance System Evolution Strategy Naturalistic Driving Instructional Design Trajectory Wise Reward

April 16, 2024

Offline Trajectory Generalization for Offline Reinforcement Learning
Ziqi Zhao, Zhaochun Ren, Liu Yang, Fajie Yuan, Pengjie Ren, Zhumin Chen, jun Ma, Xin Xin
Offline Reinforcement Learning Trajectory Wise Reward Trajectory Generalization

February 4, 2024

DiffStitch: Boosting Offline Reinforcement Learning with Diffusion-based Trajectory Stitching
Guanghe Li, Yixiang Shan, Zhengbang Zhu, Ting Long, Weinan Zhang
Offline Reinforcement Learning Offline Reinforcement Learning Algorithm Trajectory Wise Reward Trajectory Stitching

December 27, 2023

Adaptive trajectory-constrained exploration strategy for deep reinforcement learning
Guojian Wang, Faguo Wu, Xiao Zhang, Ning Guo, Zhiming Zheng
Deep Reinforcement Learning Multi Agent Reinforcement Learning Adaptive Importance Exploration Problem Trajectory Wise Reward Adaptive Exploration Constrained Exploration

December 17, 2023

Episodic Return Decomposition by Difference of Implicitly Assigned Sub-Trajectory Reward
Haoxin Lin, Hongqiu Wu, Jiaji Zhang, Yihao Sun, Junyin Ye, Yang Yu
Qualitative Difference Proxy Reward Trajectory Wise Reward Episodic Reward Return Decomposition

November 27, 2023

Reinforcement Learning from Diffusion Feedback: Q* for Image Search
Aboli Marathe
Reinforcement Learning Large Vision Language Model Image Search Robot Imitation Trajectory Wise Reward Diffusion Control

July 10, 2023

Large Language Models as General Pattern Machines
Suvir Mirchandani, Fei Xia, Pete Florence, Brian Ichter, Danny Driess, Montserrat Gonzalez Arenas, Kanishka Rao, Dorsa Sadigh, Andy Zeng
Pre Trained Large Language Model Sequence Model Probabilistic Context Free Grammar Token Sequence Trajectory Wise Reward

June 30, 2022

Watch and Match: Supercharging Imitation with Regularized Optimal Transport
Siddhant Haldar, Vaibhav Mathur, Denis Yarats, Lerrel Pinto
Imitation Learning Optimal Transport Inverse Reinforcement Learning Feature Imitation State of the Art Imitation Trajectory Wise Reward

June 13, 2022

Provably Efficient Offline Reinforcement Learning with Trajectory-Wise Reward
Tengyu Xu, Yue Wang, Shaofeng Zou, Yingbin Liang
Offline Reinforcement Learning Proxy Reward Trajectory Wise Reward rEward Decomposition Reward Redistribution

December 7, 2021

PTR-PPO: Proximal Policy Optimization with Prioritized Trajectory Replay
Xingxing Liang, Yang Ma, Yanghe Feng, Zhong Liu
Proximal Policy Optimization PPO Algorithm Trajectory Wise Reward Trajectory Preference Trajectory Replay

December 1, 2021

Robust and Adaptive Temporal-Difference Learning Using An Ensemble of Gaussian Processes
Qin Lu, Georgios B. Giannakis
Reinforcement Learning Gaussian Process Robust Version Diverse Ensemble Value Function Temporal Difference Learning Trajectory Wise Reward State Reward