First Order Policy

First-order policy methods in reinforcement learning aim to efficiently optimize policies by leveraging gradient information, offering improved sample efficiency compared to model-free approaches. Current research focuses on enhancing these methods' robustness and scalability, particularly for complex tasks involving contact dynamics, high-dimensional action spaces, and multi-agent interactions, often employing techniques like residual policy learning and world models. These advancements are significant for accelerating the training of robots and other autonomous systems, enabling more efficient and robust control in diverse environments.

Papers

October 4, 2024

Residual Policy Learning for Perceptive Quadruped Control Using Differentiable Simulation
Jing Yuan Luo, Yunlong Song, Victor Klemm, Fan Shi, Davide Scaramuzza, Marco Hutter
Policy Gradient Model Free Reinforcement Learning Differentiable Simulation Residual Reinforcement Learning Perceptive Locomotion First Order Policy

July 2, 2024

PWM: Policy Learning with Large World Models
Ignat Georgiev, Varun Giridhar, Nicklas Hansen, Animesh Garg
Reinforcement Learning Large Model First Order Policy

July 29, 2023

First-order Policy Optimization for Robust Policy Evaluation
Yan Li, Guanghui Lan
State Action Value First Order Policy Robust Policy Evaluation Frank Wolfe Policy Optimization

May 18, 2023

Deep Metric Tensor Regularized Policy Gradient
Gang Chen, Victoria Huang
Policy Gradient First Order Policy

September 21, 2022

First-order Policy Optimization for Robust Markov Decision Process
Yan Li, Guanghui Lan, Tuo Zhao
Robust Policy Policy Mirror Descent First Order Policy Robust Markov Decision

June 9, 2022

Value Memory Graph: A Graph-Structured World Model for Offline Reinforcement Learning
Deyao Zhu, Li Erran Li, Mohamed Elhoseiny
Reinforcement Learning Offline Reinforcement Learning State Action Space Graph Memory First Order Policy

January 29, 2022

Zeroth-Order Actor-Critic
Yuheng Lei, Jianyu Chen, Shengbo Eben Li, Sifa Zheng
Actor Critic Algorithm Policy Improvement Zeroth Order Optimization First Order Policy

November 22, 2021

Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectification
Ling Pan, Longbo Huang, Tengyu Ma, Huazhe Xu
Offline Reinforcement Learning Offline Reinforcement Learning Algorithm Offline Multi Agent Reinforcement Learning Rectification Network First Order Policy