Curriculum Reinforcement Learning

Curriculum reinforcement learning (CRL) aims to accelerate the training of reinforcement learning agents by structuring the learning process, progressing from simpler to more complex tasks. Current research focuses on automating curriculum generation, often employing techniques like optimal transport, goal-conditional policies, and various model architectures (e.g., mixture of experts, quantized world models) to efficiently explore the task space and improve sample efficiency. This approach holds significant promise for tackling complex real-world problems, particularly in robotics and autonomous systems, where data acquisition is expensive and traditional reinforcement learning methods often struggle with sample complexity. The development of robust and adaptable CRL methods is driving progress in diverse fields, including autonomous driving, robotic manipulation, and quantum computing.

Papers

November 4, 2024

WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning
Zehan Qi, Xiao Liu, Iat Long Iong, Hanyu Lai, Xueqiao Sun, Xinyue Yang, Jiadai Sun, Yu Yang, Shuntian Yao, Tianjie Zhang, Wei Xu, Jie Tang, Yuxiao Dong
Large Language Model Reinforcement Learning LLM Based Curriculum Reinforcement Learning Online Training Web Automation Evolutionary Curriculum

October 22, 2024

Sample-Efficient Curriculum Reinforcement Learning for Complex Reward Functions
Kilian Freitag, Kristian Ceder, Rita Laezza, Knut Åkesson, Morteza Haghir Chehreghani
Reinforcement Learning Reward Function Simple Function Curriculum Reinforcement Learning Scalar Reward

September 10, 2024

DemoStart: Demonstration-led auto-curriculum applied to sim-to-real with multi-fingered robots
Maria Bauza, Jose Enrique Chen, Valentin Dalibard, Nimrod Gileadi, Roland Hafner, Murilo F. Martins, Joss Moore, Rugile Pevceviciute, Antoine Laurens, Dushyant Rao, Martina Zambelli, Martin Riedmiller, Jon Scholz, Konstantinos Bousmalis, Francesco Nori, Nicolas Heess
Robotic Hand Real Robot Curriculum Reinforcement Learning Behavior Generation Curriculum Generation Complex Manipulation

May 6, 2024

Reverse Forward Curriculum Learning for Extreme Sample and Demonstration Efficiency in Reinforcement Learning
Stone Tao, Arth Shukla, Tse-kai Chan, Hao Su
Reinforcement Learning Evidence Piece Curriculum Reinforcement Learning Demonstration Data Learning From Demonstration Sample Quality

March 20, 2024

Reward-Driven Automated Curriculum Learning for Interaction-Aware Self-Driving at Unsignalized Intersections
Zengqi Peng, Xiao Zhou, Lei Zheng, Yubin Wang, Jun Ma
Based Policy Curriculum Reinforcement Learning Unsignalized Intersection Interaction Aware

March 11, 2024

Acquiring Diverse Skills using Curriculum Reinforcement Learning with Mixture of Experts
Onur Celik, Aleksandar Taranovic, Gerhard Neumann
Reinforcement Learning Mixture Component Expert Knowledge Curriculum Reinforcement Learning Diverse Skill Policy Parametrization Context Distribution

February 8, 2024

Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning
Zhiheng Xi, Wenxiang Chen, Boyang Hong, Senjie Jin, Rui Zheng, Wei He, Yiwen Ding, Shichun Liu, Xin Guo, Junzhe Wang, Honglin Guo, Wei Shen, Xiaoran Fan, Yuhao Zhou, Shihan Dou, Xiao Wang, Xinbo Zhang, Peng Sun, Tao Gui, Qi Zhang, Xuanjing Huang
Complex Reasoning Reasoning Task Curriculum Reinforcement Learning Outcome Supervision Program Based Reasoning

February 5, 2024

Curriculum reinforcement learning for quantum architecture search under hardware errors
Yash J. Patel, Akash Kundu, Mateusz Ostaszewski, Xavier Bonet-Monroig, Vedran Dunjko, Onur Danaci
Variational Quantum Algorithm Intermediate Scale Quantum Curriculum Reinforcement Learning Circuit Design Noisy Quantum Hardware Error Quantum Architecture Search

January 29, 2024

Curriculum-Based Reinforcement Learning for Quadrupedal Jumping: A Reference-free Design
Vassil Atanassov, Jiatao Ding, Jens Kober, Ioannis Havoutis, Cosimo Della Santina
Deep Reinforcement Learning Curriculum Reinforcement Learning Reference Free Continuous Jumping Quadrupedal Jumping Omnidirectional Jumping

October 30, 2023

October 26, 2023

CQM: Curriculum Reinforcement Learning with a Quantized World Model
Seungjae Lee, Daesol Cho, Jonghae Park, H. Jin Kim
World Model Category Theory Curriculum Reinforcement Learning Aware Curriculum Goal Space

October 24, 2023

Learning Agility and Adaptive Legged Locomotion via Curricular Hindsight Reinforcement Learning
Sicen Li, Yiming Pang, Panju Bai, Zhaojin Liu, Jiawei Li, Shihao Hu, Liquan Wang, Gang Wang
Robust Locomotion Curriculum Reinforcement Learning Agile Maneuver Hindsight Experience Replay Legged System

October 7, 2023

Accelerate Multi-Agent Reinforcement Learning in Zero-Sum Games with Subgame Curriculum Learning
Jiayu Chen, Zelai Xu, Yunfei Li, Chao Yu, Jiaming Song, Huazhong Yang, Fei Fang, Yu Wang, Yi Wu
Multi Agent Reinforcement Learning Nash Equilibrium Zero Sum Game Curriculum Reinforcement Learning

September 25, 2023

September 21, 2023

Curriculum Reinforcement Learning via Morphology-Environment Co-Evolution
Shuang Ao, Tianyi Zhou, Guodong Long, Xuan Song, Jing Jiang
Learning Dynamic Curriculum Reinforcement Learning Co Evolution Morphology Environment Co Evolution SOTA Segmentation Current Reinforcement Learning

August 31, 2023

Curriculum Proximal Policy Optimization with Stage-Decaying Clipping for Self-Driving at Unsignalized Intersections
Zengqi Peng, Xiao Zhou, Yubin Wang, Lei Zheng, Ming Liu, Jun Ma
Autonomous Driving Proximal Policy Optimization Intersection Scenario Curriculum Reinforcement Learning Unsignalized Intersection

August 25, 2023

Towards Optimal Head-to-head Autonomous Racing with Curriculum Reinforcement Learning
Dvij Kalaria, Qin Lin, John M. Dolan
Reinforcement Learning Autonomous Racing Curriculum Reinforcement Learning Safe Reinforcement Learning Algorithm Head to Head

July 12, 2023

Maneuver Decision-Making Through Automatic Curriculum Reinforcement Learning Without Handcrafted Reward functions
Zhang Hong-Peng
Reinforcement Learning Reward Function Curriculum Reinforcement Learning Maneuver Decision Unmanned Combat

Curriculum Reinforcement Learning

Papers

WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning

Sample-Efficient Curriculum Reinforcement Learning for Complex Reward Functions

DemoStart: Demonstration-led auto-curriculum applied to sim-to-real with multi-fingered robots

Reverse Forward Curriculum Learning for Extreme Sample and Demonstration Efficiency in Reinforcement Learning

Reward-Driven Automated Curriculum Learning for Interaction-Aware Self-Driving at Unsignalized Intersections

Acquiring Diverse Skills using Curriculum Reinforcement Learning with Mixture of Experts

Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning

Curriculum reinforcement learning for quantum architecture search under hardware errors

Curriculum-Based Reinforcement Learning for Quadrupedal Jumping: A Reference-free Design

Variational Curriculum Reinforcement Learning for Unsupervised Discovery of Skills

Diversify & Conquer: Outcome-directed Curriculum RL via Out-of-Distribution Disagreement

CQM: Curriculum Reinforcement Learning with a Quantized World Model

Learning Agility and Adaptive Legged Locomotion via Curricular Hindsight Reinforcement Learning

Accelerate Multi-Agent Reinforcement Learning in Zero-Sum Games with Subgame Curriculum Learning

Tracking Control for a Spherical Pendulum via Curriculum Reinforcement Learning

On the Benefit of Optimal Transport for Curriculum Reinforcement Learning

Curriculum Reinforcement Learning via Morphology-Environment Co-Evolution

Curriculum Proximal Policy Optimization with Stage-Decaying Clipping for Self-Driving at Unsignalized Intersections

Towards Optimal Head-to-head Autonomous Racing with Curriculum Reinforcement Learning

Maneuver Decision-Making Through Automatic Curriculum Reinforcement Learning Without Handcrafted Reward functions