Curriculum RL

RL: Reinforcement Learning

Curriculum reinforcement learning studies how to arrange or adapt training experiences so an agent can master tasks that are difficult to learn all at once. Methods include staged task schedules, self-paced sampling based on success or learning progress, and adaptive curricula that generate challenges through teachers or adversarial self-play; some also scaffold exploration or change reward objectives over time. These approaches are used both in language-model post-training for reasoning, coding, and tool use, and in robotics and control, with evaluations ranging from math and agent benchmarks to manipulation, locomotion, and navigation tasks. A central challenge is choosing examples that are hard enough to teach something but still learnable, while ensuring progress transfers across tasks rather than overfitting to a narrow curriculum.

Latest papers

124 papers21 papers in the last four weeks, up 50% on the four weeks before. 0.1% of all new papers.

SPARD: Self-Paced Curriculum for RL Alignment via Integrating Reward Dynamics and Data Utility
Apr 9, 2026Xuyang Zhi, Peilun zhou, Chengqiang Lu +10RL Post-Training Multi-Objective Language Model Alignment
University of Science and Technology of China · Xiaohongshu Inc.
Mitigating Distribution Sharpening in Math RLVR via Distribution-Aligned Hint Synthesis and Backward Hint Annealing
Apr 9, 2026Pei-Xi Xie, Che-Yu Lin, Cheng-Lin YangReinforcement Learning LLM Mathematical Reasoning
CyCraft AI Lab, Taiwan
Cog-DRIFT: Exploration on Adaptively Reformulated Instances Enables Learning from Hard Reasoning Problems
Apr 6, 2026Justin Chih-Yao Chen, Archiki Prasad, Zaid Khan +4RL for Language Model Reasoning LLM Reasoning
UNC Chapel Hill
Vocabulary Dropout for Curriculum Diversity in LLM Co-Evolution
Apr 3, 2026Jacob Dineen, Aswin RRV, Zhikun Xu +1Curriculum RL RL for Language Model Reasoning
Arizona State University
SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization
Apr 2, 2026Zhengxi Lu, Zhiyuan Yao, Jinyang Wu +7LLM Agent Skill Learning RL for Tool Use
Zhejiang University · Meituan · Tsinghua University
Learning to Play Blackjack: A Curriculum Learning Perspective
Mar 31, 2026Amirreza Alasti, Efe Erdal, Yücel Celik +1Curriculum RL LLM-Guided RL
Leibniz University Hannover, Hannover, Germany · Leibniz AI Academy, Hannover, Germany
COvolve: Adversarial Co-Evolution of Large-Language-Model-Generated Policies and Environments via Two-Player Zero-Sum Game
Mar 30, 2026Alkis Sygkounas, Rishi Hazra, Andreas Persson +2Lifelong Learning Agents Self-Evolving Agents
Machine Perception and Interaction Lab, Örebro University, Sweden
MedLoc-R1: Performance-Aware Curriculum Reward Scheduling for GRPO-Based Medical Visual Grounding
Mar 30, 2026Guangjing Yang, Ziyuan Qin, Chaoran Zhang +6Medical Image Grounding Reward Shaping
Beijing University of Posts and Telecommunications · Emory University · Peking University +1
Heuristic Self-Paced Learning for Domain Adaptive Semantic Segmentation under Adverse Conditions
Mar 25, 2026Shiqin Wang, Haoyang Chen, Huaizhou Huang +6Semantic Segmentation Curriculum Learning
National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence, School of Computer Science, Wuhan University · Hubei Key Laboratory of Multimedia and Network Communication Engineering · School of Computer Science, Wuhan University, Wuhan, China +2
A Deep Dive into Scaling RL for Code Generation with Synthetic Data and Curricula
Mar 25, 2026Cansu Sancaktar, David Zhang, Gabriel Synnaeve +1RL for Code Generation Synthetic Data Augmentation
University of Tübingen · Max Planck Institute for Intelligent Systems · Meta FAIR
Tutor-Student Reinforcement Learning: A Dynamic Curriculum for Robust Deepfake Detection
Mar 25, 2026Zhanhe Lei, Zhongyuan Wang, Jikang Cheng +5Deepfake Detection Curriculum RL
School of Computer Science, Wuhan University · School of Integrated Circuits, Peking University · School of Information, Huazhong Agricultural University +1
Self Paced Gaussian Contextual Reinforcement Learning
Mar 24, 2026Mohsen Sahraei Ardakani, Rui SongContextual RL Reinforcement Learning
Department of Electrical and Computer Engineering North Carolina State University Raleigh, NC · Department of Statistics North Carolina State University Raleigh, NC
WIST: Web-Grounded Iterative Self-Play Tree for Domain-Targeted Reasoning Improvement
Mar 22, 2026Fangyuan Li, Pengfei Li, Shijie Wang +4Language Model Self-Improvement RL for Language Model Reasoning
Harbin Institute of Technology · Shanghai Innovation Institute · Shanghai Artificial Intelligence Laboratory
Context Bootstrapped Reinforcement Learning
Mar 19, 2026Saaket Agashe, Jayanth Srinivasa, Gaowen Liu +2RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
University of California, Santa Barbara · Cisco Research
Learning to Reason with Curriculum I: Provable Benefits of Autocurriculum
Mar 18, 2026Nived Rajaraman, Audrey Huang, Miro Dudik +3Efficient Language Model Training Curriculum RL
Microsoft Research · University of Illinois Urbana Champaign
Thermodynamics of Reinforcement Learning Curricula
Mar 12, 2026Jacob Adamczyk, Juan Sebastian Rojas, Rahul V. KulkarniReinforcement Learning Curriculum RL
Department of Physics, University of Massachusetts Boston, Boston, MA 02125, USA · Department of Mechanical & Industrial Engineering, University of Toronto, Toronto, ON, CA
DeReason: A Difficulty-Aware Curriculum Improves Decoupled SFT-then-RL Training for General Reasoning
Mar 11, 2026Hanxu Hu, Yuxuan Wang, Maggie Huan +4RL for Language Model Reasoning RL Fine-Tuning
University of Zurich · University of Pennsylvania · ETH Zurich +2
SPAARS: Safer RL Policy Alignment through Abstract Exploration and Refined Exploitation of Action Space
Mar 10, 2026Swaminathan S K, Aritra HazraReinforcement Learning Offline-To-Online RL
Department of Computer Science and Engineering, IIT Kharagpur
Multi-level meta-reinforcement learning with skill-based curriculum
Mar 9, 2026Sichen Yang, Mauro MaggioniReinforcement Learning Hierarchical RL
Johns Hopkins University · Department of Applied Mathematics & Statistics Johns Hopkins University, 3400 N. Charles Street, Baltimore, MD 21218, USA · Department of Mathematics, Department of Applied Mathematics & Statistics Johns Hopkins University, 3400 N. Charles Street, Baltimore, MD 21218, USA
Wiki-R1: Incentivizing Multimodal Reasoning for Knowledge-based VQA via Data and Sampling Curriculum
Mar 5, 2026Shan Ning, Longtian Qiu, Xuming HeVisual Question Answering Reinforcement Learning
ShanghaiTech University, Shanghai, China · Lingang Laboratory, Shanghai, China · Shanghai Engineering Research Center of Intelligent Vision and Imaging
Decoupling Task and Behavior: A Two-Stage Reward Curriculum in Reinforcement Learning for Robotics
Mar 5, 2026Kilian Freitag, Knut Åkesson, Morteza Haghir ChehreghaniRL for Robotics Reinforcement Learning
Chalmers University of Technology · Gothenburg University
ACDC: Adaptive Curriculum Planning with Dynamic Contrastive Control for Goal-Conditioned Reinforcement Learning in Robotic Manipulation
Mar 2, 2026Xuerui Wang, Guangyu Ren, Tianhong Dai +4Goal-Conditioned RL Goal-Conditioned Robotic Manipulation
Cornell University, Ithaca, NY 14853, USA · Xi’an Jiaotong-Liverpool University, Suzhou, China · Applied Computing, UK
Curriculum Reinforcement Learning for Quadrotor Racing with Random Obstacles
Feb 27, 2026Fangyu Sun, Fanxing Li, Yu Hu +4Autonomous Drone Racing Autonomous Racing
School of Automation and Perception, Shanghai Jiao Tong University, China · Faculty of Aerospace Engineering, Delft University of Technology, The Netherlands
Self-Curriculum Model-based Reinforcement Learning for Shape Control of Deformable Linear Objects
Feb 25, 2026Zhaowei Liang, Song Wang, Zhao Jin +2Deformable Object Manipulation Curriculum RL
All authors are with the Department of Mechanical Engineering, Tsinghua University, Beijing, China.
RuCL: Stratified Rubric-Based Curriculum Learning for Multimodal Large Language Model Reasoning
Feb 25, 2026Yukun Chen, Jiaming Li, Longze Chen +10VLM Reasoning Rubric-Based RL
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Alibaba Group +1
GradAlign: Gradient-Aligned Data Selection for LLM Reinforcement Learning
Feb 25, 2026Ningyuan Yang, Weihua Du, Weiwei Sun +2RL for Language Models Reinforcement Learning
Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University · Language Technologies Institute (LTI), Carnegie Mellon University
Actor-Curator: Co-adaptive Curriculum Learning via Policy-Improvement Bandits for RL Post-Training
Feb 24, 2026Zhengyao Gu, Jonathan Light, Raul Astudillo +7Curriculum RL Curriculum Learning
University of Illinois Chicago · Caltech · RPI +2
Continual Uncertainty Learning for Robust Control of Nonlinear Systems with Multiple Heterogeneous Uncertainties
Feb 19, 2026Heisei Yonezawa, Ansei Yonezawa, Itsuro KajiwaraRobust Control Robust RL
Division of Mechanical and Aerospace Engineering, Hokkaido University N13, W8, Kita-ku, Sapporo, Hokkaido 060-8628, Japan · Faculty of Information Science and Technology, Hokkaido University Kita 14, Nishi 9, Kita-ku, Sapporo, Hokkaido 060-0814, Japan
TAROT: Test-driven and Capability-adaptive Curriculum Reinforcement Fine-tuning for Code Generation with Large Language Models
Feb 17, 2026Chansung Park, Juyong Jiang, Fan Wang +4LLM Code Generation RL for Code Generation
Electronics and Telecommunications Research Institute · The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology +2
Goldilocks RL: Tuning Task Difficulty to Escape Sparse Rewards for Reasoning
Feb 16, 2026Ilia Mahrooghi, Aryo Lotfi, Emmanuel AbbeRL for Language Model Reasoning Curriculum RL
EPFL · Apple

Apr 9, 2026J/K move · Enter open · S save

SPARD: Self-Paced Curriculum for RL Alignment via Integrating Reward Dynamics and Data Utility

Xuyang Zhi, Peilun zhou, Chengqiang Lu, Hang Lv+9

University of Science and Technology of China · Xiaohongshu Inc.

RL Post-Training Multi-Objective Language Model Alignment Curriculum RL Reward Design for RL

Open paper AskPDF arXiv

The evolution of Large Language Models (LLMs) is shifting the focus from single, verifiable tasks toward complex, open-ended real-world scenarios, imposing significant challenges on the post-training phase. In these settings, the scale and complexity of reward systems have grown significantly, transitioning toward multi-objective formulations that encompass a comprehensive spectrum of model capabilities and application contexts. However, traditional methods typically rely on fixed reward weights, ignoring non-stationary learning dynamics and struggling with data heterogeneity across dimensions. To address these issues, we propose SPARD, a framework that establishes an automated, self-paced curriculum by perceiving learning progress to dynamically adjust multi-objective reward weights and data importance, thereby synchronizing learning intent with data utility for optimal performance. Extensive experiments across multiple benchmarks demonstrate that SPARD significantly enhances model capabilities across all domains.