Curriculum RL

RL methods for scaling large reasoning models stall on datasets with low initial success rates, and thus little training signal. We investigate a fundamental question: Can a pretrained LLM leverage latent knowledge to generate an automated curriculum for problems it cannot solve? We explore this with SOAR: An asymmetric self-play framework that uses meta-RL to surface these pedagogical signals. A teacher model proposes synthetic problems for a student model, and is rewarded with its improvement on a subset of hard problems, thus grounding the curriculum in real student progress rather than intrinsic proxy rewards. Our study on the hardest subsets of math benchmarks (0/128 success) reveals three core findings. First, it is possible to realize bilevel meta-RL that unlocks learning under sparse, binary rewards by sharpening a latent capacity of pretrained models to generate useful problems. Second, grounded rewards outperform intrinsic learnability rewards used in prior LLM self-play, reliably avoiding typical instability and diversity collapse modes. Third, the structure and well-posedness of questions are more critical for learning progress than solution correctness. Our results suggest that the ability to generate useful stepping stones does not require the preexisting ability to solve the hard problems, paving a principled path to escape reasoning plateaus without additional curated data

Humanoid robots have demonstrated strong capabilities for interacting with static scenes across locomotion and manipulation, yet dynamic real-world interactions remain challenging. As a step toward fast-moving object interactions, we present an RL training pipeline that yields a unified whole-body controller for humanoid badminton, coordinating footwork and striking without motion priors or expert demonstrations. In badminton, locomotion and striking are tightly coupled, making the final hitting objective difficult to optimize directly due to sparse rewards and conflicting gradients. We address this with an annealed curriculum that first stabilizes learning through auxiliary locomotion objectives, then progressively removes them to focus optimization on the final hitting objective. For deployment, we use an Extended Kalman Filter (EKF) to estimate and predict shuttlecock trajectories for target striking, and also develop a prediction-free variant that removes the EKF and explicit prediction. We validate the framework in simulation and on hardware. In simulation, two robots sustain a rally of 21 consecutive hits. In real-world tests with both machine-fed shuttles and human-robot rallies, the robot achieves outgoing shuttle speeds up to 19.1 m/s. Moreover, the prediction-free variant attains comparable performance to the EKF-based policy. Overall, our approach enables dynamic yet precise goal striking in humanoid badminton and suggests a path toward more dynamics-critical whole-body interaction tasks.

We develop an approach for the de novo synthesis of messenger RNA (mRNA) sequences encoding target proteins. This requires optimizing multiple biological objectives over large combinatorial spaces defined by synonymous codon sequences, a setting where Reinforcement Learning (RL) methods suffer from mode collapse and Generative Flow Networks (GFlowNets) face sparse rewards over long horizons. To address these challenges, we propose Curriculum-Augmented GFlowNets (CAGFNs), a multi-objective GFlowNet trained with an adaptive length-based curriculum driven by learning progress. In a nutshell, CAGFNs learn from progressively harder tasks corresponding to longer target proteins. We evaluate CAGFNs on CodonDesignEnv, a novel codon-level environment that preserves protein identity via dynamic action masking. Our experiments show that CAGFNs achieve competitive or stronger Pareto-front coverage and diversity while converging up to 4 times faster than non-curriculum GFlowNets, and maintaining 100% sequence uniqueness compared to RL baselines.

Autonomous unmanned aerial vehicles (UAVs) increasingly rely on reinforcement learning (RL) for navigation, yet global navigation satellite system (GNSS) spoofing attacks can induce out-of-distribution observation shifts that corrupt value estimation and degrade mission performance. Existing robust RL methods often lack generalization to unseen attacks. To address this, we propose a curriculum-guided adaptation framework that progressively exposes a robust policy to gradient-based adversarial observation perturbations of increasing intensity while aligning temporal-difference (TD) error distributions across curriculum stages. By preserving TD-error consistency rather than targeting specific attack models, the framework promotes resilience across attack conditions. We further derive a TD-space generalization certificate that bounds performance degradation when the TD-error distribution under a test-time attack remains sufficiently close to that of the final curriculum stage. We evaluate the framework in a UAV deconfliction environment with dynamic 3D obstacles under unseen fixed and dynamic GNSS spoofing attacks. Under fixed spoofing, the curriculum-adapted (CurAd) policy achieves approximately 99-100% mission success across evaluated traffic densities, compared with 20-24% for the best-performing robust baseline, while also outperforming the non-robust vanilla baseline. Under dynamic obstacle-luring spoofing, CurAd achieves the highest episodic rewards and reduces median mission completion steps by up to 22% relative to the best-performing robust RL baseline. These findings demonstrate that adversarial curriculum adaptation transfers resilience from synthetic perturbations to unseen GNSS spoofing attacks without attack-specific retraining, enhancing robust autonomous UAV navigation.