cs.RONov 9, 2026

Fade to... Red? Scheduling Coloured Noise Exploration Supports Efficient Robot Learning with Reduced Movement Jerk

Authors: Francisco M. López, Francisco Cruz

Organizations: School of Computer Science and Engineering, UNSW Sydney, Australia · Frankfurt Institute for Advanced Studies, Germany

Abstract

Exploration in deep reinforcement learning (RL) for continuous control is commonly driven by temporally uncorrelated Gaussian (white) action noise, which leads to jerky motions with locally constrained exploration. In contrast, temporally correlated coloured noise, particularly pink noise, has been shown to support more efficient exploration in RL. Motivated by these findings, we inquire whether scheduling changes to the noise colour over the course of training can outperform white noise for continuous control robotics. We perform a systematic study of 49 linear schedules between white and red noise, together with generalised state-dependent exploration (gSDE), on four sparse-reward manipulation tasks with a Fetch arm and a Shadow Dexterous Hand. We find that the noise colour at the end of training does not affect learning efficiency or end performance, whereas fading towards red noise halves the jerk of exploration in every task. Balancing learning efficiency, end performance and smoothness, red noise emerges as the preferred strategy: it learns as efficiently as pink noise with almost 2-fold less movement jerk. We therefore recommend exploration schedulers that fade towards red noise as the default for off-policy robot learning.

Figures & tables

Explore similar work

CardsList
  1. Infant Spontaneous Movement Noise Improves Exploration in Deep RL

    Jun 15, 2026Francisco M. López, Markus R. Ernst, Francisco Cruz +2Reinforcement LearningRL Exploration

  2. Start Right, Arrive Right: Asynchronous Execution via Initial Noise Selection

    Jun 18, 2026Trong-Bao Ho, Quang-Tan Nguyen, Thien-Loc Ha +7Robotic ControlRobot Policy Learning

  3. DF-ExpEnse: Diffusion Filtered Exploration for Sample Efficient Finetuning

    Jun 17, 2026Calvin Luo, Chen Sun, Shuran SongRobot Policy AdaptationRobotic RL