cs.RONov 10, 2026

ExRL: Sample-Efficient Online Adaptation of Long-Chunk Policies via Execution Length Control

Authors: Xu Luo, Jiaying Yang, Hao Wu, Junlin Xie, Zijin Hong, Ji Zhang, Lianli Gao, Jingkuan Song

Organizations: UESTC · Tongji University · The Hong Kong Polytechnic University

Abstract

Predicting long action chunks, often spanning dozens of timesteps, has become a key ingredient in increasingly capable robot policies. In this paper, we find that adapting these long-chunk policies with existing reinforcement learning (RL) methods can require many online samples due to the high dimensionality of the action chunks being optimized, making such adaptation impractical in real-world scenarios where interaction data are costly to collect. Our key insight is to shift adaptation from optimizing actions to optimizing how long to execute each predicted action chunk. This keeps the adaptation space low-dimensional while its expressivity grows with the chunk size. We thus introduce Execution at the Right Length (ExRL), a simple approach which augments the base policy with a lightweight adapter that predicts the execution length of each chunk and is trained using off-policy RL algorithms. Across simulation benchmarks and real-world robotic tasks, ExRL enables sample-efficient adaptation of a diverse range of long-chunk policies with minimal task-specific hyperparameter tuning.

Figures & tables

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Adaptive Action Chunking via Multi-Chunk Q Value Estimation

    May 11, 2026Yongjae Shin, Jongseong Chae, Seongmin Kim +2Q-LearningAction Chunking

  2. RoboLongR: Causally Consistent Rewards Teach Robots to Progressively Master Long-Horizon Manipulation

    Oct 24, 2026Zixuan Chen, Lin Cao, Fenggang Zhang +5Long-Horizon Robotic ManipulationRobot Policy Adaptation

  3. Adaptive Q-Chunking for Offline-to-Online Reinforcement Learning

    May 7, 2026Nandiraju Gireesh, Yuanliang Ju, He WangQ-LearningOffline RL