Action chunking --- the practice of predicting a sequence of actions and executing a prefix open-loop --- has emerged as a key enabler of recent progress in imitation learning for robotic manipulation. However, executing long open-loop prefixes reduces reactivity, limiting policies' ability to correct for errors. Further, the mechanisms underlying these performance benefits remain poorly understood: prior works cite mitigating compounding errors, absorbing inference latency, or smoothing motions, but provide limited controlled evidence or guidance for preserving reactivity. In this work, we argue that long open-loop execution primarily helps short-context policies imitate "non-Markovian demonstrations". Across four simulation and two real-world tasks, we show that expert non-Markovianity strongly shapes the relationship between task success and open-loop execution horizon. Further, we investigate the impact of compounding errors --- the prevailing explanation for long open-loop execution in prior work --- and find that while they matter, expert non-Markovianity has a much stronger impact in our experimental setting. Finally, we show that when policies are provided with a sufficiently long context, open-loop execution is no longer beneficial and the most reactive, closed-loop policies perform best. While imitation learning has seen great success using long open-loop execution, our findings motivate long-context, reactive policies as a more principled and performant paradigm.
Figures & tables
Figure 1: Benchmark Tasks. FurnitureSimOneLeg [ 51 ] : pick up table leg and insert and screw into table top to assemble the table. Push-T [ 52 ] : Push T-shaped slider to target pose, then return robot to reset pose. GearInsertion : Pick up large gear and insert onto gear carrier, meshed with other gears. Kitchen [ 53 ] : Complete any 4 kitchen subtasks (e.g., open microwave, move kettle, turn top/bottom knob, flip switch, open slide door, open hinge door), in any order; the dataset contains random subtask permutations. All tasks use image-based observations.
Figure 2: Success-horizon curves for Furniture-Sim one-leg . The policy trained on human non-Markovian (NM) expert data ( Left ) exhibits the typical inverted U-shape curve; the policy trained on Markovian (M) scripted expert data ( Center ) exhibits an approximately monotonically decreasing curve; adding non-Markovian behaviors to the scripted expert ( Right ) restores the inverted U-shape.
Figure 3: Success-horizon curve with non-Markovian (NM) human vs scripted Markovian (M) expert demonstrations for PushT-M (Top Left), GearInsertion (Top Right), and Kitchen (Bottom) tasks. All exhibit the typical inverted U-shaped curve when trained on human data, but all become monotonically decreasing when trained on Markovian expert data.
Figure 4: Illustration of common failure modes of short-context policies using short execution horizons . (a) : FurnitureSimOneLeg : robot cycles between coherent alignment motions but fails to transition to insertion, causing timeout. (b) : FurnitureSimOneLeg : robot performs repeated alignment motions with the table leg, but fails to transition to grasp. (c) : GearInsertion : robot repeatedly aligns gear with shaft. (d) : Push-T : the robot jitters until timeout, oscillating between two modes (pushing the T rightward versus moving around to push it upward). (e) : Kitchen : robot approaches light switch but abruptly changes course toward the top knob and collides.
Figure 5: Success-horizon curve at dataset sizes ∈{100,200,400,1000} demonstrations , using both Markovian ( Right ) and non-Markovian ( Left ) scripted expert data. Only at the lowest data scale (100 demonstrations), we find compounding errors to play a significant role in shaping the curve, causing even the policy trained on Markovian expert data to prefer a long execution horizon. Above 200 demonstrations, we find increasing dataset size only modestly shifts preference toward lower execution horizons.
Figure 6: Success-horizon curves after applying HG-DAgger for Scripted Markovian ( Right ) and non-Markovian ( Left ) expert-trained policies. We find no noticeable shift for the non-Markovian expert-trained policy, and a slight shift favoring lower execution horizons for the Markovian expert-trained policy.
Figure 7: Success-horizon curves of policies trained across a range of context lengths for FurnitureSimOneLeg , on 200 human demonstrations . As context length increases, peak performance shifts toward shorter execution horizons, with the longest-context policies performing best at execution horizons of 1 or 2.
Figure 8: Success-horizon curves of policies trained across a range of context lengths for FurnitureSimOneLeg , on 200 and 1000 non-Markovian scripted expert demonstrations. We find increasing dataset size disproportionately improves performance of long-context policies at low execution horizons. With 1000 demonstrations, the highest performing policies use long context length To of 12 – 20 and short execution horizon Texec<4 and achieve 93.2% success, outperforming the best short-context ( To=2 ) policy at Texec=6 with 90.6% success.
Figure 9: Success-horizon curves of policies trained across a range of context lengths for PushT-D , GearInsertion , and Kitchen on human demonstrations . Dataset sizes are 160 demonstrations for PushT-D , 200 for GearInsertion , and 581 for Kitchen . With sufficiently long context length, Texec=1 becomes optimal or near-optimal in all tasks.
Figure 10: Real-world SinglePillDispense (Left) and SlipIntoBaggie (Right) tasks. In SinglePillDispense , a successful rollout dumps a single pill out of the bottle onto the cap and then rights the bottle. In SlipIntoBaggie , a successful rollout fully inserts the paper slip into the baggie without crumpling the paper.
SinglePillDispense
SlipIntoBaggie
Texec=2
Texec=4
Texec=8
Texec=2
Texec=4
Texec=8
To=2
50% [29.9, 70.1]
60% [38.7, 78.1]
75% [53.1, 88.8]
60% [38.7, 78.1]
40% [21.9, 61.3]
80% [58.4, 91.9]
To=8
90% [69.9, 97.2]
80% [58.4, 91.9]
60% [38.7, 78.1]
85% [64.0, 94.8]
70% [48.1, 85.5]
55% [34.2, 74.2]
To=12
75% [53.1, 88.8]
70% [48.1, 85.5]
60% [38.7, 78.1]
75% [53.1, 88.8]
65% [43.3, 81.9]
65% [43.3, 81.9]
Table 1: SinglePillDispense and SlipIntoBaggie success rates over 20 trials, with 95% confidence intervals.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Task
Proprioceptive State
Action Space
FurnitureSimOneLeg
16D: end-effector pose, velocity, and gripper width
Table 4: Training Hyperparameters Shared for All Models
Parameter
Value
U-Net Down dims
[256,512,1024]
U-Net Parameters
69 million
DDIM Inference Steps
10
Appendix
Table 5: Diffusion Model Hyperparameters
Figure 11: Illustration of hidden-state aliasing in an idle demonstration. While making high precision maneuvers in tasks like Push-T , humans often pause to align, observe, and think. Such pauses show up as a sequence of idle actions o1k→o2k→⋯→o10k in the training data. In this example, the full-history conditioned expert πE advances after it observes the 10th idle state. The imitator πθ only observes the current state o0 and cannot infer idle count, so both idling and advancing appear valid. The imitator is therefore unlikely to reproduce the expert’s exact sequence of 10 idle actions before advancing, and may idle for much longer than desired, resulting in task timeout.
Figure 12: Illustration of cycling. Two demonstrations jointly induce a cycle in the aggregated dataset. When observing only o0 or o2 , a short-context imitator cannot determine whether to continue or exit the cycle, since both actions appear valid in the training data. It may therefore repeatedly switch between the demonstrated trajectories and enter a cycle: o0→o1→o2→o3→o0 . Longer execution horizons reduce the chance of entering such a cycle: a chunk of actions originating at o0 toward o1 and executed open-loop with Texec≥3 will cross beyond o2 , whereas re-planning at o2 can increase the chance of returning to o3 and entering a cycle.
Figure 13: Illustration of how context length disambiguates the expert action distribution. The observations o2,…,ok form an aliased sequence: conditioned only on observations within this sequence, a short-context policy cannot identify the expert’s intended behavior (whether to imitate demonstration 1 or 2). A long execution horizon reduces the probability of re-planning within the sequence, but ambiguity remains whenever re-planning occurs there. By contrast, a sufficiently long context includes the preceding observation o0 , which disambiguates the correct expert action ( a1 ) distribution throughout the sequence, regardless of re-planning phase.
Figure 14: Success-horizon curves of policies trained on scripted non-Markovian expert data under exploratory noise injection. Left : ablation over the action-noise scale σu , with fraction of clean trajectories fixed at α=0.8 . Right : ablation over clean-trajectory fraction α , with σu=41 . We use To=1 in this experiment for reasons described below.
Figure 15: FurnitureSimOneLeg success-horizon curves for To∈{2,4,8,12,16,20} for policies trained with double encoder cross attention U-Net and cross attention U-Net (no double encoder) architectures on 200 human teleop demonstrations. Both plots exhibit similar trends (shorter Texec perform better as To increases) and comparable absolute performance, but the double encoder architecture exhibits much less variation in performance across execution horizons, producing smoother and more interpretable curves.
Figure 16: Success-horizon curves from long-context policies trained in PushT-M .
Figure 17: Distribution of the optimal training duration (calculated as the fraction of the highest-performing checkpoint’s epoch divided by the latest evaluated checkpoint’s epoch) as a function of execution horizon , pooled across all short-context ( To=2 ) scripted- and human-expert experiments (individual points represent individual experiments; black diamonds/bars represent median and interquartile range). We observe a consistent trend across almost all experiments that lower execution horizons prefer earlier checkpoints.
Figure 18: Distribution of the optimal training duration as a function of context length for all FurnitureSimOneLeg , SinglePillDispense , and SlipIntoBaggie context length experiments (including all execution horizons). We observe policies with longer context lengths often perform better at earlier training checkpoints.
Action chunking improves the performance of policies learned by behavioural cloning, and several mechanisms have been proposed to explain why, including temporal consistency, horizon reduction, representation learning, and reduced error compounding. We instead study what happens to an action error once it enters the system. At each state, we inject a small action error and measure how fast it grows or shrinks under two execution regimes: open-loop, where the rest of the chunk is replayed without replanning, and closed-loop, where the policy replans after the perturbation. The fitted rate labels each state as contracting, expanding, or unresolved. Across twelve manipulation tasks from three benchmark suites, we find that confidently stable states are rare, while error amplification is common among states whose propagation rate can be resolved. We further find that the measured propagation rate depends strongly on the fitting horizon: amplification is typically front-loaded, so short windows can overestimate longer-horizon propagation. Finally, we train predictors on these labels and find that a state's open-loop regime can be recovered from camera frames and proprioception alone, while its closed-loop propagation is only partially recoverable because it also depends on how the policy acts after the perturbation. These results suggest that error-compounding arguments alone do not provide a complete account of action chunking: neither passive open-loop dynamics nor policy replanning consistently contracts an injected error, and replanning rarely turns open-loop amplification into confident contraction. This suggests that closed-loop reactivity should be trained explicitly, using perturbation- and tree-coverage-oriented training to expose policies to deviations they must recover from, rather than expected to emerge reliably from standard imitation learning.
Robotic imitation learning faces a fundamental trade-off between modeling long-horizon dependencies and enabling fine-grained closed-loop control. Existing fixed-frequency action chunking approaches struggle to achieve both. Building on this insight, we propose HiPolicy, a hierarchical multi-frequency action chunking framework that jointly predicts action sequences at different frequencies to capture both coarse high-level plans and precise reactive motions. We extract and fuse hierarchical features from history observations aligned to each frequency for multi-frequency chunk generation, and introduce an entropy-guided execution mechanism that adaptively balances long-horizon planning with fine-grained control based on action uncertainty. Experiments on diverse simulated benchmarks and real-world manipulation tasks show that HiPolicy can be seamlessly integrated into existing 2D and 3D generative policies, delivering consistent improvements in performance while significantly enhancing execution efficiency.
Jiyao Zhang, Zimu Han, Junhan Wang +7
CFCS, School of CS, PKU · National Key Laboratory for Multimedia Information Processing, School of CS, PKU · XJTU +3
Dexterous manipulation policies learned by imitation are typically evaluated for robustness to variation in scenes, objects, or instructions, but their performance across task execution speeds is less often examined. This leaves open how much temporal robustness a learner retains relative to the expert it imitates. We compare an expert and learner under the same task conditions, initial-condition draws, and speedup factors. We instantiate the evaluation in ParcelStow, a contact-rich task in which the robot acquires, reorients, and inserts a parcel. The demonstrations span the speedup range for the manipulation phases after parcel acquisition. A scripted expert and an Action Chunking with Transformers (ACT) policy trained from the expert's demonstrations both achieve 100 percent task success at nominal speed. Their success rates diverge within the demonstrated range: at its maximum, expert success is 84 percent and ACT success is 53 percent. Two ACT policies with different parameter initializations show similar degradation, decreasing by 34 and 48 percentage points from nominal speed to the maximum demonstrated speed, compared with 16 points for the expert. Stage-level analysis shows that 35 of ACT's 47 failures at the maximum demonstrated speed are insertion misalignments. Under the relative-motion handoff, every ACT acquisition retains the parcel through reorientation and transfer in free space, but only 64 percent complete the overall task, compared with 95 percent after expert acquisition. Across all evaluated policies and speeds, none of the 414 acquisitions without force closure completes the task. Equal nominal task success therefore does not imply preservation of expert performance across execution speeds. Code, data, and evaluation scripts are available at https://github.com/coenwerem/parcelstow.
Clinton Enwerem, John S. Baras, Calin Belta
Institute for Systems Research University of Maryland College Park, MD, U.S.A.