Revisiting Open-Loop Execution in Robotics: Toward Reactive, Higher-Performing Policies

Michael Zeng; Abhinav Agarwal; Ajay Bati; Brian Lee; Siddharth Ancha; Russ Tedrake

cs.ROAug 16, 2026

Revisiting Open-Loop Execution in Robotics: Toward Reactive, Higher-Performing Policies

Authors: Michael Zeng, Abhinav Agarwal, Ajay Bati, Brian Lee, Siddharth Ancha, Russ Tedrake

Organizations: Massachusetts Institute of Technology · Mundane Systems Inc. · University of California, Berkeley

Abstract

Action chunking --- the practice of predicting a sequence of actions and executing a prefix open-loop --- has emerged as a key enabler of recent progress in imitation learning for robotic manipulation. However, executing long open-loop prefixes reduces reactivity, limiting policies' ability to correct for errors. Further, the mechanisms underlying these performance benefits remain poorly understood: prior works cite mitigating compounding errors, absorbing inference latency, or smoothing motions, but provide limited controlled evidence or guidance for preserving reactivity. In this work, we argue that long open-loop execution primarily helps short-context policies imitate "non-Markovian demonstrations". Across four simulation and two real-world tasks, we show that expert non-Markovianity strongly shapes the relationship between task success and open-loop execution horizon. Further, we investigate the impact of compounding errors --- the prevailing explanation for long open-loop execution in prior work --- and find that while they matter, expert non-Markovianity has a much stronger impact in our experimental setting. Finally, we show that when policies are provided with a sufficiently long context, open-loop execution is no longer beneficial and the most reactive, closed-loop policies perform best. While imitation learning has seen great success using long open-loop execution, our findings motivate long-context, reactive policies as a more principled and performant paradigm.

Figures & tables

Figure 1: Benchmark Tasks. FurnitureSimOneLeg [ 51 ] : pick up table leg and insert and screw into table top to assemble the table. Push-T [ 52 ] : Push T-shaped slider to target pose, then return robot to reset pose. GearInsertion : Pick up large gear and insert onto gear carrier, meshed with other gears. Kitchen [ 53 ] : Complete any 4 kitchen subtasks (e.g., open microwave, move kettle, turn top/bottom knob, flip switch, open slide door, open hinge door), in any order; the dataset contains random subtask permutations. All tasks use image-based observations.

Figure 2: Success-horizon curves for Furniture-Sim one-leg . The policy trained on human non-Markovian (NM) expert data ( Left ) exhibits the typical inverted U-shape curve; the policy trained on Markovian (M) scripted expert data ( Center ) exhibits an approximately monotonically decreasing curve; adding non-Markovian behaviors to the scripted expert ( Right ) restores the inverted U-shape.

Figure 3: Success-horizon curve with non-Markovian (NM) human vs scripted Markovian (M) expert demonstrations for PushT-M (Top Left), GearInsertion (Top Right), and Kitchen (Bottom) tasks. All exhibit the typical inverted U-shaped curve when trained on human data, but all become monotonically decreasing when trained on Markovian expert data.

Panel of Figure 4 — Figure 4: Illustration of common failure modes of short-context policies using short execution horizons . (a) : FurnitureSimOneLeg : robot cycles between coherent alignment motions but fails to transition to insertion, causing timeout. (b) : FurnitureSimOneLeg : robot performs repeated alignment motions with the table leg, but fails to transition to grasp. (c) : GearInsertion : robot repeatedly aligns gear with shaft. (d) : Push-T : the robot jitters until timeout, oscillating between two modes (pushing the T rightward versus moving around to push it upward). (e) : Kitchen : robot approaches light switch but abruptly changes course toward the top knob and collides.

Figure 5: Success-horizon curve at dataset sizes $\in\{100,200,400,1000\}$ demonstrations , using both Markovian ( Right ) and non-Markovian ( Left ) scripted expert data. Only at the lowest data scale (100 demonstrations), we find compounding errors to play a significant role in shaping the curve, causing even the policy trained on Markovian expert data to prefer a long execution horizon. Above 200 demonstrations, we find increasing dataset size only modestly shifts preference toward lower execution horizons.

Figure 6: Success-horizon curves after applying HG-DAgger for Scripted Markovian ( Right ) and non-Markovian ( Left ) expert-trained policies. We find no noticeable shift for the non-Markovian expert-trained policy, and a slight shift favoring lower execution horizons for the Markovian expert-trained policy.

Figure 7: Success-horizon curves of policies trained across a range of context lengths for FurnitureSimOneLeg , on 200 human demonstrations . As context length increases, peak performance shifts toward shorter execution horizons, with the longest-context policies performing best at execution horizons of 1 or 2.

Figure 8: Success-horizon curves of policies trained across a range of context lengths for FurnitureSimOneLeg , on 200 and 1000 non-Markovian scripted expert demonstrations. We find increasing dataset size disproportionately improves performance of long-context policies at low execution horizons. With 1000 demonstrations, the highest performing policies use long context length $T_{o}$ of 12 – 20 and short execution horizon $T_{\text{exec}}<4$ and achieve 93.2% success, outperforming the best short-context ( $T_{o}=2$ ) policy at $T_{\text{exec}}=6$ with 90.6% success.

Figure 9: Success-horizon curves of policies trained across a range of context lengths for PushT-D , GearInsertion , and Kitchen on human demonstrations . Dataset sizes are 160 demonstrations for PushT-D , 200 for GearInsertion , and 581 for Kitchen . With sufficiently long context length, $T_{\text{exec}}=1$ becomes optimal or near-optimal in all tasks.

Panel of Figure 10 — Figure 10: Real-world SinglePillDispense (Left) and SlipIntoBaggie (Right) tasks. In SinglePillDispense , a successful rollout dumps a single pill out of the bottle onto the cap and then rights the bottle. In SlipIntoBaggie , a successful rollout fully inserts the paper slip into the baggie without crumpling the paper.

	SinglePillDispense			SlipIntoBaggie
	$T_{\mathrm{exec}}=2$	$T_{\mathrm{exec}}=4$	$T_{\mathrm{exec}}=8$	$T_{\mathrm{exec}}=2$	$T_{\mathrm{exec}}=4$	$T_{\mathrm{exec}}=8$
$T_{o}=2$	50% [29.9, 70.1]	60% [38.7, 78.1]	75% [53.1, 88.8]	60% [38.7, 78.1]	40% [21.9, 61.3]	80% [58.4, 91.9]
$T_{o}=8$	90% [69.9, 97.2]	80% [58.4, 91.9]	60% [38.7, 78.1]	85% [64.0, 94.8]	70% [48.1, 85.5]	55% [34.2, 74.2]
$T_{o}=12$	75% [53.1, 88.8]	70% [48.1, 85.5]	60% [38.7, 78.1]	75% [53.1, 88.8]	65% [43.3, 81.9]	65% [43.3, 81.9]

Table 1: SinglePillDispense and SlipIntoBaggie success rates over 20 trials, with 95% confidence intervals.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Proprioceptive State	Action Space
FurnitureSimOneLeg	16D: end-effector pose, velocity, and gripper width	10D: delta end-effector command (3D translation, 6D rotation, 1D gripper)
Push-T	3D: planar end-effector position and orientation	2D: delta end-effector position in the XY plane
GearInsertion	9D: 3D end-effector position, 4D orientation quaternion, 2D gripper state	7D: delta end-effector pose and gripper command
Kitchen	9D: seven arm-joint positions and two gripper-finger positions	9D: velocity commands for the seven arm joints and two gripper joints

Appendix

Table 2: Benchmark Tasks – Proprioceptive Inputs and Action Spaces

Task	Predicted-Action Frequency
FurnitureSimOneLeg	10 Hz
Push-T	10 Hz
GearInsertion	30 Hz
Kitchen	12.5 Hz

Appendix

Table 3: Benchmark Tasks – Policy Action Prediction Frequencies

Parameter	Value
Max LR	$10^{-4}$
LR Scheduler	Cosine
Weight Decay	$10^{-6}$
Warmup Steps	500
Batch Size	64
Image Size Input	$N_{\text{cameras}}\times 320\times 240\times 3$

Appendix

Table 4: Training Hyperparameters Shared for All Models

Parameter	Value
U-Net Down dims	$[256,512,1024]$
U-Net Parameters	69 million
DDIM Inference Steps	10

Appendix

Table 5: Diffusion Model Hyperparameters

Figure 11: Illustration of hidden-state aliasing in an idle demonstration. While making high precision maneuvers in tasks like Push-T , humans often pause to align, observe, and think. Such pauses show up as a sequence of idle actions $o^{k}_{1}\rightarrow o^{k}_{2}\rightarrow\cdots\rightarrow o^{k}_{10}$ in the training data. In this example, the full-history conditioned expert $\pi_{E}$ advances after it observes the 10th idle state. The imitator $\pi_{\theta}$ only observes the current state $o_{0}$ and cannot infer idle count, so both idling and advancing appear valid. The imitator is therefore unlikely to reproduce the expert’s exact sequence of 10 idle actions before advancing, and may idle for much longer than desired, resulting in task timeout.

Figure 12: Illustration of cycling. Two demonstrations jointly induce a cycle in the aggregated dataset. When observing only $o^{0}$ or $o^{2}$ , a short-context imitator cannot determine whether to continue or exit the cycle, since both actions appear valid in the training data. It may therefore repeatedly switch between the demonstrated trajectories and enter a cycle: $o^{0}\rightarrow o^{1}\rightarrow o^{2}\rightarrow o^{3}\rightarrow o^{0}$ . Longer execution horizons reduce the chance of entering such a cycle: a chunk of actions originating at $o^{0}$ toward $o^{1}$ and executed open-loop with $T_{exec}\geq 3$ will cross beyond $o^{2}$ , whereas re-planning at $o^{2}$ can increase the chance of returning to $o^{3}$ and entering a cycle.

Figure 13: Illustration of how context length disambiguates the expert action distribution. The observations $o^{2},\ldots,o^{k}$ form an aliased sequence: conditioned only on observations within this sequence, a short-context policy cannot identify the expert’s intended behavior (whether to imitate demonstration 1 or 2). A long execution horizon reduces the probability of re-planning within the sequence, but ambiguity remains whenever re-planning occurs there. By contrast, a sufficiently long context includes the preceding observation $o^{0}$ , which disambiguates the correct expert action ( $a^{1}$ ) distribution throughout the sequence, regardless of re-planning phase.

Figure 14: Success-horizon curves of policies trained on scripted non-Markovian expert data under exploratory noise injection. Left : ablation over the action-noise scale $\sigma_{u}$ , with fraction of clean trajectories fixed at $\alpha=0.8$ . Right : ablation over clean-trajectory fraction $\alpha$ , with $\sigma_{u}=\frac{1}{4}$ . We use $T_{o}=1$ in this experiment for reasons described below.

Figure 15: FurnitureSimOneLeg success-horizon curves for $T_{o}\in\{2,4,8,12,16,20\}$ for policies trained with double encoder cross attention U-Net and cross attention U-Net (no double encoder) architectures on 200 human teleop demonstrations. Both plots exhibit similar trends (shorter $T_{\text{exec}}$ perform better as $T_{o}$ increases) and comparable absolute performance, but the double encoder architecture exhibits much less variation in performance across execution horizons, producing smoother and more interpretable curves.

Figure 16: Success-horizon curves from long-context policies trained in PushT-M .

Figure 17: Distribution of the optimal training duration (calculated as the fraction of the highest-performing checkpoint’s epoch divided by the latest evaluated checkpoint’s epoch) as a function of execution horizon , pooled across all short-context ( $T_{o}=2$ ) scripted- and human-expert experiments (individual points represent individual experiments; black diamonds/bars represent median and interquartile range). We observe a consistent trend across almost all experiments that lower execution horizons prefer earlier checkpoints.

Figure 18: Distribution of the optimal training duration as a function of context length for all FurnitureSimOneLeg , SinglePillDispense , and SlipIntoBaggie context length experiments (including all execution horizons). We observe policies with longer context lengths often perform better at earlier training checkpoints.

Explore similar work

Measuring the Stability Assumption Behind Action Chunking
Oct 1, 2026Aryan GoyalImitation Learning Action Chunking
Independent Researcher
HiPolicy: Hierarchical Multi-Frequency Action Chunking for Policy Learning
Apr 7, 2026Jiyao Zhang, Zimu Han, Junhan Wang +7Policy Learning Action Chunking
CFCS, School of CS, PKU · National Key Laboratory for Multimedia Information Processing, School of CS, PKU · XJTU +3
Does Imitation Learning Preserve Temporal Robustness in Dexterous Manipulation? An Expert-Learner Comparison Across Task Execution Speeds
Sep 1, 2026Clinton Enwerem, John S. Baras, Calin BeltaContact-Rich Robotic Manipulation Robotic Policy Evaluation
Institute for Systems Research University of Maryland College Park, MD, U.S.A.

Oct 1, 2026J/K move · Enter open · S save

Measuring the Stability Assumption Behind Action Chunking

Aryan Goyal

Independent Researcher

Imitation Learning Action Chunking

Open paper AskPDF arXiv

Action chunking improves the performance of policies learned by behavioural cloning, and several mechanisms have been proposed to explain why, including temporal consistency, horizon reduction, representation learning, and reduced error compounding. We instead study what happens to an action error once it enters the system. At each state, we inject a small action error and measure how fast it grows or shrinks under two execution regimes: open-loop, where the rest of the chunk is replayed without replanning, and closed-loop, where the policy replans after the perturbation. The fitted rate labels each state as contracting, expanding, or unresolved. Across twelve manipulation tasks from three benchmark suites, we find that confidently stable states are rare, while error amplification is common among states whose propagation rate can be resolved. We further find that the measured propagation rate depends strongly on the fitting horizon: amplification is typically front-loaded, so short windows can overestimate longer-horizon propagation. Finally, we train predictors on these labels and find that a state's open-loop regime can be recovered from camera frames and proprioception alone, while its closed-loop propagation is only partially recoverable because it also depends on how the policy acts after the perturbation. These results suggest that error-compounding arguments alone do not provide a complete account of action chunking: neither passive open-loop dynamics nor policy replanning consistently contracts an injected error, and replanning rarely turns open-loop amplification into confident contraction. This suggests that closed-loop reactivity should be trained explicitly, using perturbation- and tree-coverage-oriented training to expose policies to deviations they must recover from, rather than expected to emerge reliably from standard imitation learning.

Abstract

Explore similar work

Measuring the Stability Assumption Behind Action Chunking

HiPolicy: Hierarchical Multi-Frequency Action Chunking for Policy Learning

Does Imitation Learning Preserve Temporal Robustness in Dexterous Manipulation? An Expert-Learner Comparison Across Task Execution Speeds