PACE: Phase-Progress-Aware Credit for Long-Horizon Embodied Manipulation

Chengye Song; Jiawei Zhang; Rui Song; Shengqi Wang; Xiangrong Zhang; Ziyi Wang; Huanbin Zhou; Hongzhou Wang

cs.ROAug 15, 2026

PACE: Phase-Progress-Aware Credit for Long-Horizon Embodied Manipulation

Authors: Chengye Song, Jiawei Zhang, Rui Song, Shengqi Wang, Xiangrong Zhang, Ziyi Wang, Huanbin Zhou, Hongzhou Wang

Organizations: Intelligence Science and Technology, Dalian University of Technology, No. 2 Linggong Road, Ganjingzi District, Dalian 116024, China · Jilin University, No. 2699 Qianjin Street, Changchun 130012, China

Abstract

Post-training of vision-language-action (VLA) models typically relies on expert demonstrations and policy interaction trajectories. However, in long-horizon manipulation, a single episode often spans hundreds of control steps and multiple phases, while success or failure is only revealed at episode termination. Policy improvement therefore requires step-level credit signals to distinguish behaviors that advance the task from those that stall or regress. We present PACE, a credit-assignment framework for post-training on long-horizon manipulation, centered on a phase-progress-aware critic. PACE consists of two key modules: (1) the Global-Local Cooperative Value-Correction Critic (GLC-Critic) aggregates visual and motion-difference features within local temporal windows to infer the phase and intra-phase progress of each step, and applies residual correction to a discretized remaining-cost distribution accordingly, enabling step-level credit assignment; (2) Progressive Policy Distillation (PPD) converts credit into positive and negative conditions via task-wise thresholds and trains a credit-conditioned action generation policy: it first protects the pretrained policy with high-credit positive samples, then incorporates all positive and negative credits to learn the quality boundary, and at inference amplifies high-credit behaviors through the difference between conditional outputs. Extensive simulation experiments and diverse real-world robotic-arm experiments demonstrate that PACE consistently achieves significant improvements over the strongest baseline.

Figures & tables

Figure 1: Raw and fused remaining-cost predictions on successful (top) and incomplete (bottom) trajectories.

Figure 2: Overview of PACE. Post-training trajectories supervise the phase–progress-aware GLC-Critic, which produces corrected remaining-cost estimates for credit relabeling. Progressive Policy Distillation then adapts the use of positive and negative credit as iterative data collection improves critic coverage, updating the VLA policy for simulation and real-robot deployment.

Figure 3: Temporal and phase–progress branches of the GLC-Critic. Local observations and motion differences produce the coordinates used for value correction.

Figure 4: Real-robot experimental platform with dual AgileX PiPER slave arms, bilateral master arms for teleoperation, and three camera views.

Method	Avg. SR (%) $\uparrow$	AvgT (steps) $\downarrow$	Held-out MAE $\downarrow$	Boundary MAE $\downarrow$
SFT (OpenVLA)	$52.7\pm 1.6$	$310.8\pm 78.4$	—	—
SFT (XR-0)	$57.2\pm 1.3$	$298.7\pm 75.1$	—	—
SFT ( $\pi_{0.5}$ )	$63.3\pm 1.3$	$276.9\pm 68.3$	—	—
ReCAP	$73.8\pm 1.8$	$254.26\pm 63.54$	133.03	136.02
PACE	$83.3\pm 2.5$	$252.55\pm 69.07$	97.78	77.01

Table 1: Main LIBERO-Long results. Critic errors use inverse-mapped return units; Boundary MAE uses

\pm 10

-step transition neighborhoods.

Panel of Figure 5 — Figure 5: Task-wise LIBERO-Long SR (a; higher is better) and Boundary MAE (b; lower is better).

Critic Variant	SR (%) $\uparrow$	Held-out MAE $\downarrow$	Boundary MAE $\downarrow$
Raw Critic	$73.8\pm 1.8$	133.03	136.02
w/o temporal aggregation	$81.6\pm 2.3$	99.31	78.03
w/o motion differences	$80.9\pm 2.2$	100.01	79.43
w/o phase head $z$	$81.2\pm 2.2$	100.17	78.73
w/o progress head $p$	$82.7\pm 2.4$	98.03	77.23
Full GLC-Critic	$83.3\pm 2.5$	97.78	77.01

Table 2: GLC-Critic ablation under the protocol of Table 1 .

Simulation
Update (shared init.)	PR-CFG SR (%) $\uparrow$	FACD SR (%) $\uparrow$
Round 1 on $\Delta_{1}$ (base, $44.2$ )	$54.5\pm 2.0^{*}$	$49.0\pm 2.5$
Round 2 on $\Delta_{2}$ (same R1 PR-CFG, $54.5$ )	$61.0\pm 2.0$	$64.5\pm 1.5^{*}$
Real Robot
Round 1 on $\Delta_{1}$ (base, $16.4$ )	$32.9\pm 1.9^{*}$	$22.2\pm 2.3$
Round 2 on $\Delta_{2}$ (same R1 PR-CFG, $32.9$ )	$49.6\pm 2.1^{*}$	$41.3\pm 0.7$

Table 3: Controlled PPD ablation with shared initialization, data, critic predictions, and update budget. ∗ marks a significant paired two-sided Wilcoxon difference over task–evaluation blocks after Holm correction across the six rows.

Method	Task 1		Task 2		Task 3		Held-out MAE $\downarrow$	Boundary MAE $\downarrow$
	SR (%) $\uparrow$	AvgT (s) $\downarrow$	SR (%) $\uparrow$	AvgT (s) $\downarrow$	SR (%) $\uparrow$	AvgT (s) $\downarrow$
SFT (OpenVLA)	$40.7\pm 1.2^{*}$	$89.7\pm 1.7$	$35.3\pm 4.6^{*}$	$84.8\pm 3.2$	$16.7\pm 3.1^{*}$	$90.4\pm 3.5$	—	—
SFT (XR-0)	$38.0\pm 2.0^{*}$	$83.8\pm 10.9$	$36.0\pm 3.5^{*}$	$82.9\pm 6.6$	$30.7\pm 5.0^{*}$	$77.1\pm 6.1$	—	—
SFT ( $\pi_{0.5}$ )	$54.7\pm 2.3^{*}$	$78.2\pm 3.6$	$51.3\pm 2.3^{*}$	$76.7\pm 3.1$	$47.3\pm 4.6^{*}$	$74.9\pm 4.5$	—	—
ReCAP	$62.7\pm 3.1^{*}$	$72.5\pm 4.3$	$70.7\pm 3.1^{*}$	$76.9\pm 3.9$	$66.0\pm 6.0^{*}$	$79.5\pm 3.3$	68.17	58.73
PACE (ours)	$84.7\pm 4.2$	$53.0\pm 9.8$	$83.3\pm 2.3$	$58.9\pm 16.6$	$77.3\pm 1.2$	$64.3\pm 11.5$	18.89	4.12

Table 4: Task-wise real-robot results. AvgT is successful-trial duration in seconds. Tasks 1–3 are Table Wiping, Towel Folding, and Mug Placement and Knob Turning. For SR, ∗ indicates that PACE significantly outperforms the corresponding baseline on that task (two-sided Fisher exact test on 150 pooled trials per method, Holm-adjusted

p\leq 0.04

Figure 6: Real-robot task-wise SR over three repeated evaluations; higher is better.

Step-Level Credit Assignment in RL

Step-level credit assignment in reinforcement learning links delayed rewards to the actions or reasoning steps that produced them.

See all 165 papers

PACE: Stage-Consistent Long-Horizon Robot Manipulation via Progress-Aligned Context for Execution
Oct 6, 2026Yenan Chen, Junjie Shi, Lu Chen +2Long-Horizon Robotic Manipulation Robotic Manipulation
College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China
PACE: Phase-Aware Chunk Execution for Robot Policies with Action Chunking
May 30, 2026Junnan Nie, Jiayi Li, Chenghao Liu +5Robotic Manipulation Action Chunking
Peking University · Peking University. · JD Explore Academy. +1
AtomicRL: Atomic Reinforcement Learning for Long-Horizon Robotic Manipulation
Oct 21, 2026Likui Zhang, Zhe Zhu, Zhihao Zhan +7RL for Robotics Long-Horizon Robotic Manipulation
Sun Yat-sen University · JD Explore Academy · The Hong Kong Polytechnic University

Oct 6, 2026J/K move · Enter open · S save

PACE: Stage-Consistent Long-Horizon Robot Manipulation via Progress-Aligned Context for Execution

Yenan Chen, Junjie Shi, Lu Chen, Zhongxiang Zhou+1

College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China

Long-Horizon Robotic Manipulation Robotic Manipulation Robot Imitation Learning

Open paper AskPDF arXiv

Demonstration-conditioned policies provide a natural interface for specifying robot behavior, yet long-horizon manipulation remains difficult when visually similar states recur across different stages or when demonstrations and executions proceed at different speeds. We identify the resulting failure mode as stage confusion and introduce Progress-Aligned Context for Execution (PACE), a stateful method that continually reinterprets a complete demonstration according to realized execution progress. PACE compresses the demonstration into ordered multimodal prompt tokens and uses training-only dual-edge attention supervision to expose its latent stage structure. During execution, an episode-local fast-weight memory causally encodes realized action-observation transitions and modulates prompt cross-attention, producing a progress-aligned context for a unified diffusion action expert without test-time stage labels or stage-specific policies. PACE improves success from 88.9% to 94.0% on LIBERO-Gen Goal Chain, from 79.1% to 83.3% on Spatial Combination, and from 33.3% to 73.3% on the two-step Block Routing tasks. Failure analysis further indicates that structured demonstration alignment and causal execution memory jointly mitigate stage confusion.

Abstract

Step-Level Credit Assignment in RL

Explore similar work

PACE: Stage-Consistent Long-Horizon Robot Manipulation via Progress-Aligned Context for Execution

PACE: Phase-Aware Chunk Execution for Robot Policies with Action Chunking

AtomicRL: Atomic Reinforcement Learning for Long-Horizon Robotic Manipulation