ProgressCompass: Embodied Progress Reward Models Are Lost Without the Right Context

Jianshu Zhang; Keliang Wu; Chengxuan Qian; Xiyuan Yang; Ce Zhang; Ariel Tian; Anbang Liu; Haoran Lu

cs.CLSep 29, 2026

ProgressCompass: Embodied Progress Reward Models Are Lost Without the Right Context

Authors: Jianshu Zhang, Keliang Wu, Chengxuan Qian, Xiyuan Yang, Ce Zhang, Ariel Tian, Anbang Liu, Haoran Lu,

Organizations: Northwestern · UCSB · UIUC · CMU

Abstract

Embodied agents now take on ever longer tasks. For long tasks, knowing only whether a task finally succeeds or fails says little; the steps along the way matter. Progress Reward Models (PRMs) score how far a task has come at every step, and serve as dense rewards, verifiers and monitors. Yet in long tasks the current frame alone often cannot tell how far the task has come, because progress depends on what happened before. We call this problem context-dependent progress estimation. Existing benchmarks on progress estimation mostly focus on short tasks whose progress can be read from the current observation, and whether PRMs can estimate progress when context is needed remains underexplored. We therefore build ContextProgress-Bench, with 24 manipulation tasks for 120 episodes. The benchmark covers three settings: (i) State Recall, where information needed for progress appeared earlier but is not in the current frame; (ii) Sequence Tracking, where steps follow a fixed order, so progress requires knowing which steps are done and which comes next; and (iii) Recurrence Disambiguation, where look-alike frames sit at very different progress. We then run a paired diagnosis: each PRM keeps the same input format in both runs, and in one run its instruction integrates the right context. Even PRMs that read the entire history get lost in estimating progress, yet with the right context the same five models cut their progress error by 77-82%. Embodied PRMs are thus not incapable of progress estimation, but lost without the right context. We therefore propose ProgressCompass, an autonomous agentic loop that reorients an existing PRM and uses current general-purpose VLMs to supply the context the PRM needs. Wrapped in the loop, the same frozen PRM cuts its progress error by 63% and raises its rank agreement by 76%. With such a compass, PRMs estimate progress far better on longer, more complex tasks.

Figures & tables

Figure 1: Why a Progress Reward Model (PRM) needs context. Given the overall task instruction (top), the current frame alone may not suffice to estimate progress. (i) State Recall : information needed for progress, such as which mat the block came from, appeared earlier but is not in the current frame. (ii) Sequence Tracking : steps follow a fixed order, so progress needs knowing which steps are done. (iii) Recurrence Disambiguation : look-alike frames sit at different progress.

Figure 2: Statistics of ContextProgress-Bench . (a) Episodes grouped by the context forms they require. (b) Tasks, episodes and annotated steps per form. (c) Episode duration. (d) Annotated steps per episode; every episode needs context at least once.

Model	w/o context		w/ context: oracle		w/ context: self-chained
Model	MAE $\downarrow$	$\rho$ $\uparrow$	MAE $\downarrow$	$\rho$ $\uparrow$	MAE $\downarrow$	$\rho$ $\uparrow$
ProgressLM	17.95	0.74	4.18 ( $-77\%$ )	0.99 ( $+35\%$ )	16.70 ( $-7\%$ )	0.99 ( $+34\%$ )
Robo-Dopamine	15.59	0.74	3.43 ( $-78\%$ )	0.98 ( $+31\%$ )	4.66 ( $-70\%$ )	0.97 ( $+31\%$ )
RoboMeter	25.42	0.53	4.91 ( $-81\%$ )	0.93 ( $+75\%$ )	11.39 ( $-55\%$ )	0.92 ( $+74\%$ )
TOPReward	30.95	0.56	6.70 ( $-78\%$ )	0.90 ( $+62\%$ )	9.42 ( $-70\%$ )	0.89 ( $+59\%$ )
VLAC	24.43	0.72	4.50 ( $-82\%$ )	0.97 ( $+35\%$ )	17.13 ( $-30\%$ )	0.94 ( $+30\%$ )

Table 1: The Context Gap. Left: the same frozen model with the same input format, first without context, then with the correct context for each annotated subtask. The two with-context settings differ in where that context puts the estimate: oracle at the true position of its subtask, self-chained at where the model ended the previous one. ( Green values ) are the relative change from the same model w/o context. Right: each dot is one subtask of one model, at its MAE without context (

x

) and with context (

y

), coloured as on the left; dots below the diagonal are where context helped.

Figure 3: Three typical errors, one episode each. The task each panel runs on is (a) move each red block onto the green mat and back to where it came from, from left to right; (b) insert three tubes into a rack in a prescribed order; (c) press the same button eight times.

Figure 4: ProgressCompass . The Navigator

\mathcal{N}

runs the loop.

\mathcal{O}

gives the context

c_{t}

to the frozen PRM

\mathcal{P}

and the expected transition

\tau_{t}

\mathcal{V}

;

\mathcal{P}

proposes a completion and

\mathcal{V}

checks it until the step is accepted, then

\mathcal{O}

reorients.

Figure 5: Main results. Progress MAE (top, lower is better) and Spearman

\rho

(bottom, higher is better) per context form; green is the change over the frozen RoboMeter.

Figure 6: Progress curves on three episodes. Ticks mark moments that need context; vertical lines join ours (filled) and RoboMeter (hollow).

Figure 7: Negatives. Mean deviation of each method from the correct progress on Early stop, Extra steps and Mismatch; lower is better.

Figure 8: Ablating

\tau_{t}

. Error at the end of each subtask; lower is better.

Process Reward Models

Process reward models assess intermediate reasoning steps and actions to guide learning, search, and evaluation.

See all 132 papers

Jul 22, 2026

Progress Reward Modeling for Robotic Learning: A Comprehensive Survey

Robotic learning takes place in dynamic environments with large behavior spaces. A terminal success signal only tells the robot whether the task is completed. It does not explain whether the current behavior is making progress, remaining unchanged, or undoing earlier progress. For this reason, recent studies have increasingly explored progress rewards that provide feedback during task execution. However, the current literature lacks a shared framework. Existing methods use different observations, goal specifications, output signals, supervision sources, and evaluation protocols. This makes it difficult to compare them and understand what their results actually validate. In this survey, we provide a unified view of progress reward modeling for robotic learning. We organize the field in three connected steps. We first study the interface of a progress model. This defines the problem from the outside by asking what information the model receives and what form of progress signal it produces. We then move inside the model and study the methods used to construct this signal. This reveals the different assumptions and mechanisms behind progress estimation and reward generation. Finally, we examine the data and benchmarks that support these methods. This shows how progress supervision is obtained and what different evaluations actually measure. Together, these three perspectives connect what a progress model is, how it is built, and how its quality is validated. We further summarize the main limitations of current approaches and discuss future research directions.

Jianshu Zhang, Keliang Wu, Haoran Lu +8

Northwestern University · Carnegie Mellon University · University of Wisconsin–Madison +2

Reward Modeling Process Reward Models Robotic RL Robot Learning

Ask PDF

Mar 18, 2026

Recurrent Reasoning with Vision-Language Models for Estimating Long-Horizon Embodied Task Progress

Accurately estimating task progress is critical for embodied agents to plan and execute long-horizon, multi-step tasks. Despite promising advances, existing Vision-Language Models (VLMs) based methods primarily leverage their video understanding capabilities, while neglecting their complex reasoning potential. Furthermore, processing long video trajectories with VLMs is computationally prohibitive for real-world deployment. To address these challenges, we propose the Recurrent Reasoning Vision-Language Model (

\text{R}^2

VLM). Our model features a recurrent reasoning framework that processes local video snippets iteratively, maintaining a global context through an evolving Chain of Thought (CoT). This CoT explicitly records task decomposition, key steps, and their completion status, enabling the model to reason about complex temporal dependencies. This design avoids the high cost of processing long videos while preserving essential reasoning capabilities. We train

\text{R}^2

VLM on large-scale, automatically generated datasets from ALFRED and Ego4D. Extensive experiments on progress estimation and downstream applications, including progress-enhanced policy learning, reward modeling for reinforcement learning, and proactive assistance, demonstrate that

\text{R}^2

VLM achieves strong performance and generalization, achieving a new state-of-the-art in long-horizon task progress estimation. The models and benchmarks are publicly available at huggingface.

Yuelin Zhang, Sijie Cheng, Chen Li +4

Gaoling School of Artificial Intelligence, Renmin University of China · Beijing Key Laboratory of Research on Large Models and Intelligent Governance · Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE +2

VLMs for Robotics VLM Reasoning Embodied Reasoning

Ask PDF

Oct 16, 2026

Not All Actions Contribute Equally: Learning Precise Progress Estimation from Imprecise Demonstrations

Progress estimators play an essential role in providing dense reward signals for reinforcement learning in robotic manipulation, but accurately assessing task progress can be much challenging. Existing methods heavily rely on expert data and naively assume that all actions within a demonstration contribute to task success. This assumption often breaks down in real-world scenarios, especially in high-precision manipulation, where subtle geometric errors can lead to task failure. Therefore, these methods can erroneously interpret human mistakes as meaningful progress, further corrupting progress learning. We argue that a key limitation is the lack of a global prior regarding task evaluation criteria, which is crucial for distinguishing meaningful progress from human mistakes. To this end, we propose PRIME, a novel progress estimator capable of learning precise progress estimation from imprecise demonstrations by aggregating experience across trajectories. PRIME constructs a weighted graph over observed states, connecting states across trajectories based on spatial proximity and visual consistency. Edge weights encode spatial distances as transition costs, and shortest-path search approximates the geodesic distance to success. Changes in this distance naturally distinguish advancement, stagnation, and setbacks without assuming monotonic progress over time. Extensive experiments in both simulation and real world on precise manipulation tasks show that PRIME is consistently effective across diverse tasks, and robust to data with varying degrees of suboptimality. Notably, the advantage of PRIME gets more pronounced in higher-precision scenarios, yielding up to a

5\times

improvement in policy success rate over the SFT baselines on charger insertion.

Huanqian Wang, Boshi Tang, Botong Xiao +5

BNRist, Tsinghua University

RL for Robotics Robotic Manipulation Learning from Suboptimal Demonstrations

Ask PDF

Abstract

Process Reward Models

Explore similar work

Progress Reward Modeling for Robotic Learning: A Comprehensive Survey

Recurrent Reasoning with Vision-Language Models for Estimating Long-Horizon Embodied Task Progress

Not All Actions Contribute Equally: Learning Precise Progress Estimation from Imprecise Demonstrations