cs.ROMar 12, 2026
SaveDeployment-Time Reliability of Learned Robot Policies
Organizations: THE DEPARTMENT OF COMPUTER SCIENCE AND THE COMMITTEE ON GRADUATE STUDIES OF STANFORD UNIVERSITY
Abstract
Recent advances in learning-based robot manipulation have produced policies with remarkable capabilities. Yet, reliability at deployment remains a fundamental barrier to real-world use, where distribution shift, compounding errors, and complex task dependencies collectively undermine system performance. This dissertation investigates how the reliability of learned robot policies can be improved at deployment time through mechanisms that operate around them. We develop three complementary classes of deployment-time mechanisms. First, we introduce runtime monitoring methods that detect impending failures by identifying inconsistencies in closed-loop policy behavior and deviations in task progress, without requiring failure data or task-specific supervision. Second, we propose a data-centric framework for policy interpretability that traces deployment-time successes and failures to influential training demonstrations using influence functions, enabling principled diagnosis and dataset curation. Third, we address reliable long-horizon task execution by formulating policy coordination as the problem of estimating and maximizing the success probability of behavior sequences, and we extend this formulation to open-ended, language-specified tasks through feasibility-aware task planning. By centering on core challenges of deployment, these contributions advance practical foundations for the reliable, real-world use of learned robot policies. Continued progress on these foundations will be essential for enabling trustworthy and scalable robot autonomy in the future.
Figures & tables
Figure 2.1 : We present Sentinel , a runtime monitor that detects unknown failures of generative robot policies at deployment time. Constructing Sentinel requires only a set of successful policy rollouts and a description of the task, from which it detects diverse failures by monitoring (a) the temporal consistency of action-chunk distributions generated by the policy and (b) the task progress of the robot(s) through video QA with Vision-Language Models. More details can be found on the Sentinel website: https://sites.google.com/stanford.edu/sentinel .
Figure 2.2 : Action sequence prediction overlap during policy rollout.
Figure 2.3 : Overview of Sentinel. The images depict a policy rollout for timesteps . Temporal Consistency Detector: At each timestep , the state is passed to the generative policy to obtain action distributions between which statistical distances are computed to measure temporal consistency. The statistical distances are summed up to the current timestep (as in Eq. 2.1 ) and thresholded by to detect policy failure. Vision-Language Model (VLM) Detector: The VLM classifies whether the policy is failing to make progress on its task given a video up to timestep and a description of the task. Execution stops if either detector raises a warning.
Figure 2.4 : Temporal consistency scores grow faster when the policy fails. Error bars indicate the 5-th and 95-th score quantiles.
Figure 2.5 : Detecting failures in PushT. Left: Our failure detector ( STAC ) which measures the temporal consistency of a generative policy outperforms several families of out-of-distribution detectors. Right: The best performance comes from measuring temporal consistency with statistical distance functions; augmenting baselines with temporal consistency does not always increase their performance.
Figure 2.6 : Detecting task progression failures. Combining VLMs with STAC yields an accurate overall detector ( Sentinel ) for both task progression and erratic failures ( Table 2.1 ). See Section A.3.2 for extended results and analysis.
Figure 3.1 : We present CUPID , a robot data curation method that leverages influence functions to predictively answer counterfactual questions about the effect of each demonstration on downstream policy performance. More details can be found on the CUPID website: https://cupid-curation.github.io .
Figure 3.2 : Data curation with Cupid . Upon training a policy on a set of demonstrations using behavior cloning, we evaluate it online to collect closed-loop rollout trajectories and estimate the policy’s expected return. Cupid ranks demonstration based on their measured influence on this performance estimate and selects the top- . Thus, curating with Cupid results in a dataset of demonstrations that most strongly influences closed-loop policy success.
Figure 3.3 : RoboMimic mixed-quality curation results. Top: Data Quality. Baselines often prioritize demonstration quality (e.g., DemInf [ 109 ] ), but higher demonstration quality does always translate to higher policy success rates. In contrast, Cupid targets demonstrations that most strongly contribute to downstream policy performance. Bottom: Policy Performance. Diffusion policies trained on data curated by Cupid achieve higher success rates than baselines, despite using demonstrations of perceived lower quality. Although combining performance and quality measures ( Cupid-Quality ) yields the best policies on mixed-quality datasets, quality measures can degrade performance in other settings (see Fig. 3.4 ). Results are averaged over 3 random seeds (500 policies trained across settings). Success rates are computed over 50 rollouts from the last 10 checkpoints (500 rollouts total).
Figure 3.4 : Franka real-world diffusion policy performance. Cupid , which curates demonstrations w.r.t. policy performance, improves success rates on mixed-quality datasets, identifies robust strategies, and disentangles spurious correlations that hinder performance. Although quality measures (e.g., DemInf, Cupid-Quality ) help in mixed-quality settings (Figure-8; Fig. 3.3 ), they degrade performance when higher-quality demonstrations induce brittle strategies at test time (TuckBox), or when quality is not the primary factor limiting policy success (Bookshelf). Overall, curating data based on performance ( Cupid ) maintains robustness across these settings.
Figure 3.5 : Franka diffusion policy curated dataset distributions for filtering ( Task 1 ). Cupid filters out lower-quality demonstrations (Figure-8), brittle strategies (TuckBox), and spuriously correlated examples (Bookshelf), improving policy performance across tasks. While curation heuristics employed by baselines may be effective in some cases (e.g., DemInf and Cupid-Quality in Figure-8), they can lead to suboptimal pruning in others.
Figure 3.6 : Cupid ablation on the number of policy rollouts. Performance influences ( Eq. 3.3 ) converge with rollouts on “Lift MH” and “Square MH” (yielding similar quality trends), and rollouts on “Transport MH,” validating the practical applicability of Cupid under realistic rollout budgets. Curation is performed on diffusion policies. Results are averaged over 3 random seeds. Errors bars represent standard error.
Figure 3.7 : Data curated for single-task diffusion policies improves [ 25 ] post-training performance. Additional results in Fig. B.8 .
Figure 4.1 : Sequential manipulation tasks often contain geometric dependencies between actions. In this example, the robot needs to use the hook to pull the block into its kinematic workspace so it is close enough to pick up. The top row shows how greedy execution of skills results in the robot picking up the hook in a way that prevents it from reaching the block. We present a method for planning with skills to maximize long-horizon success without the need to train the skills on long-horizon tasks. More details can be found on the STAP website: https://sites.google.com/stanford.edu/stap . © 2023 IEEE.
Figure 4.2 : Planning in a 2D toy domain. The agent needs to get the green block under the brown receptacle with two skills: and that operate on the horizontal position of the green block. Plots (a) and (b) show the Q-functions across for each skill. is only trained to get the green block on the ground, so the planner must determine s.t. is unobstructed. The optimal action maximizes the probability of long-horizon task success ( Eq. 4.3 ), approximated by the product of Q-functions in plot (c). © 2023 IEEE.
Figure 4.3 : Left: Training pipeline. We train each skill independently on single-step environments with skill-specific rewards. We use the experience collected by each skill to train (1) a dynamics model and (2) a SCOD [ 239 ] model to predict OOD critic inputs per skill. The benefit of training each skill independently is that we can easily add skills to the library or even mix skill acquisition strategies (e.g. RL, imitation learning, and handcrafted skills). Our planning framework ensures that the skills can be composed to solve any long-horizon task even if the skills were not explicitly trained to perform those tasks. Right: Example evaluation tasks. We evaluate our method on 9 tasks: 3 Hook Reach tasks, where the robot needs to use the hook to bring objects closer, 3 Constrained Packing tasks, where the robot needs to place blocks on the rack, and 3 Rearrangement Push tasks, where the robot needs to remove obstacles to push a target block under the rack. The 9 tasks feature a range of geometric complexities and plan skeleton lengths. © 2023 IEEE.
Figure 4.4 : A small-scale experiment comparing the performance of our method to Oracle planning. The left plot shows the average success rates across two domains ( Hook Reach and Constrained Packing ). The dark bars indicate the ground truth task success, and the light bars indicate sub-goal completion rate, which measures how close the plan was to successfully completing the task. The predicted task success computed from the product of Q-values is indicated by a dotted line. Our method with Policy CEM is able to nearly match the success rate of Oracle while taking 4 orders of magnitude less time, as shown in the plot on the right. © 2023 IEEE.
Figure 4.5 : Planning experiment with 3 domains, each with 3 tasks. Our method with Policy CEM is able to generalize to all of these tasks without ever seeing them during training. On 6 out of the 9 tasks, our method either matches or outperforms DAF-Skills , which is trained directly on the evaluation task. DAF-Gen shows the generalization performance of the DAF-Skills models when evaluated on unseen tasks within the same domain. © 2023 IEEE.
Figure 4.6 : Integration of our planning framework with task planning and UQ to solve TAMP problems. The poor performance of the methods without SCOD in the Constrained Packing problem highlights the importance of UQ when attempting to solve unseen TAMP problems with learned skills. © 2023 IEEE.
Figure 5.1 : To carry out the instruction “get two primary-colored objects onto the rack,” the robot must apply symbolic reasoning over the scene description and language instruction to deduce what skills should be executed to acquire a second primary-colored object, after noticing that a red object is already on the rack (i.e. on(red, rack) ). It must also apply geometric reasoning to ensure that skills are sequenced in a manner that is likely to succeed. Unlike prior work [ 6 , 118 ] that myopically executes skills at the current timestep, Text2Motion constructs sequences of skills and coordinates their geometric dependencies with geometric feasibility planning [ 3 ] . Upon planning the skill sequence Pick(hook) , Pull(blue, hook) , Pick(blue) , Place(blue, rack) , our method computes a grasp position on the hook that enables pulling the blue object into the robot workspace so that it can be successfully picked up in the next step. More details can be found on the Text2Motion website: https://sites.google.com/stanford.edu/text2motion . © 2023 Springer Nature.
Figure 5.2 : shooting and greedy-search planning overview . Both shooting and greedy-search planners use the LLM to predict the set of valid goal states given the user’s natural language instruction and a description of the current state of the environment. These predicted goals are used to decide when the instruction is satisfied and planning can terminate. Left: The shooting strategy uses the LLM to propose full skill sequences first and then runs geometric feasibility planning afterwards. As shown in the experiments, this approach fails when the space of candidate task plans is large but few skill sequences are geometrically feasible. Right: In the greedy-search strategy, the LLM is used to propose candidate skills with the top LLM scores. The geometric feasibility planner then evaluates the feasibility of each candidate skill, and the one with the highest product of LLM and geometric feasibility scores is selected. The successor state of this skill is predicted by the geometric feasibility planner’s dynamics model. If the successor state does not satisfy any of the predicted goals, then it is given to the LLM to plan the next skill. If a goal is satisfied, then the planner returns the skill sequence for execution. By interleaving LLM task planning with geometric feasibility planning at each planning iteration, greedy-search is able to reliably find feasible plans across the different families of tasks we study in the experiments. © 2023 Springer Nature.
Figure 5.3 : Proposed hybrid planner . After predicting goals for a given instruction, Text2Motion iterates the process: i) invoke shooting to plan full skill sequences, and if no goal-reaching plan is found, ii) take a greedy-search step and check if executing the selected “best” skill would reach the goal. Note that the entire planning process occurs before execution. See Fig. 5.2 for a visualization of the shooting and greedy-search planners. © 2023 Springer Nature.
Figure 5.4 : TableEnv manipulation evaluation task suite . We evaluate the performance of all methods on tasks based on the above manipulation domain. The tasks considered vary in terms of difficulty and each task contains a subset of three properties: being long horizon (Tasks 1, 2, 3, 5, 6), containing lifted goals (Tasks 4, 5, 6), and having partial affordance perception (Tasks 4, 5, 6). During evaluation, we randomize the geometric parameters of each task. © 2023 Springer Nature.
Figure 5.5 : Results on the TableEnv manipulation domain with 10 random seeds for each task. Top: Our method (Text2Motion) significantly outperforms all baselines on tasks involving partial affordance perception (Task 4, 5, 6). For tasks without partial affordance perception, the methods that use geometric feasibility planning (Text2Motion, shooting , greedy-search ) convincingly outperform the methods ( saycan-gs and innermono-gs ) that do not. We note that shooting performs well on the tasks without partial affordance perception as it has the advantage of outputting multiple goal-reaching candidate plans and selecting the one with the highest execution success probability. Bottom: Methods without geometric feasibility planning tend to have high sub-goal completion rates but very low success rates. This divergence arises because it is possible to make progress on tasks without resolving geometric dependencies in the earlier timesteps; however, failure to account for geometric dependencies results in failure of the overall task. © 2023 Springer Nature.
Figure 5.6 : Failure modes of language-based planners on two categories of tasks . In this plot, we analyse the various types of failure modes that occur with Text2Motion, shooting and greedy-search when evaluated on tasks with partial affordance perception (PAP; see Section 5.5.4 for an explanation) and tasks without partial affordance perception (non-PAP). Top : For the PAP tasks, shooting incurs many planning failures because the space of possible plans is large but only few can be feasibly executed. In contrast, greedy-search uses value functions during search to narrow down the space of plans to those that are feasible. Text2Motion relies on greedy-search as a fallback if shooting fails, and thus can also contend with PAP tasks. Bottom : For the non-PAP tasks, shooting outperforms greedy-search . We attribute this difference to shooting ’s ability to output multiple task plans while greedy-search can only output a single plan. Finally, Text2Motion matches the performance of shooting as it also outputs and selects among multiple task plans. © 2023 Springer Nature.
| Hybrid planning breakdown | Task 4 | Task 5 | Task 6 |
|---|---|---|---|
| % shooting only | 14% | 0% | 0% |
| % greedy-search only | 0% | 0% | 0% |
| % Combination | 86% | 100% | 100% |
| Avg. greedy-search Steps | 1.0 | 2.6 | 3.0 |
| Avg. Plan Length | 5.0 | 7.0 | 7.0 |
Table 5.1 : Ablation on hybrid planning method. We analyze the usage percentages of both shooting and greedy-search in successful plans found by our hybrid planner (see Fig. 5.3 ). We find that, as tasks increase in difficulty (Task 4, 5, 6), the majority of solutions involve a combination of both planners. This result indicates that shooting-based and search-based planning strategies play complementing roles in the success of Text2Motion. © 2023 Springer Nature.
Figure 5.7 : Ablation on termination method: goal proposition prediction vs stop scoring . We compare the performance of greedy-search using two different plan termination methods: using the LLM to predict goals a priori (ours) and scoring a stop skill [ 6 ] during search. We present results averaged across all six tasks and ten seeds for each variation (120 experiments in total). We find that terminating planning when LLM-predicted goals are satisfied results in a 10% boost in success rate over stop scoring. © 2023 Springer Nature.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A.1 : Evaluation Domains. We evaluate our failure detection framework across three simulation domains and one real-world domain. These domains provide coverage over different data distributions (e.g., action multimodality, high-dimensional actions) and modes of generative policy failure. For example, generative policies tend to fail erratically in the Close Box domain, but smoothly in the Cover Object domain. An effective failure detector should be performant across multiple domains, which entails coverage over heterogeneous failure modes.
Figure A.2 : Performance variation of STAC subject to different policy prediction and execution horizons in PushT .
Figure A.3 : Performance variation of STAC based on the choice of statistical distance function in PushT .
Figure B.1 : RoboMimic state ablation: Data quality trends under varying number of rollouts. Performance influences ( Eq. 3.3 ) converge around rollouts for “Lift MH” and “Square MH” (yielding similar quality trends), but continue to evolve until rollouts for “Transport MH.” Curation performed on state-based diffusion policies. Results are averaged over 3 random seeds. Errors bars represent the standard error.
Figure B.2 : RoboMimic image ablation: Data quality trends under varying number of rollouts. Performance influences ( Eq. 3.3 ) converge around rollouts for “Lift MH” and “Square MH” (yielding similar quality trends), but continue to evolve until rollouts for “Transport MH.” Curation performed on image-based diffusion policies. Results are averaged over 3 random seeds. Errors bars represent the standard error.
Figure B.3 : RoboMimic state data quality results. Curation performed on state-based diffusion policies. Results are averaged over 3 random seeds. Errors bars represent the standard error.
Figure B.4 : RoboMimic image data quality results. Curation performed on image-based diffusion policies. Results are averaged over 3 random seeds. Errors bars represent the standard error.
Figure B.5 : Franka diffusion policy – distribution of demonstrations filtered ( in Task 1 ). See Fig. 3.5 for distributions of the corresponding curated datasets used for policy training.
Figure B.6 : Franka diffusion policy curated dataset distributions for selection ( Task 2 ). Cupid selects higher-quality demonstrations (Figure-8) and robust strategies (TuckBox), improving policy performance across tasks (see Fig. 3.4 ). While curation heuristics employed by baselines may be effective in some cases (e.g., Cupid-Quality in Figure-8), they can lead to suboptimal selection in others.
Figure B.7 : Franka diffusion policy – distribution of demonstrations selected ( in Task 2 ). See Fig. B.6 for distributions of the corresponding curated datasets used for policy training.
Figure B.8 : Data curated for single-task diffusion policies improves [ 25 ] post-training performance. As in Fig. 3.4 , quality measures ( Cupid-Quality ) may degrade performance when higher-quality demonstrations induce brittle strategies at test time (TuckBox), whereas curating based on performance ( Cupid ) is robust across settings.
| Task ID | Properties | Instruction |
|---|---|---|
| Task 1 | LH | How would you pick and place all of the boxes onto the rack?” |
| Task 2 | LH + LG | How would you pick and place the yellow box and blue box onto the table, |
| then use the hook to push the cyan box under the rack?” | ||
| Task 3 | LH + PAP | How would you move three of the boxes to the rack?” |
| Task 4 | LG + PAP | How would you put one box on the rack?” |
| Task 5 | LH + LG + PAP | How would you get two boxes onto the rack?” |
Table D.1: TableEnv manipulation task suite . We use the following shorthands as defined in Chapter 5 : LH: Long-Horizon, LG: Lifted Goals, PAP: Partial Affordance Perception. © 2023 Springer Nature.
More in this topic
Robotics
Robotics studies the design and control of machines that perceive, move, and act in physical environments.
See all 751 papers