cs.RONov 7, 2026

GUARD: Anomaly-Informed Q-Steering for Generative Robot Policies in Dexterous Manipulation

Authors: Aryaman Gupta, Xiaoyue Wu, Rana Soltani Zarrin, Mark Van der Merwe

Organizations: Stanford University. · Honda Research Institute USA.

Abstract

Generative robot policies based on diffusion and flow matching can model complex, multimodal behaviors, but remain susceptible to covariate shift during closed-loop execution. Existing approaches mitigate such failures through additional demonstrations, policy fine-tuning, or test-time guidance, but often require human intervention, engineered rewards, or other task-specific signals. We introduce GUARD, an anomaly-informed Q-steering framework that leverages the policy's training distribution to construct a dense, failure-anticipatory steering signal. GUARD pairs sparse task rewards with dense anomaly scores learned from policy training data and propagates both through a learned Q-function. Its gradients then steer the denoising process toward actions that improve task return while preventing trajectories from entering poorly supported regions, without directly modifying the pretrained policy. Across four Adroit dexterous manipulation tasks, GUARD achieves average success rates of 96.0%96.0\% and 80.0%80.0\% under nominal and perturbed dynamics, outperforming the strongest baseline by 21.021.0 and 20.020.0 percentage points while maintaining the lowest anomaly score. GUARD further demonstrates direct sim-to-real transfer on in-hand screwdriver rotation and reorientation tasks, achieving 4040 and 2525 percentage-point gains in success rate. These results demonstrate that anomaly signals can serve not only for failure monitoring but also to complement sparse task rewards as effective supervision for robust policy steering.

Figures & tables

Explore similar work

CardsList
  1. PPGuide: Steering Diffusion Policies with Performance Predictive Guidance

    Mar 11, 2026Zixing Wang, Devesh K. Jha, Ahmed H. Qureshi +1Diffusion GuidanceDiffusion Policy

  2. Steering Generative Reinforcement Learning into Stable Robotic Controller

    Jun 15, 2026Yixuan Wang, Shutong Ding, Ke Hu +3Latent Action LearningRobotic RL

  3. Proxy Policy Steering

    Sep 8, 2026Chuanruo Ning, Tianrui Wang, Wei-Chiu Ma +1Robot Policy AdaptationRobot Foundation Models