cs.RONov 11, 2026

Fast Inference-Time Policy Refinement via Action-Affine World Models

Authors: Run Wang, Yilin Mo, Liang Wu

Organizations: Department of Automation, Tsinghua University, Beijing, China · Johns Hopkins University, Baltimore, MD 21218, USA

Abstract

Imitation policies can perform complex manipulation, yet small execution errors can accumulate and cause failure. Refinement can correct these errors, but existing approaches often require additional on-policy learning or costly online planning. Our key insight is that a pretrained policy already supplies a useful nominal behavior: refinement can focus on predicting how small action corrections change its outcome. We therefore learn this local response in a form that is cheap to optimize. Our action-affine world model (A\textsuperscript{2}WM) learns local action sensitivities offline from action-perturbed policy rollouts. Given interaction history and a policy proposal, it predicts task-relevant geometry as a nominal trajectory plus a linear response to corrections. It predicts the full planning horizon directly, avoiding the propagation of intermediate errors through recursive model evaluations. Quadratic penalties on predicted task errors and linear correction constraints then yield a cheap convex quadratic program (QP). Online refinement requires one model evaluation and one QP solve, while both policy and model weights remain frozen. Across four simulated manipulation tasks, our framework achieves the highest success rate among evaluated baselines, adding only 3.8 ms of mean refinement overhead per control step, excluding base-policy sampling.

Figures & tables

Explore similar work

CardsList
  1. Refine Actions If Needed: Adaptive Action Chunk Refinement for Robot Manipulation

    Oct 14, 2026Hanlin Liu, Zichen Zhou, Jiacheng Zhang +6Robot Policy LearningRobotic Manipulation

  2. R2R^2-WAM: Repair-and-Reject Post-Training for World Action Models

    Oct 4, 2026Ruiyan Xu, Haisheng Su, Sixu Lin +4World Action ModelsAction-Conditioned World Models

  3. Enhancing Policy Learning with World-Action Model

    Mar 30, 2026Yuci Han, Alper YilmazRobot Policy LearningWorld Action Models