Referring-Aware Visuomotor Policy Learning for Closed-Loop Manipulation

Jiahua Ma; Yiran Qin; Xin Wen; Yixiong Li; Yuyu Sun; Yulan Guo; Liang Lin; Ruimao Zhang

cs.ROApr 7, 2026

Referring-Aware Visuomotor Policy Learning for Closed-Loop Manipulation

Authors: Jiahua Ma, Yiran Qin, Xin Wen, Yixiong Li, Yuyu Sun, Yulan Guo, Liang Lin, Ruimao Zhang

Organizations: Sun Yat-sen University · Oxford University.

Abstract

This paper addresses a fundamental problem of visuomotor policy learning for robotic manipulation: how to enhance robustness in out-of-distribution execution errors or dynamically re-routing trajectories, where the model relies solely on the original expert demonstrations for training. We introduce the Referring-Aware Visuomotor Policy (ReV), a closed-loop framework that can adapt to unforeseen circumstances by instantly incorporating sparse referring points provided by a human or a high-level reasoning planner. Specifically, ReV leverages the coupled diffusion heads to preserve standard task execution patterns while seamlessly integrating sparse referring via a trajectory-steering strategy. Upon receiving a specific referring point, the global diffusion head firstly generates a sequence of globally consistent yet temporally sparse action anchors, while identifies the precise temporal position for the referring point within this sequence. Subsequently, the local diffusion head adaptively interpolates adjacent anchors based on the current temporal position for specific tasks. This closed-loop process repeats at every execution step, enabling real-time trajectory replanning in response to dynamic changes in the scene. In practice, rather than relying on elaborate annotations, ReV is trained only by applying targeted perturbations to expert demonstrations. Without any additional data or fine-tuning scheme, ReV achieve higher success rates across challenging simulated and real-world tasks.

Figures & tables

Figure 1: The role of the proposed Referring-Aware Visuomotor Policy (ReV) in mitigating out-of-distribution failures. In executing the task “Grabbing the steak from the kitchen table” , several challenges can be encountered. Case A : a small covariate shift at $t_{k}$ quickly leads to a compounding misalignment with expert demonstrations, driving the robot into unseen states and resulting in task failure. Case B : an unforeseen obstacle ( e . g ., a pot) emerges at $t_{k}$ to dynamically block the robot’s path, leading to failure as well. Unlike traditional imitation learning methods that often struggle to generalize to unseen states or observations, the proposed ReV employs coupled diffusion heads to online react to external sparse referring provided by a human or a high-level reasoning planner. Without requiring additional training data or complex fine-tuning in post-processing, the model can effectively address Case A and Case B in real-world applications.

Figure 2: Overview of our Referring-Aware Visuomotor Policy. Given the observation $\mathcal{O}$ and robot proprioception history $\mathcal{A}_{-}$ , the Temporal-Position Prediction Module on the left assign the optimal temporal position $k$ for the referring point $\mathcal{P}$ . This temporally-positioned referring point $\mathcal{P}$ is then transformed into a temporally-positioned partially-constrained referring action $\mathcal{P}^{a}$ (detailed in Sec. 3.3 ) and injected into the right Policy Model with Coupled Diffusion Heads to generate the final trajectory $\mathcal{A}$ satisfying the Manipulation Steering Choreography . Specifically, at step $i$ , the GDH incorporates both anchor history $\mathcal{A}^{\prime}_{-}$ and partially-constrained referring action $\mathcal{P}^{a}$ via the trajectory-steering strategy to yield a sparse yet globally consistent anchor sequence $\mathcal{A}^{\prime}=\{a^{\prime}_{j}\}_{j=1}^{N_{1}}$ , in which $\{a^{\prime}_{j}\}_{j=1}^{i-1}$ is anchor history $\mathcal{A}^{\prime}_{-}$ , $\{a^{\prime}_{j}\}_{j=i}^{i+1}$ is interpolated by the LDH through a learnable, temporal-dependent interpolation strategy, and $\{a^{\prime}_{j}\}_{j=i+1}^{N_{1}}$ is discarded and will be updated at upcoming inference step. At inference step $i\!+\!1$ , the anchor history $\mathcal{A}^{\prime}_{-}$ is updated by appending the newly generated anchor $a_{i+1}^{\prime}$ at step $i$ , and the entire policy model repeats, continuously updating $\mathcal{A}$ .

Figure 3: Temporal-Position Prediction . The slots buffer $\mathcal{S}$ covers both retained history and extrapolated future steps. Specifically, at step $i$ , we load the historical action anchors $\{a^{\prime}_{j}\}_{j=1}^{i}$ to initialize history part, and the future part is padded by copies of the last action anchor $a^{\prime}_{i}$ . Then, the whole slots buffer is augmented with a monotonically-increasing temporal-position embeddings.

Method	Pick Meat-via			Lift Barrier-via			Place Food-via			Camera Alignment-via
Method	RePR( $\uparrow$ )	SuR( $\uparrow$ )	SmS( $\uparrow$ )	RePR( $\uparrow$ )	SuR( $\uparrow$ )	SmS( $\uparrow$ )	RePR( $\uparrow$ )	SuR( $\uparrow$ )	SmS( $\uparrow$ )	RePR( $\uparrow$ )	SuR( $\uparrow$ )	SmS( $\uparrow$ )
ACT	2%	1%	0.9890	1%	1%	0.9904	0%	0%	-	0%	0%	-
DP3	80%	1%	0.9899	99%	25%	0.9945	1%	1%	0.9883	0%	0%	-
CDP	14%	14%	0.9924	99%	99%	0.9933	47%	33%	0.9878	0%	0%	-
OCTO	18%	9%	0.9606	32%	32%	0.9702	1%	1%	0.9597	0%	0%	-
MPD	20%	3%	0.9903	39%	39%	0.9949	3%	3%	0.9887	1%	1%	0.9861

Table 1: Quantitative results on modified simulated benchmark , highlighting the effectiveness of ReV in referring-aware manipulation.

Figure 4: Visualization of the trajectories generated by ReV on Pick-Meat-via and Lift-Barrier-via . Here, we use green bounding boxes to mark the frames in which the end-effector passes through the designated via-point (green ball).

Method	0.1		0.2		0.3		0.4
Method	RePR( $\uparrow$ )	SuR( $\uparrow$ )	RePR( $\uparrow$ )	SuR( $\uparrow$ )	RePR( $\uparrow$ )	SuR( $\uparrow$ )	RePR( $\uparrow$ )	SuR( $\uparrow$ )
ReV	100%	93%	100%	92%	100%	89%	100%	87%

Table 2: Ablation study on OOD-yet-feasible referring points. 0.1, 0.2, 0.3, and 0.4 indicate the degree of deviation from the center of the expert distribution. Details are provided in Appendix B.4 .

Method	Adroit		DexArt			MetaWorld				RoboFactory
Method	Pen	Door	Laptop	Toilet	Bucket	Reach	Soccer	Sweep Into	Shelf Place	Pick Meat	Lift Barrier	Place Food	Camera Alignment
ACT	47%	66%	35%	8%	6%	21%	28%	23%	43%	91%	40%	13%	21%
DP3	53%	69%	81%	65%	32%	28%	30%	15%	37%	81%	90%	30%	87%
CDP	68%	74%	84%	68%	32%	22%	23%	24%	35%	84%	93%	51%	90%
ReV	73%	79%	87%	71%	61%	36%	33%	27%	47%	94%	99%	57%	94%

Table 3: Quantitative results across simulated benchmarks , highlighting the effectiveness of our Coupled Diffusion Heads architecture.

Method	Single-Agent						Dual-Agent
	Collecting Objects-via		Push T-via		Stacking Playing Card-via		Grabbing Rod-via		Handing Eraser Over-via
	PeRP $\uparrow$	SuR $\uparrow$	PeRP $\uparrow$	SuR $\uparrow$	PeRP $\uparrow$	SuR $\uparrow$	PeRP $\uparrow$	SuR $\uparrow$	PeRP $\uparrow$	SuR $\uparrow$
ACT	3 / 30	1 / 30	5 / 30	3 / 30	3 / 30	0 / 30	2 / 30	0 / 30	3 / 30	1 / 30
DP	6 / 30	5 / 30	12 / 30	9 / 30	9 / 30	3 / 30	3 / 30	1 / 30	6 / 30	2 / 30
ReV	30 / 30	20 / 30	30 / 30	21 / 30	30 / 30	15 / 30	30 / 30	18 / 30	30 / 30	12 / 30

Table 4: Quantitative results on real-world tasks , highlighting the robustness of our ReV in real-world settings.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Demonstration Construction Strategy for GDH and LDH . The orange blocks represent the sparse action anchors. Among them, the lighter orange exemplifies the historical context, while the darker orange exemplifies the future targets. The green blocks denote the fine-grained interpolation actions.

Figure 6: Via-Points Generation. Via-points are generated via Gaussian sampling around a centroid derived from the robot’s initial configuration.

Panel of Figure 7 — Figure 7: Visualization of ablation studies on key hyperparameters. (a) Total trajectory length $N$ : performance peaks when $N$ matches task complexity, enabling the model to capture task-specific execution patterns. (b) Trajectory-length allocation $N_{1}\!:\!N_{2}$ : balanced ratios ( $1\!:\!2\sim 2\!:\!1$ ) allow our policy model to learn the entire trajectory most effectively. (c) Update Frequencies of T-P prediction : excessive updates cause anchor drift and degrade performance. Here, success rate is defined as the fraction of trials that simultaneously achieve task completion and pass through the designated referring point, and all ablations were conducted under controlled variables.

Method	Inside Camera	Inside Pot	Under Table	Out of Reach
DP3	0%	0%	0%	0%
CDP	0%	11%	0%	0%
ReV	100%	100%	0%	0%

Appendix

Table 5: Ablation study on infeasible referring points.

Figure 8: OOD-yet-Feasible Referring Points Generation.

Benchmark	Parameter	Value
Adroit	Demonstrations Number	10
	Size of Point Clouds	(512, 3)
	Size of Images	(84, 84, 3)
	Batch Size	32
	Epoch	3000
Dexart	Demonstrations Number	100

Appendix

Table 6: All Training Settings for simulation experiments.

Benchmark	Task	$N$	$N_{1}$	$N_{2}$
Adroit	Door	54	6	12
Adroit	Pen	70	6	16
Dexart	Bucket	11	3	8
	Laptop	20	4	8
	Toilet	70	6	16
MetaWorld	Shelf Place	11	3	8

Appendix

Table 7: Trajectory-Length Allocation for each task in simulation and real-world experiments.

Figure 9: Visualization of the trajectories generated by ReV on Place-Food-via and Camera-Alignment-via . Here, we use green bounding boxes to mark the frames in which the end-effector passes through the designated via-point (green ball).

Figure 10: Real-world Experimental Platform. The setup comprises a dual-robot arm system, a top-down RGB-D camera, and a black table serving as the workspace.

Task	Agent Number	Description
Collecting Objects	1	A doll is placed on the table. The robotic manipulator first grasps it and then transports it into the green box.
Push T	1	A T-shaped object is initially positioned on the table. The robotic arm pushes it into a predefined T-shaped location.
Stacking Playing Card	1	A playing card is placed flat in the central region of the table. The robotic arm first grasps it precisely and then places it vertically into another playing card to achieve a stacking effect.
Grabbing Rod	2	A long rod is placed on a block. The two robotic arms first simultaneously grasp each end of the rod and then collaboratively lift it to a specified height.
Handing Eraser Over	2	One robotic arm first grasps an eraser precisely. It then passes the eraser to another robotic arm through a coordinated handover motion.

Appendix

Table 8: Original Task Descriptions for real-world experiments.

Visuomotor Policy Learning

Visuomotor policy learning trains robots to turn camera observations, often with language and proprioception, into actions for manipulation and navigation.

See all 279 papers

ARP: Enhancing Quantized Skill Abstractions via Visual Alignment and Iterative Refinement for Robotic Manipulation
Jun 21, 2026Yuntian Wang, Zesheng Jia, Yuhui Duan +5Long-Horizon Robotic Manipulation Robot Skill Learning
School of Future Science and Engineering, Soochow University · Key Laboratory of General Artificial Intelligence and Large Models in Provincial Universities, Soochow University · College of Mechanical and Electronic Engineering, China Jiliang University +2
AdaWorldPolicy: World-Model-Driven Diffusion Policy with Online Adaptive Learning for Robotic Manipulation
Feb 23, 2026Ge Yuan, Qiyuan Qiao, Jing Zhang +1Robot Policy Adaptation Robotic Manipulation
The University of Hong Kong · Beihang University
SegDiff: Segmented Trajectory Diffusion for Consistent and Adaptive Robot Manipulation
Jul 13, 2026Haidong Cao, Wenjun Cao, Quanhao Li +5Long-Horizon Robotic Manipulation Visuomotor Policy Learning
Institute of Trustworthy Embodied AI, Fudan University, China · Shanghai Key Laboratory of Multimodal Embodied AI, China

Jun 21, 2026J/K move · Enter open · S save

ARP: Enhancing Quantized Skill Abstractions via Visual Alignment and Iterative Refinement for Robotic Manipulation

Yuntian Wang, Zesheng Jia, Yuhui Duan, Qibing Wang+4

School of Future Science and Engineering, Soochow University · Key Laboratory of General Artificial Intelligence and Large Models in Provincial Universities, Soochow University · College of Mechanical and Electronic Engineering, China Jiliang University+2

Long-Horizon Robotic Manipulation Robot Skill Learning Visuomotor Policy Learning Imitation Learning Interactive Imitation Learning Robot Imitation Learning

Open paper AskPDF arXiv

Learning visuomotor policies for long-horizon manipulation remains a fundamental challenge. Recent skill-based imitation learning methods based on discrete quantization have shown promising results by representing complex behaviors as temporally extended skills. However, most existing approaches primarily encode action trajectories into latent skills, yielding weak visual-semantic grounding and limiting the ability to leverage visual observations for skill selection. Moreover, discrete tokenization inevitably incurs precision errors during continuous action generation. To alleviate these issues, we propose Aligned Refinement Policy (ARP), a discrete-skill framework that couples semantic grounding with execution-level refinement. Specifically, ARP introduces (i) a visual--action alignment objective that contrastively aligns visual embeddings with pre-quantized action representations in a shared latent space while preserving a state-independent skill decoder, and (ii) a lightweight Iterative Residual Head (IRH) that performs a two-step refinement to recover fine-grained control for precise execution. Extensive experiments show that ARP achieves state-of-the-art performance on the LIBERO and Meta-World benchmarks. Moreover, real-robot experiments on the Kuavo 4 Pro humanoid platform further validate its effectiveness, yielding consistent performance gains over several baselines on two challenging manipulation tasks.

Abstract

Visuomotor Policy Learning

Explore similar work

ARP: Enhancing Quantized Skill Abstractions via Visual Alignment and Iterative Refinement for Robotic Manipulation

AdaWorldPolicy: World-Model-Driven Diffusion Policy with Online Adaptive Learning for Robotic Manipulation

SegDiff: Segmented Trajectory Diffusion for Consistent and Adaptive Robot Manipulation