GR-VIP: Geometric representation conditioning for precise visuomotor policy

Chanhwi Lee; Jiwon Seong; Eunje Choi; Hohyung Lee; Jongsoo Lee; Junyeong Bae; Inveom Kwak; Woosang Shin

cs.ROOct 16, 2026

GR-VIP: Geometric representation conditioning for precise visuomotor policy

Authors: Chanhwi Lee, Jiwon Seong, Eunje Choi, Hohyung Lee, Jongsoo Lee, Junyeong Bae, Inveom Kwak, Woosang Shin,

Organizations: Pohang University of Science and Technology (POSTECH) · Polaris3D · the Korea Institute of Industrial Technology (KITECH)

Abstract

Achieving precise terminal position and heading alignment remains a significant challenge for learning-based mobile robot control. Recent vision-based learning methods have shown promise for precise target-relative control and generalization to unseen environments with pretrained visual encoders, but general-purpose visual representations are not specifically designed to capture the fine-grained spatial differences between current and goal views required for precise control, and real-world demonstrations combining millimeter-scale terminal accuracy with zero-shot transfer across environments remain limited. To address these challenges, we propose Geometric Representation Conditioning for Precise Visuomotor Policy (GR-VIP), a goal-conditioned visuomotor policy that leverages a visual encoder pretrained for relative camera pose estimation. Rather than using the final relative-pose estimate, GR-VIP conditions action prediction on the dense relational features produced before pose regression, preserving fine-grained cross-view information for learning corrective actions from demonstrations. We evaluate GR-VIP on a real-world docking task and demonstrate high docking success rates and millimeter-scale terminal accuracy in both the training and unseen environments, without additional training for transfer. Furthermore, supplementary experiments illustrate the policy's ability to align with non-docking targets and complete docking under a tested low-light condition.

Figures & tables

Fig. 1: Overview of GR-VIP. (a) GR-VIP performs precise real-world docking with autonomous repositioning to correct misalignment. (b) A policy trained in a single environment transfers to unseen environments without additional training. The same policy also demonstrates alignment to non-docking targets and successful docking under a low-light condition.

Fig. 2: GR-VIP architecture. Dense relational tokens from the frozen Reloc3r capture geometric relationships between each observation and the goal image. The tokens are resampled and fused with wheel-velocity history to condition a diffusion policy that predicts future velocity commands.

Fig. 3: Closed-loop execution of GR-VIP. Multimodal observations condition diffusion-based action generation. Successive action plans are blended to maintain temporal continuity and executed in a receding-horizon manner as new predictions become available. Here, $H_{m}$ and $H_{a}$ are both set to 60 steps.

Fig. 4: Experimental setup. (a) Mobile robot platform. (b) Docking setup and initial poses. Dashed arcs indicate 0.5- and 1.0-m approach distances, and purple arrows denote intentionally misaligned initial poses. (c) Marker uncovered for terminal-pose evaluation after policy execution. (d) Training environment (4F-H). (e) Unseen-background environment (4F-I). (f) Environment with an unseen docking station and background (5F-I). (g) Low-light condition. (h, i) Alignment to a non-docking target during policy execution and subsequent terminal-pose evaluation.

Model	Train	Eval setting	Success % (count)	$n/N$	Lateral (mm)	Longitudinal (mm)	Heading ( ∘ )
(a) Training-environment docking performance
GR-VIP	4F-H	4F-H	100.0 (16/16)	16/16	$\mathbf{2.53}\pm 2.24$	$\mathbf{1.95}\pm 1.41$	$2.28\pm 1.64$
GC-DP (NoMaD adapted)			56.3 (9/16)	13/16	$9.44\pm 6.23$	$9.86\pm 10.62$	$2.09\pm 1.17$
DINOv3-based			50.0 (8/16)	15/16	$12.79\pm 10.40$	$12.66\pm 12.63$	$\mathbf{1.81}\pm 2.11$
(b) Zero-shot transfer and environment-specific baselines
GR-VIP	4F-H	4F-I	100.0 (16/16)	16/16	$7.20\pm 4.78$	$7.15\pm 4.86$	$3.75\pm 1.71$

TABLE I: Quantitative evaluation of docking performance, zero-shot transfer, representation ablations, and alignment to non-docking targets.

Fig. 5: Overview of terminal alignment errors for policies trained on the common 4F-H demonstrations. The top row shows the full error range, while the bottom row provides a detailed view near zero error.

n

denotes the number of valid error samples. GR-VIP consistently exhibits low terminal alignment errors relative to the other policies across all three environments.

Robotic Control

Robotic control studies how machines use sensing and computation to produce reliable movement and interaction in changing environments.

See all 587 papers

GAP: Geometric Anchor Pre-training for Data-Efficient Visuomotor Learning of Manipulation Tasks
May 15, 2026Davide Buoso, Andrea Protopapa, Stefano Di Carlo +2Vision Foundation Model Adaptation Visuomotor Policy Learning
Department of Control and Computer Engineering, Polytechnic University of Turin, Italy
MAVP: Map-Aware Visuomotor Policies for Mobile Manipulation
Sep 22, 2026Jinhe Tang, Ruixiao Dai, Weiming ZhiRobot Trajectory Tracking Robot Policy Learning
School of Computer Science, The University of Sydney, Australia
GHOST: Hierarchical Sub-Goal Policies for Generalizing Robot Manipulation
Jun 8, 2026Sriram Krishna, Ben Eisner, Haotian Zhan +5Robot Policy Learning Robot Skill Learning
Robotics Institute, Carnegie Mellon University · UMass Amherst

May 15, 2026J/K move · Enter open · S save

GAP: Geometric Anchor Pre-training for Data-Efficient Visuomotor Learning of Manipulation Tasks

Davide Buoso, Andrea Protopapa, Stefano Di Carlo, Francesca Pistilli+1

Department of Control and Computer Engineering, Polytechnic University of Turin, Italy

Vision Foundation Model Adaptation Visuomotor Policy Learning Geometric Representation Learning Robot Imitation Learning Robot Learning

Open paper AskPDF arXiv

Learning visuomotor policies from scarce expert demonstrations remains a core challenge in robotic manipulation. A primary hurdle lies in distilling high-dimensional RGB representations into control-relevant geometry without overfitting. While using frozen pre-trained Vision Foundation Models (VFMs) improves data efficiency, it also shifts most task adaptation onto a small spatial pooling module, which can latch onto task-irrelevant shortcuts and lose geometric grounding when finetuned with few data samples. More broadly, pre-trained visual representations used for policy learning have been observed to struggle under even minor scene perturbations, highlighting the need for robustness-oriented inductive biases. We propose Geometric Anchor Pre-training (GAP), a simple, action-free warm-up stage that regularizes the spatial adapter before downstream imitation learning. GAP pre-trains the pooling layer on a lightweight simulated proxy task where object masks are available at no cost, encouraging the adapter to produce keypoints that lie on the object, cover its spatial extent, and remain sharp and repeatable over time. This yields stable geometric anchors that provide a reliable coordinate interface for few-shot policy learning, while keeping the VFM frozen. We evaluate GAP on RoboMimic and ManiSkill under severe data scarcity (15-50 demonstrations) and domain shift. A simple adapter regularized with GAP consistently outperforms stronger attention-based poolers and end-to-end fine-tuning, achieving 62% success on RoboMimic Can with 15 demonstrations (+16% over AFA), 63% on the long-horizon high-precision Tool Hang task with 50 demonstrations, and 61% on ManiSkill StackCube with 30 demonstrations (+11% over full fine-tuning). The proxy stage is lightweight and fully decoupled from downstream tasks, making it practical to reuse across environments and manipulation skills.

Abstract

Robotic Control

Explore similar work

GAP: Geometric Anchor Pre-training for Data-Efficient Visuomotor Learning of Manipulation Tasks

MAVP: Map-Aware Visuomotor Policies for Mobile Manipulation

GHOST: Hierarchical Sub-Goal Policies for Generalizing Robot Manipulation