cs.RONov 9, 2026

PAIR: Proprioception-Aligned Image-Ray Representations for Viewpoint-Robust Manipulation

Authors: Sung-Wook Lee, Xuhui Kang, Dylan Losey, Yen-Ling Kuo

Organizations: University of Virginia · Virginia Tech

Abstract

While imitation learning has made substantial progress toward generalizable robot manipulation through scaling, viewpoint robustness remains a major bottleneck. Visuomotor policies rely heavily on visual inputs, yet images provide an inherently ambiguous signal under viewpoint change: similar scenes may appear markedly different across viewpoints, while distinct states may share certain visual overlaps. A policy must therefore infer not only the task-relevant state, but also how it is visually projected under a particular camera configuration. This entanglement of viewpoint-dependent appearance and task-relevant state can make learning inefficient and limit generalization to unseen viewpoints. Recent work has begun to address this challenge through camera-ray conditioning and representation learning using geometric cues to induce invariance. However, providing camera information as additional context does not by itself ground visual representations in how the task scene is visually projected, while geometric alignment without camera context can introduce spurious correspondences by ignoring viewpoint-dependent projection. We propose PAIR, a contrastive learning framework that aligns joint image-ray representations using robot proprioception. PAIR encourages observations with similar robot states to remain close in representation space across viewpoints, providing a structured grounding signal for learning viewpoint-robust features. PAIR is a highly adaptable framework: it can be used to train Diffusion Policy (DP) or post-train vision-language-action (VLA) models. In simulated and real-world manipulation benchmarks with camera variation, PAIR consistently outperforms the behavior cloning baseline without ray conditioning by 12.7% and 26.7% points, respectively, demonstrating stronger generalization to unseen viewpoints. Project website: PAIRrobot.github.io.

Figures & tables

Explore similar work

CardsList
  1. RayViT: Ray-Conditioned Visual Representations for Viewpoint-Robust Imitation Learning

    Jul 31, 2026Qian Wang, Longrui Chen, Peiran Sun +8Visual Representation LearningRobot Skill Learning

  2. GeoProp: Grounding Robot State in Vision for Generalist Manipulation

    Jul 8, 2026Guoyang Zhao, Quanhao Qian, Gongjie Zhang +5Generalization in Robotic ManipulationVisuomotor Policy Learning

  3. VistaBot: View-Robust Robot Manipulation via Spatiotemporal-Aware View Synthesis

    Apr 23, 2026Songen Gu, Yuhang Zheng, Weize Li +6Generalization in Robotic ManipulationNovel View Synthesis