Geometric Relative Positional Encoding has become the primary mechanism for injecting spatial inductive biases into multi-view transformers. Existing designs condition attention on camera-level frustums or ray-marched depth, associating cross-view tokens through the sensor geometry. Such sensor-centric formulations, however, describe only where a token is observed from, leaving the rich surface geometry of the scene region depicted by the token. In this paper, we advocate a shift toward surface-aware positional encoding that explicitly grounds patch tokens in the geometry of physical surfaces. We introduce HRoPE, which represents the local geometry of each patch as a slanted 3D plane, following the slanted support-window assumption of classical multi-view stereo. We inject token-level, plane-induced homographies into attention as relative transformations. This equips the transformer with a geometric prior for rectifying perspective distortions and aligning patch features across viewpoints. Our encoding requires no ground-truth geometry: the plane parameters can be predicted end-to-end under a view-synthesis objective alone, or derived from foundation models. Extensive experiments on RealEstate10K, CO3Dv2, and DL3DV show that our best variant of HRoPE consistently outperforms the evaluated geometric positional encodings for novel view synthesis. Extensive ablations and analysis show the effectiveness of the proposed encoding. Please refer to our project page at: https://gaohchen.github.io/HRoPE
Figures & tables
Figure 1: From sensor-centric to surface-aware geometric positional encoding. Left: PRoPE ( Li et al., 2025 ) assigns every token in a view the same camera projection P=K[R∣t] . Middle: RayRoPE ( Wu et al., 2026 ) predicts a depth d for each token and places it at a 3D point X along the camera ray, but does not model local surface orientation. Right: HRoPE associates each patch with a slanted tangent plane and relates token pairs through the plane-induced homography Hi,uHj,v−1 (Eq. 2 ), thereby encoding both depth and local surface orientation in the attention comparison.
Figure 2: Overview of HRoPE. Left: per-patch parameters (R,t,n,κ) are predicted from patch-level image features (Sec. 3.3 ), where n and κ=1/d represent the local plane’s unit normal and inverse plane distance, respectively. Together with intrinsics K , these parameters define \mathrm{H}=\mathrm{K}\bigl(\mathord{\hbox{\pagecolor{paramRT!12!white}\raisebox{0.0pt}[0.6pt][1.5pt]{\makebox[11.99998pt][c]{\textstyle\color[rgb]{0.1211,0.6094,0.1016}\vphantom{\mathbf{n}^{\top}}\mathrm{R}}}}}+\mathord{\hbox{\pagecolor{paramKappa!12!white}\raisebox{0.0pt}[0.6pt][1.5pt]{\makebox[11.99998pt][c]{\textstyle\color[rgb]{0.75,0.3047,0.4297}\vphantom{\mathbf{n}^{\top}}\kappa}}}}\mkern 1.0mu\mathord{\hbox{\pagecolor{paramRT!12!white}\raisebox{0.0pt}[0.6pt][1.5pt]{\makebox[11.99998pt][c]{\textstyle\color[rgb]{0.1211,0.6094,0.1016}\vphantom{\mathbf{n}^{\top}}\mathbf{t}}}}}\mkern 1.0mu\mathord{\hbox{\pagecolor{paramN!12!white}\raisebox{0.0pt}[0.6pt][1.5pt]{\makebox[11.99998pt][c]{\textstyle\color[rgb]{0.2031,0.3438,0.8125}\vphantom{\mathbf{n}^{\top}}\mathbf{n}^{\top}}}}}\bigr) (Eq. 5 ). Middle: token-specific transforms modulate the cross-view attention, yielding the relative factor Hi,uHj,v−1 in each attention logit (Eq. 4 ). Right: the transform is repeated block-diagonally over the geometric half of each head, while the remaining dimensions retain the 2 D rotary blocks Φx and Φy (Eq. 17 ).
CO3Dv2
RealEstate10K
Objaverse
Method
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
LVSM ( Jin et al., 2025 )
18.617
0.601
0.618
25.282
0.802
0.130
29.676
0.923
0.098
GTA ( Miyato et al., 2024 )
19.482
0.612
0.573
26.076
0.824
0.110
30.500
0.929
0.079
PRoPE ( Li et al., 2025 )
19.495
0.626
0.583
26.142
0.826
0.111
30.588
0.929
0.078
RayRoPE ( Wu et al., 2026 )
18.948
0.617
0.601
26.348
0.830
0.107
30.871
0.932
0.079
URoPE ( Xie et al., 2026 )
-
-
-
26.836
0.838
0.099
30.890
0.930
0.073
Table 1: Novel view synthesis on CO3Dv2, RealEstate10K, and Objaverse. HRoPE uses image features to predict per-patch transformation parameters. Best in green , second in blue .
2 views
4 views
8 views
Method
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
LVSM ( Jin et al., 2025 )
17.718
0.459
0.474
19.331
0.571
0.317
20.441
0.615
0.273
PRoPE ( Li et al., 2025 )
18.574
0.503
0.459
19.957
0.602
0.289
20.653
0.621
0.271
HRoPE
19.284
0.544
0.447
20.373
0.624
0.270
20.940
0.635
0.257
Table 2: Scaling on the DL3DV benchmark with a 53 M LVSM under 2 -, 4 -, and 8 -view inputs. HRoPE consistently improves over PRoPE across all three input-view settings. Best in green .
Figure 3: Qualitative novel-view synthesis on RealEstate10K and CO3Dv2. Each row shows one scene or object. From left to right, the columns contain two input views, predictions from LVSM, PRoPE, RayRoPE, and HRoPE, and the ground-truth target view (GT).
Figure 4: Ablations on RealEstate10K, measured by PSNR. (a) Transformation parameterizations using frozen VGGT features (blue) or a predictor trained from scratch (green), evaluated after 80 k training steps. The x-axis labels denote the LVSM baseline (None), a depth term (Depth), a rotation ( SO(3) ), the calibrated projection composed with a per-patch tangent-frame transform ( P⋅F ), a rigid transformation ( SE(3) ), our homography parameterization (HRoPE), and an unconstrained 4×4 matrix (Full). Stars denote the corresponding 768 -dimensional additive positional encodings (APE). (b) Feature sources with the HRoPE parameterization fixed, evaluated after 40 k and 80 k training steps. DINOv2 † denotes the fine-tuned DINOv2 features from VGGT.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
PRoPE
26.142
0.826
0.111
PRoPE ( P⋅F )
27.115
0.840
0.104
HRoPE
27.410
0.848
0.091
Table 3: Local surface geometry on RealEstate10K (Left) & Oracle geometry on ASE (Right). On RealEstate10K, PRoPE ( P⋅F ) and HRoPE both use VGGT features for parameter prediction. On ASE, HRoPE uses either VGGT-predicted parameters or ground-truth camera and plane parameters. Best and second-best results are highlighted in green and blue .
Figure 5: Predicted local planes on RealEstate10K. Two scenes are shown side by side; within each scene, the two rows correspond to the reference views. For each view, the columns show the input image, predicted plane orientations overlaid on the image (Visual Plane), and per-patch normals encoded as a color map (Normal Map). The red boxes mark the same approximately planar surface across views, where the predicted orientations remain locally coherent and cross-view consistent.
Figure 6: Out-of-distribution generalization on RealEstate10K. Left pair: PSNR ( ↑ ) and LPIPS ( ↓ ) as the unseen target-camera zoom factor increases from 1× to 5× . Right pair: the PSNR–LPIPS trade-off under soft target-view extrapolation and an enlarged baseline between the two input views.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Applying HRoPE to Multi-view Attention. Here, τi,u denotes the token corresponding to image patch u in view i . A predictor maps each patch token τi,u to the transformation parameters (Ri,u,ti,u,ni,u,κi,u) , from which Hi,u=KiAi,u is constructed. Queries are transformed by Hi,u⊤ , while keys and values are transformed by Hj,v−1 . The aggregated value is subsequently mapped to the query token’s frame by Hi,u . Only one query–key pair is shown for clarity.
Source
PSNR ↑
GPU-hours
Original (LVSM)
25.282
33
Learnable
26.419
33
DINOv2 ( Oquab et al., 2024 )
26.701
46
DINOv2 † ( Oquab et al., 2024 )
27.085
72
DA3 ( Lin et al., 2025 )
27.137
48
VGGT ( Wang et al., 2025 )
27.410
76
Appendix
Table 4: Training cost of the parameter-source variants (Sec. 4.2.1 ), all under the same 80 k-iteration schedule; PSNR is the final RealEstate10K score. DINOv2 † is the fine-tuned DINOv2 taken from inside VGGT. Best in bold , second underlined .
1 view
5 views
10 views
Method
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
EscherNet + CaPE
17.689
0.858
0.181
22.065
0.899
0.114
23.666
0.914
0.089
EscherNet + HRoPE
19.900
0.883
0.135
23.075
0.911
0.092
24.515
0.923
0.074
Appendix
Table 5: Transferring HRoPE to EscherNet ( Kong et al., 2024 ) , an ∼800 M-parameter multi-view diffusion model, on Objaverse. We replace EscherNet’s native CaPE encoding with HRoPE and evaluate with 1 , 5 , and 10 input views. Best in green .
Numerical question
Multiple-choice question
Method
Obj. Cnt.
Abs. Dist.
Obj. Size
Room Size
Rel. Dist.
Rel. Dir.
Route Plan
Appr. Order
Avg. ↑
Rank
Proprietary models
GPT-4o ( Hurst et al., 2024 )
46.2
5.3
43.8
38.2
37.0
41.3
31.5
28.5
34.0
9
Gemini-1.5 Pro ( Team et al., 2024 )
56.2
30.9
64.1
43.6
51.3
46.3
36.0
34.6
45.4
4
Open-source models
LongVILA-8B ( Chen et al., 2025 )
29.1
9.1
16.7
0.0
29.6
30.7
32.5
25.5
21.6
13
Appendix
Table 6: Transferring HRoPE to a video multimodal LLM, evaluated on VSI-Bench ( Yang et al., 2025 ) . Our model uses LLaVA-Video-7B-Qwen2 as the backbone with HRoPE applied to the visual tokens, fine-tuned on the VSI-Bench training split at 32 frames; the preceding row reports a fine-tuned 7B video model without any geometric encoding ( Fan et al., 2026 ) . Frame budgets for the remaining methods are given in the text. Rank is taken over the overall average. Best in green .
Figure 8: Additional qualitative comparison on RealEstate10K. Each row is one scene. From left to right, the columns show novel views synthesized by LVSM, PRoPE, RayRoPE, and HRoPE (Ours), followed by the ground-truth target view (GT). All methods use the same backbone and inputs and differ only in their positional encoding.
Figure 9: Additional qualitative comparison on CO3Dv2. Each row is one object. From left to right, the columns show novel views synthesized by LVSM, PRoPE, RayRoPE, and HRoPE (Ours), followed by the ground-truth target view (GT). All methods use the same backbone and inputs and differ only in their positional encoding.
Figure 10: Additional qualitative comparison with EscherNet on Objaverse. Each row is one object. From left to right, the columns show three input views, the novel views synthesized with CaPE and HRoPE (Ours), and the ground-truth target view (GT). Both methods use the same EscherNet backbone and inputs and differ only in their positional encoding.
We study positional encodings for multi-view transformers that process tokens from a set of posed input images, and seek a mechanism that encodes patches uniquely, allows SE(3)-invariant attention with multi-frequency similarity, and can adapt to the geometry of the underlying 3D scene. We find that prior (absolute or relative) encoding schemes for multi-view attention do not meet these desiderata, and present RayRoPE to address this gap. RayRoPE represents patch positions based on associated rays and computes query-frame projective coordinates to ensure SE(3) invariance. To adapt to scene geometry, RayRoPE predicts (without direct supervision) a per-token depth to obtain its position along the corresponding ray, while also modeling uncertainty and analytically computing the expected positional encoding. We validate our method on the tasks of novel-view synthesis, stereo depth estimation, and feed-forward 3DGS reconstruction. While remaining efficient, RayRoPE consistently improves over alternate position encoding schemes (e.g., 24% relative improvement on LPIPS in RE10K).
Rotary Positional Embedding (RoPE) is widely used to encode relative positions in Transformers. In multi-view settings, however, patch coordinates are defined independently in each image: the same 3D point can therefore have different coordinates across views, making them insufficient for establishing cross-view correspondences. We introduce WorldRoPE, a probabilistic world-centric RoPE for multi-view attention. At each layer, WorldRoPE back-projects each patch token into a set of volumetric Gaussian primitives in a shared world frame. These primitives represent multiple plausible 3D supports for a token, jointly capturing alternative depth hypotheses, depth uncertainty, and the finite spatial extent of its patch frustum. When a key token's support is projected into a query view, it induces a distribution over where that key may appear in the query image. WorldRoPE integrates the rotary operator over this distribution in closed form, combining rotations at the projected primitive centers according to their mixture weights with covariance-dependent attenuation. This makes attention less sensitive to uncertain key locations in the query view, without rasterizing or warping token features. The formulation reduces to deterministic RoPE when the support collapses to a point. Across layers, the predicted geometry guides cross-view attention, while the updated features refine the geometry used at the next layer. Experiments on novel view synthesis, depth estimation, and 3D geometry reconstruction demonstrate the effectiveness of WorldRoPE for multi-view Transformers.
Mijeong Kim, Kyujin Lim, Bohyung Han
ECE, Seoul National University · IPAI, Seoul National University
Relative position embedding has become a standard mechanism for encoding positional information in Transformers. However, existing formulations are typically limited to a fixed geometric space, namely 1D sequences or regular 2D/3D grids, which restricts their applicability to many computer vision tasks that require geometric reasoning across camera views or between 2D and 3D spaces. To address this limitation, we propose URoPE, a universal extension of Rotary Position Embedding (RoPE) to cross-view or cross-dimensional geometric spaces. For each key/value image patch, URoPE samples 3D points along the corresponding camera ray at predefined depth anchors and projects them into the query image plane. Standard 2D RoPE can then be applied using the projected pixel coordinates. URoPE is a parameter-free and intrinsics-aware relative position embedding that is invariant to the choice of global coordinate systems, while remaining fully compatible with existing RoPE-optimized attention kernels. We evaluate URoPE as a plug-in positional encoding for transformer architectures across a diverse set of tasks, including novel view synthesis, 3D object detection, object tracking, and depth estimation, covering 2D-2D, 2D-3D, and temporal scenarios. Experiments show that URoPE consistently improves the performance of transformer-based models across all tasks, demonstrating its effectiveness and generality for geometric reasoning. Our code is available on our project website: https://urope-pe.github.io/.
Yichen Xie, Depu Meng, Chensheng Peng +4
Applied Intuition · University of California, Berkeley