WorldRoPE: Probabilistic World-Centric Rotary Embedding for Multi-View Attention

Mijeong Kim; Kyujin Lim; Bohyung Han

cs.CVOct 12, 2026

WorldRoPE: Probabilistic World-Centric Rotary Embedding for Multi-View Attention

Authors: Mijeong Kim, Kyujin Lim, Bohyung Han

Organizations: ECE, Seoul National University · IPAI, Seoul National University

Abstract

Rotary Positional Embedding (RoPE) is widely used to encode relative positions in Transformers. In multi-view settings, however, patch coordinates are defined independently in each image: the same 3D point can therefore have different coordinates across views, making them insufficient for establishing cross-view correspondences. We introduce WorldRoPE, a probabilistic world-centric RoPE for multi-view attention. At each layer, WorldRoPE back-projects each patch token into a set of volumetric Gaussian primitives in a shared world frame. These primitives represent multiple plausible 3D supports for a token, jointly capturing alternative depth hypotheses, depth uncertainty, and the finite spatial extent of its patch frustum. When a key token's support is projected into a query view, it induces a distribution over where that key may appear in the query image. WorldRoPE integrates the rotary operator over this distribution in closed form, combining rotations at the projected primitive centers according to their mixture weights with covariance-dependent attenuation. This makes attention less sensitive to uncertain key locations in the query view, without rasterizing or warping token features. The formulation reduces to deterministic RoPE when the support collapses to a point. Across layers, the predicted geometry guides cross-view attention, while the updated features refine the geometry used at the next layer. Experiments on novel view synthesis, depth estimation, and 3D geometry reconstruction demonstrate the effectiveness of WorldRoPE for multi-view Transformers.

Figures & tables

Method	Depth	Uncertainty	Multi-Hypothesis	Spatial Extent	Token Geometry
Plücker Ray ( Jin et al., 2025 )	–	–	–	–	Camera rays
GTA ( Miyato et al., 2024 )	–	–	–	–	–
PRoPE ( Li et al., 2025b )	–	–	–	–	–
RayRoPE ( Wu et al., 2026 )	Predicted	✓	–	–	Ray segments
URoPE ( Xie et al., 2026 )	Fixed	–	$\checkmark^{\dagger}$	–	Points on rays
WorldRoPE (ours)	Predicted	✓	✓	✓	Volumetric Gaussians

Table 1: Comparison of camera encodings for multi-view attention. Our method predicts depth hypotheses and their uncertainty for each token, representing possible 3D locations with volumetric Gaussian primitives.

\dagger

: URoPE adopts predefined depth anchors across heads, while our method combines predicted hypotheses.

Figure 1: WorldRoPE overview. We model probabilistic token positions with world-space Gaussian primitives , capturing multiple depths, uncertainty, and patch extent. Their projections define possible key locations in the query view, over which closed-form RoPE integration guides multi-view attention without rasterizing or warping token features.

Figure 2: Multi-view attention with WorldRoPE. At each attention layer, we predict world-space Gaussian primitives from token features. Projecting them into the query view places key and query positions in a common image frame. We integrate RoPE over the projected key distribution to guide cross-view attention, and the updated features guide geometry prediction in the next layer.

Method	RE10K ( Zhou et al., 2018 )			Objaverse ( Deitke et al., 2023 )
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Plücker Ray ( Jin et al., 2025 )	24.62	0.790	0.108	15.15	0.838	0.355
GTA ( Miyato et al., 2024 )	25.05	0.799	0.102	22.22	0.892	0.123
PRoPE ( Li et al., 2025b )	25.44	0.811	0.095	22.41	0.894	0.117
RayRoPE ( Wu et al., 2026 )	26.17	0.833	0.081	22.25	0.892	0.121
URoPE ( Xie et al., 2026 )	26.26	0.834	0.078	21.99	0.890	0.128

Table 2: Comparison of positional encodings for novel view synthesis with LVSM on RE10K and Objaverse. WorldRoPE achieves the best performance across all metrics on both datasets.

Figure 3: Qualitative comparison of novel view synthesis on RE10K and Objaverse. WorldRoPE better preserves thin structures and fine details like stair railings and bicycle wheels.

Table 6

Panel of Figure 4 — Figure 4: Visualization of predicted depth hypotheses. (a) Red highlights tokens with multiple prominent peaks in their depth distributions. (b) A patch spanning vegetation and sky exhibits two distinct peaks, while a patch within vegetation has one dominant peak.

Figure 6: Robustness to (a) camera zoom and (b) rotation noise during evaluation, and (c) depth configuration during training. WorldRoPE achieves robustness across all tested settings.

Figure 7: World-space Gaussian primitives across attention layers (left) and their projection from the key view into the query view (right). RGB rasterization is used only for visualization; the primitives carry token features and are not rasterized during attention.

Method	Scenes11		SUN3D		RGBD		ScanNet++ (unseen)
Method	Abs Rel $\downarrow$	RMSE $\downarrow$	Abs Rel $\downarrow$	RMSE $\downarrow$	Abs Rel $\downarrow$	RMSE $\downarrow$	Abs Rel $\downarrow$	RMSE $\downarrow$
w/o camera encoding	0.176	1.182	0.179	0.524	0.185	0.953	0.286	0.482
Plücker Ray ( Jin et al., 2025 )	0.123	0.943	0.130	0.391	0.145	0.790	0.276	0.464
GTA ( Miyato et al., 2024 )	0.116	0.830	0.133	0.384	0.134	0.693	0.325	0.520
PRoPE ( Li et al., 2025b )	0.106	0.778	0.136	0.390	0.134	0.695	0.263	0.434
RayRoPE ( Wu et al., 2026 )	0.101	0.743	0.125	0.370	0.128	0.677	0.257	0.426

Table 6: Comparison of camera-aware positional encodings for stereo depth estimation with UniMatch. We report depth errors using AbsRel and RMSE on the Scenes11, SUN3D, and RGBD datasets, and assess cross-dataset generalization on the unseen ScanNet++ dataset. WorldRoPE outperforms existing positional encodings in RMSE on all four datasets.

Method	ScanNet++
Method	Depth AbsRel $\downarrow$	Norm. RMSE $\downarrow$	Cam.-pts AbsRel $\downarrow$	Rot. Err. $\downarrow$
RayRoPE	0.0579	0.0624	0.0575	1.04
URoPE	0.0581	0.0626	0.0576	1.02
WorldRoPE	0.0567	0.0619	0.0565	0.98

Table 7: Comparison of camera-aware positional encodings for feed-forward 3D reconstruction with VGGT. WorldRoPE achieves the lowest depth, camera-space point, and camera rotation errors.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Projected Gaussian supports and their responses under axial and mixed rotary operators. We set $\mathbf{q}=\mathbf{k}=\mathbf{1}/\sqrt{d}$ to isolate the positional response. Panel (a) shows a point hypothesis; panels (b) to (d) have the same marginal variances $\Sigma_{uu}$ and $\Sigma_{vv}$ but different off-diagonal covariance $\Sigma_{uv}$ and $\Sigma_{vu}$ . Across these panels, the axial operator gives the same response, whereas the mixed operator responds to the covariance between image coordinates.

Method	Operator	RE10K ( Zhou et al., 2018 )			Objaverse ( Deitke et al., 2023 )
Method	Operator	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
RayRoPE ( Wu et al., 2026 )	Axial	26.17	0.833	0.081	22.25	0.892	0.121
RayRoPE ( Wu et al., 2026 )	Mixed	26.14 ( $-0.03$ )	0.832 ( $-0.001$ )	0.082 ( $+0.001$ )	22.33 ( $+0.08$ )	0.894 ( $+0.002$ )	0.118 ( $-0.003$ )
URoPE ( Xie et al., 2026 )	Axial	26.26	0.834	0.078	21.99	0.890	0.128
URoPE ( Xie et al., 2026 )	Mixed	26.27 ( $+0.01$ )	0.834 ( $0.000$ )	0.079 ( $+0.001$ )	22.03 ( $+0.04$ )	0.890 ( $0.000$ )	0.126 ( $-0.002$ )
WorldRoPE (ours)	Axial	26.75	0.846	0.073	22.52	0.896	0.113

Appendix

Table 8: Comparison of axial and mixed rotary operators for novel view synthesis on the RE10K and Objaverse datasets. Parenthesized values denote changes from the axial variant.

Panel of Figure 9 — Figure 9: Sensitivity to initialization scale on RE10K. The scale multiplies the initial anchors of URoPE and the initial bias of WorldRoPE, with $\times 1$ denoting the default initialization. WorldRoPE remains comparatively stable across a broad range of initialization scales.

Figure 10: Visualization of the probabilistic world-space primitive projection process. For each example, tokens from the key views are unprojected into probabilistic world-space primitives using the predicted depth and uncertainty. Each primitive uses the mean RGB of its source patch as color and its mixture weight as opacity for visualization. The primitives are then projected into the query views and compared with the corresponding ground-truth views.

Figure 11: Layer-wise visualization of predicted world-space Gaussian primitives. We render the Gaussian primitives predicted at Layers 1–6 using the original depth uncertainty ( $\sigma_{d}\times 1.0$ ) and a reduced uncertainty ( $\sigma_{d}\times 0.2$ ). For visualization, each primitive is colored by the mean RGB of its corresponding image token, with its mixture weight used as opacity.

Panel of Figure 12 — Figure 12: Robustness evaluation on RE10K under varying multi-view context and camera conditions. (a)–(c) vary the number of reference views at inference time. (d)–(f) evaluate image zoom-in. (g)–(i) evaluate reference-camera rotation noise. (j)–(l) evaluate reference-camera distance. Higher PSNR and SSIM and lower LPIPS indicate better performance.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
w/o Temperature	26.30	0.835	0.082
w/ Learned Temperature	26.33	0.836	0.082
WorldRoPE	26.75	0.846	0.073

Appendix

Table 9: Ablation of token-adaptive temperature on RE10K.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
w/o V/O rotation	26.39	0.838	0.079
WorldRoPE	26.75	0.846	0.073

Appendix

Table 10: Ablation of value and output rotations on RE10K.

Panel of Figure 13 — Figure 13: Training and inference time as the number of reference views increases. 2D-Axial denotes standard 2D image-space RoPE.

Setting	Novel-view synthesis	Depth estimation	3D geometry estimation
Backbone	LVSM-25M	UniMatch	VGGT-1B
Training data	RE10K or Objaverse	Scenes11, SUN3D, RGB-D SLAM	ScanNet++ (896 scenes)
Evaluation data	RE10K or Objaverse test split	Scenes11, SUN3D, RGB-D SLAM, and ScanNet++	ScanNet++ (30 held-out scenes)
Training views	2 references + 1 target	2 views	4 views
Resolution (train)	$256\times 256$	$224\times 288$	$518\times 294$
Resolution (eval.)	$256\times 256$	$240\times 320$	$518\times 294$

Appendix

Table 11: Summary of the task-specific implementations of WorldRoPE.

Figure 14: Qualitative comparison of novel-view synthesis on RE10K and Objaverse. The top five rows show RE10K examples, and the bottom five rows show Objaverse examples. The first two columns contain the reference views, followed by the target ground truth and predictions from the compared methods.

Figure 15: Qualitative comparison of the axial and mixed RoPE variants on RE10K and Objaverse. The top five rows show RE10K examples, and the bottom five rows show Objaverse examples. For each positional encoding, the axial and mixed RoPE variants are shown side by side.

Figure 16: Qualitative comparison of depth estimation on Scenes11, SUN3D, RGBD, and unseen ScanNet++. The row groups follow this dataset order from top to bottom. The first three columns contain the reference image, target image, and ground-truth depth, followed by predictions from the compared methods.

Rotary Positional Embeddings

Rotary positional embeddings (RoPE) encode token positions by rotating attention features, supporting relative-position reasoning in language and multimodal models.

See all 98 papers

RayRoPE: Projective Ray Positional Encoding for Multi-view Attention
Jan 21, 2026Yu Wu, Minsik Jeon, Jen-Hao Rick Chang +2Multi-View 3D Reconstruction Novel View Synthesis
Carnegie Mellon University · Apple
HRoPE: Homography Relative Positional Encoding
Oct 20, 2026Huachen Gao, Yang Yang, Xiaoqian Liang +6Positional Encoding Relative Positional Encoding
Zhejiang University · Westlake University · Huazhong University of Science and Technology +2
URoPE: Universal Relative Position Embedding across Geometric Spaces
Apr 20, 2026Yichen Xie, Depu Meng, Chensheng Peng +4Rotary Positional Embeddings Relative Positional Encoding
Applied Intuition · University of California, Berkeley

Jan 21, 2026J/K move · Enter open · S save

RayRoPE: Projective Ray Positional Encoding for Multi-view Attention

Yu Wu, Minsik Jeon, Jen-Hao Rick Chang, Oncel Tuzel+1

Carnegie Mellon University · Apple

Multi-View 3D Reconstruction Novel View Synthesis Positional Encoding Attention Mechanisms Stereo Depth Estimation

Open paper AskPDF arXiv

We study positional encodings for multi-view transformers that process tokens from a set of posed input images, and seek a mechanism that encodes patches uniquely, allows SE(3)-invariant attention with multi-frequency similarity, and can adapt to the geometry of the underlying 3D scene. We find that prior (absolute or relative) encoding schemes for multi-view attention do not meet these desiderata, and present RayRoPE to address this gap. RayRoPE represents patch positions based on associated rays and computes query-frame projective coordinates to ensure SE(3) invariance. To adapt to scene geometry, RayRoPE predicts (without direct supervision) a per-token depth to obtain its position along the corresponding ray, while also modeling uncertainty and analytically computing the expected positional encoding. We validate our method on the tasks of novel-view synthesis, stereo depth estimation, and feed-forward 3DGS reconstruction. While remaining efficient, RayRoPE consistently improves over alternate position encoding schemes (e.g., 24% relative improvement on LPIPS in RE10K).

Abstract

Rotary Positional Embeddings

Explore similar work

RayRoPE: Projective Ray Positional Encoding for Multi-view Attention

HRoPE: Homography Relative Positional Encoding

URoPE: Universal Relative Position Embedding across Geometric Spaces