HRoPE: Homography Relative Positional Encoding

Huachen Gao; Yang Yang; Xiaoqian Liang; Xingyu Chen; Youjia Zhang; Qi Zhang; Baorui Ma; Anpei Chen

cs.CVOct 20, 2026

HRoPE: Homography Relative Positional Encoding

Authors: Huachen Gao, Yang Yang, Xiaoqian Liang, Xingyu Chen, Youjia Zhang, Qi Zhang, Baorui Ma, Anpei Chen,

Organizations: Zhejiang University · Westlake University · Huazhong University of Science and Technology · VIVO ·

Abstract

Geometric Relative Positional Encoding has become the primary mechanism for injecting spatial inductive biases into multi-view transformers. Existing designs condition attention on camera-level frustums or ray-marched depth, associating cross-view tokens through the sensor geometry. Such sensor-centric formulations, however, describe only where a token is observed from, leaving the rich surface geometry of the scene region depicted by the token. In this paper, we advocate a shift toward surface-aware positional encoding that explicitly grounds patch tokens in the geometry of physical surfaces. We introduce HRoPE, which represents the local geometry of each patch as a slanted 3D plane, following the slanted support-window assumption of classical multi-view stereo. We inject token-level, plane-induced homographies into attention as relative transformations. This equips the transformer with a geometric prior for rectifying perspective distortions and aligning patch features across viewpoints. Our encoding requires no ground-truth geometry: the plane parameters can be predicted end-to-end under a view-synthesis objective alone, or derived from foundation models. Extensive experiments on RealEstate10K, CO3Dv2, and DL3DV show that our best variant of HRoPE consistently outperforms the evaluated geometric positional encodings for novel view synthesis. Extensive ablations and analysis show the effectiveness of the proposed encoding. Please refer to our project page at: https://gaohchen.github.io/HRoPE

Figures & tables

Figure 1: From sensor-centric to surface-aware geometric positional encoding. Left: PRoPE ( Li et al., 2025 ) assigns every token in a view the same camera projection $\mathrm{P}=\mathrm{K}[\mathrm{R}\mid\mathbf{t}]$ . Middle: RayRoPE ( Wu et al., 2026 ) predicts a depth $d$ for each token and places it at a 3D point $\mathbf{X}$ along the camera ray, but does not model local surface orientation. Right: HRoPE associates each patch with a slanted tangent plane and relates token pairs through the plane-induced homography $\mathrm{H}_{i,u}\mathrm{H}_{j,v}^{-1}$ (Eq. 2 ), thereby encoding both depth and local surface orientation in the attention comparison.

Figure 2: Overview of HRoPE. Left: per-patch parameters $(\mathrm{R},\mathbf{t},\mathbf{n},\kappa)$ are predicted from patch-level image features (Sec. 3.3 ), where $\mathbf{n}$ and $\kappa=1/d$ represent the local plane’s unit normal and inverse plane distance, respectively. Together with intrinsics $\mathrm{K}$ , these parameters define $\mathrm{H}=\mathrm{K}\bigl(\mathord{\hbox{\pagecolor{paramRT!12!white}\raisebox{0.0pt}[0.6pt][1.5pt]{\makebox[11.99998pt][c]{$\textstyle\color[rgb]{0.1211,0.6094,0.1016}\vphantom{\mathbf{n}^{\top}}\mathrm{R}$}}}}+\mathord{\hbox{\pagecolor{paramKappa!12!white}\raisebox{0.0pt}[0.6pt][1.5pt]{\makebox[11.99998pt][c]{$\textstyle\color[rgb]{0.75,0.3047,0.4297}\vphantom{\mathbf{n}^{\top}}\kappa$}}}}\mkern 1.0mu\mathord{\hbox{\pagecolor{paramRT!12!white}\raisebox{0.0pt}[0.6pt][1.5pt]{\makebox[11.99998pt][c]{$\textstyle\color[rgb]{0.1211,0.6094,0.1016}\vphantom{\mathbf{n}^{\top}}\mathbf{t}$}}}}\mkern 1.0mu\mathord{\hbox{\pagecolor{paramN!12!white}\raisebox{0.0pt}[0.6pt][1.5pt]{\makebox[11.99998pt][c]{$\textstyle\color[rgb]{0.2031,0.3438,0.8125}\vphantom{\mathbf{n}^{\top}}\mathbf{n}^{\top}$}}}}\bigr)$ (Eq. 5 ). Middle: token-specific transforms modulate the cross-view attention, yielding the relative factor $\mathrm{H}_{i,u}\mathrm{H}_{j,v}^{-1}$ in each attention logit (Eq. 4 ). Right: the transform is repeated block-diagonally over the geometric half of each head, while the remaining dimensions retain the $2$ D rotary blocks $\Phi_{x}$ and $\Phi_{y}$ (Eq. 17 ). — Figure 2: Overview of HRoPE. Left: per-patch parameters $(\mathrm{R},\mathbf{t},\mathbf{n},\kappa)$ are predicted from patch-level image features (Sec. 3.3 ), where $\mathbf{n}$ and $\kappa=1/d$ represent the local plane’s unit normal and inverse plane distance, respectively. Together with intrinsics $\mathrm{K}$ , these parameters define $\mathrm{H}=\mathrm{K}\bigl(\mathord{\hbox{\pagecolor{paramRT!12!white}\raisebox{0.0pt}[0.6pt][1.5pt]{\makebox[11.99998pt][c]{$ \textstyle\color[rgb]{0.1211,0.6094,0.1016}\vphantom{\mathbf{n}^{\top}}\mathrm{R} $}}}}+\mathord{\hbox{\pagecolor{paramKappa!12!white}\raisebox{0.0pt}[0.6pt][1.5pt]{\makebox[11.99998pt][c]{$ \textstyle\color[rgb]{0.75,0.3047,0.4297}\vphantom{\mathbf{n}^{\top}}\kappa $}}}}\mkern 1.0mu\mathord{\hbox{\pagecolor{paramRT!12!white}\raisebox{0.0pt}[0.6pt][1.5pt]{\makebox[11.99998pt][c]{$ \textstyle\color[rgb]{0.1211,0.6094,0.1016}\vphantom{\mathbf{n}^{\top}}\mathbf{t} $}}}}\mkern 1.0mu\mathord{\hbox{\pagecolor{paramN!12!white}\raisebox{0.0pt}[0.6pt][1.5pt]{\makebox[11.99998pt][c]{$ \textstyle\color[rgb]{0.2031,0.3438,0.8125}\vphantom{\mathbf{n}^{\top}}\mathbf{n}^{\top} $}}}}\bigr)$ (Eq. 5 ). Middle: token-specific transforms modulate the cross-view attention, yielding the relative factor $\mathrm{H}_{i,u}\mathrm{H}_{j,v}^{-1}$ in each attention logit (Eq. 4 ). Right: the transform is repeated block-diagonally over the geometric half of each head, while the remaining dimensions retain the $2$ D rotary blocks $\Phi_{x}$ and $\Phi_{y}$ (Eq. 17 ).

	CO3Dv2			RealEstate10K			Objaverse
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
LVSM ( Jin et al., 2025 )	18.617	0.601	0.618	25.282	0.802	0.130	29.676	0.923	0.098
GTA ( Miyato et al., 2024 )	19.482	0.612	0.573	26.076	0.824	0.110	30.500	0.929	0.079
PRoPE ( Li et al., 2025 )	19.495	0.626	0.583	26.142	0.826	0.111	30.588	0.929	0.078
RayRoPE ( Wu et al., 2026 )	18.948	0.617	0.601	26.348	0.830	0.107	30.871	0.932	0.079
URoPE ( Xie et al., 2026 )	-	-	-	26.836	0.838	0.099	30.890	0.930	0.073

Table 1: Novel view synthesis on CO3Dv2, RealEstate10K, and Objaverse. HRoPE uses image features to predict per-patch transformation parameters. Best in green , second in blue .

	$2$ views			$4$ views			$8$ views
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
LVSM ( Jin et al., 2025 )	17.718	0.459	0.474	19.331	0.571	0.317	20.441	0.615	0.273
PRoPE ( Li et al., 2025 )	18.574	0.503	0.459	19.957	0.602	0.289	20.653	0.621	0.271
HRoPE	19.284	0.544	0.447	20.373	0.624	0.270	20.940	0.635	0.257

Table 2: Scaling on the DL3DV benchmark with a

53

M LVSM under

2

4

-, and

8

-view inputs. HRoPE consistently improves over PRoPE across all three input-view settings. Best in green .

Figure 3: Qualitative novel-view synthesis on RealEstate10K and CO3Dv2. Each row shows one scene or object. From left to right, the columns contain two input views, predictions from LVSM, PRoPE, RayRoPE, and HRoPE, and the ground-truth target view (GT).

Figure 4: Ablations on RealEstate10K, measured by PSNR. (a) Transformation parameterizations using frozen VGGT features (blue) or a predictor trained from scratch (green), evaluated after

80

k training steps. The x-axis labels denote the LVSM baseline (None), a depth term (Depth), a rotation (

\mathrm{SO}(3)

), the calibrated projection composed with a per-patch tangent-frame transform (

P\!\cdot\!F

), a rigid transformation (

\mathrm{SE}(3)

), our homography parameterization (HRoPE), and an unconstrained

4\times 4

matrix (Full). Stars denote the corresponding

768

-dimensional additive positional encodings (APE). (b) Feature sources with the HRoPE parameterization fixed, evaluated after

40

k and

80

k training steps. DINOv2 † denotes the fine-tuned DINOv2 features from VGGT.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
PRoPE	26.142	0.826	0.111
PRoPE ( $P\!\cdot\!F$ )	27.115	0.840	0.104
HRoPE	27.410	0.848	0.091

Table 3: Local surface geometry on RealEstate10K (Left) & Oracle geometry on ASE (Right). On RealEstate10K, PRoPE (

P\!\cdot\!F

) and HRoPE both use VGGT features for parameter prediction. On ASE, HRoPE uses either VGGT-predicted parameters or ground-truth camera and plane parameters. Best and second-best results are highlighted in green and blue .

Figure 5: Predicted local planes on RealEstate10K. Two scenes are shown side by side; within each scene, the two rows correspond to the reference views. For each view, the columns show the input image, predicted plane orientations overlaid on the image (Visual Plane), and per-patch normals encoded as a color map (Normal Map). The red boxes mark the same approximately planar surface across views, where the predicted orientations remain locally coherent and cross-view consistent.

Figure 6: Out-of-distribution generalization on RealEstate10K. Left pair: PSNR (

\uparrow

) and LPIPS (

\downarrow

) as the unseen target-camera zoom factor increases from

1\times

5\times

. Right pair: the PSNR–LPIPS trade-off under soft target-view extrapolation and an enlarged baseline between the two input views.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Applying HRoPE to Multi-view Attention. Here, $\tau_{i,u}$ denotes the token corresponding to image patch $u$ in view $i$ . A predictor maps each patch token $\tau_{i,u}$ to the transformation parameters $(R_{i,u},\mathbf{t}_{i,u},\mathbf{n}_{i,u},\kappa_{i,u})$ , from which $H_{i,u}=K_{i}A_{i,u}$ is constructed. Queries are transformed by $\mathcal{H}_{i,u}^{\top}$ , while keys and values are transformed by $\mathcal{H}_{j,v}^{-1}$ . The aggregated value is subsequently mapped to the query token’s frame by $\mathcal{H}_{i,u}$ . Only one query–key pair is shown for clarity.

Source	PSNR $\uparrow$	GPU-hours
Original (LVSM)	$25.282$	$33$
Learnable	$26.419$	$33$
DINOv2 ( Oquab et al., 2024 )	$26.701$	$46$
DINOv2 † ( Oquab et al., 2024 )	$27.085$	$72$
DA3 ( Lin et al., 2025 )	$27.137$	$48$
VGGT ( Wang et al., 2025 )	27.410	$76$

Appendix

Table 4: Training cost of the parameter-source variants (Sec. 4.2.1 ), all under the same

80

k-iteration schedule; PSNR is the final RealEstate10K score. DINOv2 † is the fine-tuned DINOv2 taken from inside VGGT. Best in bold , second underlined .

	$1$ view			$5$ views			$10$ views
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
EscherNet + CaPE	17.689	0.858	0.181	22.065	0.899	0.114	23.666	0.914	0.089
EscherNet + HRoPE	19.900	0.883	0.135	23.075	0.911	0.092	24.515	0.923	0.074

Appendix

Table 5: Transferring HRoPE to EscherNet ( Kong et al., 2024 ) , an

\sim\!800

M-parameter multi-view diffusion model, on Objaverse. We replace EscherNet’s native CaPE encoding with HRoPE and evaluate with

1

5

, and

10

input views. Best in green .

	Numerical question				Multiple-choice question
Method	Obj. Cnt.	Abs. Dist.	Obj. Size	Room Size	Rel. Dist.	Rel. Dir.	Route Plan	Appr. Order	Avg. $\uparrow$	Rank
Proprietary models
GPT-4o ( Hurst et al., 2024 )	46.2	5.3	43.8	38.2	37.0	41.3	31.5	28.5	34.0	9
Gemini-1.5 Pro ( Team et al., 2024 )	56.2	30.9	64.1	43.6	51.3	46.3	36.0	34.6	45.4	4
Open-source models
LongVILA-8B ( Chen et al., 2025 )	29.1	9.1	16.7	0.0	29.6	30.7	32.5	25.5	21.6	13

Appendix

Table 6: Transferring HRoPE to a video multimodal LLM, evaluated on VSI-Bench ( Yang et al., 2025 ) . Our model uses LLaVA-Video-7B-Qwen2 as the backbone with HRoPE applied to the visual tokens, fine-tuned on the VSI-Bench training split at

32

frames; the preceding row reports a fine-tuned 7B video model without any geometric encoding ( Fan et al., 2026 ) . Frame budgets for the remaining methods are given in the text. Rank is taken over the overall average. Best in green .

Figure 8: Additional qualitative comparison on RealEstate10K. Each row is one scene. From left to right, the columns show novel views synthesized by LVSM, PRoPE, RayRoPE, and HRoPE (Ours), followed by the ground-truth target view (GT). All methods use the same backbone and inputs and differ only in their positional encoding.

Figure 9: Additional qualitative comparison on CO3Dv2. Each row is one object. From left to right, the columns show novel views synthesized by LVSM, PRoPE, RayRoPE, and HRoPE (Ours), followed by the ground-truth target view (GT). All methods use the same backbone and inputs and differ only in their positional encoding.

Figure 10: Additional qualitative comparison with EscherNet on Objaverse. Each row is one object. From left to right, the columns show three input views, the novel views synthesized with CaPE and HRoPE (Ours), and the ground-truth target view (GT). Both methods use the same EscherNet backbone and inputs and differ only in their positional encoding.

Positional Encoding

Positional encoding gives Transformers signals about token order, distance, and spatial or temporal relationships.

See all 116 papers

RayRoPE: Projective Ray Positional Encoding for Multi-view Attention
Jan 21, 2026Yu Wu, Minsik Jeon, Jen-Hao Rick Chang +2Multi-View 3D Reconstruction Novel View Synthesis
Carnegie Mellon University · Apple
WorldRoPE: Probabilistic World-Centric Rotary Embedding for Multi-View Attention
Oct 12, 2026Mijeong Kim, Kyujin Lim, Bohyung HanRotary Positional Embeddings Multi-View 3D Reconstruction
ECE, Seoul National University · IPAI, Seoul National University
URoPE: Universal Relative Position Embedding across Geometric Spaces
Apr 20, 2026Yichen Xie, Depu Meng, Chensheng Peng +4Rotary Positional Embeddings Relative Positional Encoding
Applied Intuition · University of California, Berkeley

Jan 21, 2026J/K move · Enter open · S save

RayRoPE: Projective Ray Positional Encoding for Multi-view Attention

Yu Wu, Minsik Jeon, Jen-Hao Rick Chang, Oncel Tuzel+1

Carnegie Mellon University · Apple

Multi-View 3D Reconstruction Novel View Synthesis Positional Encoding Attention Mechanisms Stereo Depth Estimation

Open paper AskPDF arXiv

We study positional encodings for multi-view transformers that process tokens from a set of posed input images, and seek a mechanism that encodes patches uniquely, allows SE(3)-invariant attention with multi-frequency similarity, and can adapt to the geometry of the underlying 3D scene. We find that prior (absolute or relative) encoding schemes for multi-view attention do not meet these desiderata, and present RayRoPE to address this gap. RayRoPE represents patch positions based on associated rays and computes query-frame projective coordinates to ensure SE(3) invariance. To adapt to scene geometry, RayRoPE predicts (without direct supervision) a per-token depth to obtain its position along the corresponding ray, while also modeling uncertainty and analytically computing the expected positional encoding. We validate our method on the tasks of novel-view synthesis, stereo depth estimation, and feed-forward 3DGS reconstruction. While remaining efficient, RayRoPE consistently improves over alternate position encoding schemes (e.g., 24% relative improvement on LPIPS in RE10K).

Abstract

Positional Encoding

Explore similar work

RayRoPE: Projective Ray Positional Encoding for Multi-view Attention

WorldRoPE: Probabilistic World-Centric Rotary Embedding for Multi-View Attention

URoPE: Universal Relative Position Embedding across Geometric Spaces