Relative Positional Encoding

Follow this topic Ask about this topic

Latest papers

16 papers2 papers in the last four weeks, down 33% on the four weeks before. 0.0% of all new papers.

HRoPE: Homography Relative Positional Encoding
Oct 20, 2026Huachen Gao, Yang Yang, Xiaoqian Liang +6Positional Encoding Relative Positional Encoding
Zhejiang University · Westlake University · Huazhong University of Science and Technology +2
WavePrune: One period is often enough for RoPE
Oct 3, 2026Guancheng Du, Luotian Huang, Shaowen Wang +2Rotary Positional Embeddings Positional Encoding
Tsinghua University
Analytic-Walk Rotary Positional Encodings for Graphs
Sep 26, 2026Jiaqing Xie, Yuxin Wang, Xipeng QiuGraph Positional Encoding Rotary Positional Embeddings
Shanghai Innovation Institute · Fudan University
RoPE attention is an exact forward-pass gradient step with softmax intact
Sep 6, 2026Julie Huang, Maggie Chlon, Leon ChlonSoftmax Attention Rotary Positional Embeddings
Hassana Labs · Department of Information Engineering, Oxford University
MeRoPE: Metric Rotary Position Embedding for Camera-Controlled Video Generation
Sep 1, 2026Zhijian Qiao, Xinjiang Wang, Jiajie Chen +5Rotary Positional Embeddings Camera-Controlled Video Generation
The Hong Kong University of Science and Technology · Zhuoyu Technology
Position Encoding in Transformers: From Absolute and Relative Methods to Rotary Position Embeddings and Long-Context Scaling
Aug 9, 2026Jiguo LiRotary Positional Embeddings Long-Context Modeling
Enhancing Transformer-based Routing by Encoding Distance via Relative Positional Encoding
Jul 21, 2026Leyre Encío, Daniel Fuertes, Carlos R. del-Blanco +1Relative Positional Encoding Combinatorial Optimization
Grupo de Tratamiento de Imágenes, Information Processing and Telecomunications Center, ETSI Telecomunicación, Universidad Politécnica de Madrid, 28040, Madrid, Spain
Relative Positions Generalize, Absolute Positions Memorize: An Implicit-Bias Account of Length Generalization in Attention
Jul 21, 2026Subham Singh, Ashutosh Mishra, Subha RautSoftmax Attention Rotary Positional Embeddings
Department of Mathematics and Statistics, Mississippi State University · Independent Researcher
LeRoPE: Learnable RoPE Frequencies Improve Language Modeling
Jul 11, 2026Petros Karypis, Sean O'Brien, Shreyas Kadekodi +2Language Model Pretraining Rotary Positional Embeddings
UC San Diego · Independent Researcher
PJ-RoPE: A Fourier-Jet-Affine Position Space for Relative Attention
Jun 3, 2026Yaobo ZhangRotary Positional Embeddings Relative Positional Encoding
School of Physics, Ningxia University Yinchuan, 750021, China
ReFLEX: Length-Generalizable CSI Denoising for MIMO-OFDM via Relative-Frequency Bias
May 29, 2026Zhibin Zhang, Robert Potekhin, Ziwei Wan +2Transformer Relative Positional Encoding
Moscow Institute of Physics and Technology (State University), Moscow, Russia · Yangtze Delta Region Academy, Beijing Institute of Technology (Jiaxing), Jiaxing 314019, China · School of Interdisciplinary Science, Beijing Institute of Technology, Beijing 100081, China
RelFlexformer: Efficient Attention 3D-Transformers for Integrable Relative Positional Encodings
May 11, 2026Byeongchan Kim, Arijit Sehanobish, Avinava Dubey +23D ViTs Self-Attention
1Seoul National University · 2Independent · 3Google Research +2
Jordan-RoPE: Non-Semisimple Relative Positional Encoding via Complex Jordan Blocks
May 5, 2026Yaobo ZhangRotary Positional Embeddings Self-Attention
School of Physics, Ningxia University Yinchuan, 750021, China
URoPE: Universal Relative Position Embedding across Geometric Spaces
Apr 20, 2026Yichen Xie, Depu Meng, Chensheng Peng +4Rotary Positional Embeddings Relative Positional Encoding
Applied Intuition · University of California, Berkeley
A Theoretical Analysis of Why Masked Diffusion Models Mitigate the Reversal Curse
Feb 2, 2026Moongyu Jeon, Sangwoo Shin, BumJun Kim +2Relative Positional Encoding Masked Diffusion Models
Department of Artificial Intelligence, Yonsei University
HyPE: Attention with Hyperbolic Biases for Relative Positional Encoding
Oct 30, 2023Giorgio AngelottiRelative Positional Encoding Positional Encoding
Independent Researcher.

Oct 20, 2026J/K move · Enter open · S save

HRoPE: Homography Relative Positional Encoding

Huachen Gao, Yang Yang, Xiaoqian Liang, Xingyu Chen+5

Zhejiang University · Westlake University · Huazhong University of Science and Technology+2

Positional Encoding Relative Positional Encoding Multi-View Image Synthesis Multi-View Geometry Novel View Synthesis

Open paper AskPDF arXiv

Geometric Relative Positional Encoding has become the primary mechanism for injecting spatial inductive biases into multi-view transformers. Existing designs condition attention on camera-level frustums or ray-marched depth, associating cross-view tokens through the sensor geometry. Such sensor-centric formulations, however, describe only where a token is observed from, leaving the rich surface geometry of the scene region depicted by the token. In this paper, we advocate a shift toward surface-aware positional encoding that explicitly grounds patch tokens in the geometry of physical surfaces. We introduce HRoPE, which represents the local geometry of each patch as a slanted 3D plane, following the slanted support-window assumption of classical multi-view stereo. We inject token-level, plane-induced homographies into attention as relative transformations. This equips the transformer with a geometric prior for rectifying perspective distortions and aligning patch features across viewpoints. Our encoding requires no ground-truth geometry: the plane parameters can be predicted end-to-end under a view-synthesis objective alone, or derived from foundation models. Extensive experiments on RealEstate10K, CO3Dv2, and DL3DV show that our best variant of HRoPE consistently outperforms the evaluated geometric positional encodings for novel view synthesis. Extensive ablations and analysis show the effectiveness of the proposed encoding. Please refer to our project page at: https://gaohchen.github.io/HRoPE