Relative Positional Encoding

Latest papers

CardsList

17 papers2 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

  1. Fast and Exact Relative-Position Attention via Structure-Aware Kernels

    Oct 31, 2026Delyan Boychev, Ivo Petrov, Dimitar I. DimitrovEfficient AttentionMemory-Efficient Attention

  2. HRoPE: Homography Relative Positional Encoding

    Oct 20, 2026Huachen Gao, Yang Yang, Xiaoqian Liang +6Positional EncodingRelative Positional Encoding

  3. WavePrune: One period is often enough for RoPE

    Oct 3, 2026Guancheng Du, Luotian Huang, Shaowen Wang +2Rotary Positional EmbeddingsPositional Encoding

  4. Analytic-Walk Rotary Positional Encodings for Graphs

    Sep 26, 2026Jiaqing Xie, Yuxin Wang, Xipeng QiuGraph Positional EncodingRotary Positional Embeddings

  5. RoPE attention is an exact forward-pass gradient step with softmax intact

    Sep 6, 2026Julie Huang, Maggie Chlon, Leon ChlonSoftmax AttentionRotary Positional Embeddings

  6. MeRoPE: Metric Rotary Position Embedding for Camera-Controlled Video Generation

    Sep 1, 2026Zhijian Qiao, Xinjiang Wang, Jiajie Chen +5Rotary Positional EmbeddingsCamera-Controlled Video Generation

  7. Enhancing Transformer-based Routing by Encoding Distance via Relative Positional Encoding

    Jul 21, 2026Leyre Encío, Daniel Fuertes, Carlos R. del-Blanco +1Relative Positional EncodingCombinatorial Optimization

  8. Relative Positions Generalize, Absolute Positions Memorize: An Implicit-Bias Account of Length Generalization in Attention

    Jul 21, 2026Subham Singh, Ashutosh Mishra, Subha RautSoftmax AttentionRotary Positional Embeddings

  9. LeRoPE: Learnable RoPE Frequencies Improve Language Modeling

    Jul 11, 2026Petros Karypis, Sean O'Brien, Shreyas Kadekodi +2Language Model PretrainingRotary Positional Embeddings

  10. PJ-RoPE: A Fourier-Jet-Affine Position Space for Relative Attention

    Jun 3, 2026Yaobo ZhangRotary Positional EmbeddingsRelative Positional Encoding

  11. ReFLEX: Length-Generalizable CSI Denoising for MIMO-OFDM via Relative-Frequency Bias

    May 29, 2026Zhibin Zhang, Robert Potekhin, Ziwei Wan +2TransformerRelative Positional Encoding

  12. RelFlexformer: Efficient Attention 3D-Transformers for Integrable Relative Positional Encodings

    May 11, 2026Byeongchan Kim, Arijit Sehanobish, Avinava Dubey +23D ViTsSelf-Attention

  13. Jordan-RoPE: Non-Semisimple Relative Positional Encoding via Complex Jordan Blocks

    May 5, 2026Yaobo ZhangRotary Positional EmbeddingsSelf-Attention

  14. URoPE: Universal Relative Position Embedding across Geometric Spaces

    Apr 20, 2026Yichen Xie, Depu Meng, Chensheng Peng +4Rotary Positional EmbeddingsRelative Positional Encoding

  15. A Theoretical Analysis of Why Masked Diffusion Models Mitigate the Reversal Curse

    Feb 2, 2026Moongyu Jeon, Sangwoo Shin, BumJun Kim +2Relative Positional EncodingMasked Diffusion Models