Positional Encoding

Follow this topic Ask about this topic

Momentum

12 papers in the last four weeks, up 300% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 85

All topics

Higher-Order Positional Encodings for Graph Representation Learning
Oct 1, 2026Caleb Stam, Aagrim Hoysal, Sanjukta KrishnagopalGraph Representation Learning Positional Encoding
University of California, Santa Barbara
Shifting Mechanisms: How Positional Encoding Choice Shapes In-Context Retrieval
Sep 29, 2026Eric Enouen, Sainyam GalhotraPositional Encoding Efficient Long-Context Inference
Cornell University
How Local Mixing Encodes Relative Position in Global NoPE Attention
Sep 29, 2026Cutter Dawes, Nick Alonso, Tom Figliolia +1Positional Encoding Transformer Architectures
Zyphra Research
Motion Concept Unlearning in Video Diffusion Models
Sep 29, 2026Ping Liu, Chi ZhangText-To-Video Diffusion Models Video Diffusion Models
Computer Science and Engineering University of Nevada, Reno Reno, NV, USA · Department of Mathematics National University of Singapore Singapore
CoDrive: Cross-Vehicle World-Consistent Video Generation with Precise Trajectory Control for Cooperative Driving
Sep 28, 2026Yu Meng, Baining Zhao, Junta Wu +10Drives Camera Trajectories
Tsinghua University · Tencent Hunyuan · Zhongguancun Academy
DirectUV: Image-Conditioned UV Texture Generation with Surface-Aware Positional Encoding
Sep 28, 2026Jiantao Lin, Yingjie Xu, Mingzhi Sheng +3Texture Extreme Ultraviolet
The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology
RoPE is Dead, Long Live RoPE: Towards Scalable Data-aware Positional Encodings
Sep 28, 2026Jarod Lévy, Mathurin Videau, Jad Yehya +3Rotary Position Embedding Positional Encoding
Meta AI, Paris · Inria, Université Paris-Saclay, Palaiseau, France
MinkowskiPE: Minkowski Positional Encoding for Spatiotemporal Perception
Sep 27, 2026Yuhao Li, Louie Hong Yao, Tianyi Shi +4Positional Encoding Spatio-Temporal Reasoning
Wave Intelligence Lab · Department of Computer Science and Engineering, The Chinese University of Hong Kong · Shanghai Artificial Intelligence Laboratory
Layer-Aware Position Embeddings for Visual Token Pruning in Multimodal Large Language Models
Sep 20, 2026Yahong Wang, Zhangkai Ni, Juncheng Wu +3Visual Token Pruning Multimodal Large Language Models
Tongji University · University of California, Santa Cruz · East China Normal University +1
Accelerated Decoding of Centroid Positional Encoding for Instance Segmentation
Sep 15, 2026Carmelo Scribano, Filippo Muzzini, Nedyalko Prisadnikov +6Intermediate Decoder Layers Positional Encoding
University of Modena and Reggio Emilia, Italy · INSAIT, Sofia University “St. Kliment Ohridski”, Bulgaria
G-ray: Ray-Level Relative Geometric Position Encoding in Multi-View Vision Transformers under Camera Heterogeneity
Sep 14, 2026Shuo Zhang, Xin Su, Wei Wang +8Visual Geometry Grounded Transformer Positional Encoding
Wuhan University, Wuhan, China · Rongyun Robot (Guizhou) Co., Ltd. · Institute of Automation, Chinese Academy of Sciences, Beijing, China
Distance generalization in transformers: why bother with positional encoding?
Sep 11, 2026Daniel Henrik Nevermann, Claudius GrosPositional Encoding Transformer Architectures
Institute for Theoretical Physics, Goethe University Frankfurt, Germany
RoLA: Rotary-Positioned Low-Rank Linear Attention for Efficient Diffusion Transformers
Sep 6, 2026Zekun Zhang, Yixiang Cai, Yuxi Liu +9Video Diffusion Models Spatio-Temporal Attention
Peking University, Melon Group · Alibaba Group · University of Electronic Science and Technology of China +1
Converse and Collision-Based Achievability for Node Localization with Hybrid Distance-Spectral Graph Positional Encodings
Aug 31, 2026Zimo Yan, Yifan Li, Hao Li +4Positional Encoding Graph Laplacians
National University of Defense Technology, Changsha, China · Wuhan University, Wuhan, China
ZetaGPT: A Reference Implementation of Positional--Encoding--Free State--Space--Attention Language Models
Aug 10, 2026Róisín LuoPositional Encoding Linear Attention
University of Galway, Ireland
Position Encoding in Transformers: From Absolute and Relative Methods to Rotary Position Embeddings and Long-Context Scaling
Aug 9, 2026Jiguo LiRotary Position Embedding Positional Encoding
VectraYX-Vision-1B: A Sub-2B Spanish/LATAM Cybersecurity Vision-Language Model with Structured Visual Reasoning and Native Tool Use
Aug 9, 2026Juan S. SantillanaRecent Vision-Language Models Large Language Model Safety
Globant
When Attention Goes Blind: Numerical Failure in ALiBi Positional Encodings
Aug 4, 2026Christopher Schröder, Lukas Gienapp, Ferdinand Schlatt +2Positional Encoding Degradation
Institute for Applied Informatics at Leipzig University (InfAI) · ScaDS.AI Dresden/Leipzig · Seltz +1
MDLMPE: Distribution Aware Positional Encoding for Masked Diffusion Language Models
Aug 4, 2026Tong Ling, Hang Lei, Feng Xiao +5Masked Diffusion Language Models Diffusion Language Models
University Chinese Academic of Science · HUJING Digital Media & Entertainment Group · State Key Laboratory for Novel Software Technology, Nanjing University, China +1
Learning to Tessellate: Point Cloud Generation via Recursive Spectral Partitioning
Aug 3, 2026Monan Sun, Bangzhen Liu, Huaidong Zhang +1Point Clouds Partitioning
South China University of Technology, China · University of Chinese Academy of Sciences, China · Singapore Management University, Singapore +1
Latent Stability Analysis of Malware Representations Under Feature-Space Perturbations
Jul 27, 2026Bamidele Ajayi, Ken McGarryMalware Evasion
School of Computer Science and Engineering, University of Sunderland, Sunderland, UK
Anti-Periodic Positional Encoding: Möbius Boundary Conditions Make In-Context Retrieval Reliable
Jul 23, 2026Ji Ho BaeRotary Position Embedding Positional Encoding
JRTI, Seoul, 05855, Republic of Korea
Hash-QNeRF: Multiresolution Hash Encoding for Quantum Neural Radiance Fields
Jul 23, 2026Digonto Biswas, Tana Ballove, Anjan Bandyopadhyay +2Quantum Neural Networks Neural Radiance Fields
Centres of Innovation and Research KIIT Deemed to be University Bhubaneswar, India
AI-Driven Surrogate Models for Predicting Electrode-Scale Discharge Behavior in Lithium-Ion Batteries
Jul 22, 2026Mengda Xing, Jean-Marie Lagniez, Alejandro FrancoLithium-Ion Batteries Surrogate Models
CRIL, UA · CRIL, UMR 8188, Université d’Artois, Rue Jean Souvraz SP 18, F-62307 Lens Cedex, France · LRCS +1
Enhancing Transformer-based Routing by Encoding Distance via Relative Positional Encoding
Jul 21, 2026Leyre Encío, Daniel Fuertes, Carlos R. del-Blanco +1Multiplex Graph Transformers Positional Encoding
Grupo de Tratamiento de Imágenes, Information Processing and Telecomunications Center, ETSI Telecomunicación, Universidad Politécnica de Madrid, 28040, Madrid, Spain
Relative Positions Generalize, Absolute Positions Memorize: An Implicit-Bias Account of Length Generalization in Attention
Jul 21, 2026Subham Singh, Ashutosh Mishra, Subha RautPositional Encoding Relative Position
Department of Mathematics and Statistics, Mississippi State University · Independent Researcher
Frontier Language Models Struggle to Copy: Text Can Be Better Viewed in 2D
Jul 17, 2026Haodong Wen, Yiran Zhang, Yingfa Chen +1Language Modeling Positional Encoding
Tsinghua University
PE-Field 4D: Video Generation Models as Canvas
Jul 17, 2026Yunpeng Bai, Haoxiang Li, Qixing HuangVideo Generation Camera Trajectories
University of Texas at Austin, USA · Pixocial Technology, USA
What Does a Temporal Benchmark Score Measure? Decomposing Channel Use in Video VLM Evaluation
Jul 14, 2026Farrukh RahmanReverse Positional Encoding
Microsoft · Georgia Institute of Technology
LeRoPE: Learnable RoPE Frequencies Improve Language Modeling
Jul 11, 2026Petros Karypis, Sean O'Brien, Shreyas Kadekodi +2Rotary Position Embedding Positional Encoding
UC San Diego · Independent Researcher
Eigenbasis-Independent Learnable Spectral Positional Encodings for Directed Graphs via Hermitian Block Krylov Subspaces
Jul 8, 2026Jiaqing Xie, Yuxin WangGraph Edge Sparsification Graph Laplacians
Fudan University
MemDefrag: Latent Memory Defragmentation for Large Language Models
Jul 7, 2026Ruiyi Yan, Zhuoyuan Mao, Yiwen GuoLarge Language Model Memory Long-Term Memory
Kyoto University · LIGHTSPEED · Independent Researcher
Observable- and Positional-Encoding-Dependent Symmetry Readout from Neural Network Weights
Jul 3, 2026Naoya Chiba, Satoshi Sugiyama, Yuki UranishiSymmetry Model Weights
D3 Center, The Unversity of Osaka
HyFL-CLIP: Hyperbolic Fine-Tuning of CLIP for Robust Long-Context Understanding
Jul 1, 2026Ji Ha Jang, Hayeon Kim, Chulwon Lee +2Contrastive Language-Image Pre-Training Model Efficient Long-Context Inference
1Dept. of Electrical and Computer Engineering, 2IPAI, 3INMC & AIIS, Seoul National University, Republic of Korea
DPPE: Rethinking Camera-Based Positional Encoding for Scaling Multi-View Transformers
Jun 30, 2026Shun Kenney, Teppei SuzukiPositional Encoding Detection Transformer
Keio University · Work was done during an internship at SB Intuitions. · SB Intuitions
DMV-Bench: Diagnosing Long-Horizon Multimodal Agents' Visual Memory with Incidental Cue Injection
Jun 25, 2026Yujin Tang, Chenming Shang, Ruize Xu +1Multimodal Memory Visual Memory
Dartmouth College
RayPE: Ray-Space Positional Encoding for 3D-Aware Video Generation
Jun 25, 2026Minghao Yin, Jiahao Lu, Wenbo Hu +3Video Diffusion Models Positional Encoding
1The University of Hong Kong · 2The Hong Kong University of Science and Technology · 3ARC Lab, Tencent
Communicability-Inspired Positional Encoding (CIPE)
Jun 24, 2026Yipeng Zhang, Zhongtian Sun, Pietro Liò +1Multiplex Graph Transformers Positional Encoding
Division of Mathematical Sciences, School of Physical and Mathematical Sciences, Nanyang Technological University, Singapore 637371, Singapore · School of Computing, University of Kent, Canterbury, Kent CT2 7NZ, UK · Department of Computer Science and Technology, University of Cambridge, 15 JJ Thomson Avenue, Cambridge CB3 0FD, UK
Randomized YaRN Improves Length Generalization for Long-Context Reasoning
Jun 22, 2026Manas Mehta, Fangcong Yin, Greg DurrettEfficient Long-Context Inference Long-Context Reasoning
New York University
Robustness of Similarity-based Positional Encoding Under Rotations: Theoretical Analysis and Experimental Validation
Jun 16, 2026Andrea Santomauro, Luigi Portinale, Giorgio LeonardiPositional Encoding Transformer Architectures
Computer Science Institute, DiSIT, University of Piemonte Orientale, Alessandria, Italy
Functional Equivalence in Attention: A Comprehensive Study with Applications to Linear Mode Connectivity
Jun 16, 2026Viet-Hoang Tran, Vinh Khanh Bui, Van-Hoan Trinh +2Positional Encoding Transformer Architectures
National University of Singapore · Center for AI Research, VinUniversity · Technical University of Munich +1
DreamX-World 1.0: A General-Purpose Interactive World Model
Jun 15, 2026DreamX Team, Yancheng Bai, Rui Chen +20Video World Models World Models
DreamX Team
ParkingTransformer: LLM-Enhanced End-to-End Trajectory Planning for Autonomous Parking
Jun 12, 2026Hauteng Wu, Xu Li, Dong Kong +4Autonomous Driving Autonomous Navigation
School of Instrument Science and Engineering, Southeast University, Nanjing 210096, China · School of Electronic and Information Engineering, Tongji University, Shanghai, 201804, China · College of Transportation, Shandong University of Science and Technology, Qingdao 266590, China +1
Understanding Truncated Positional Encodings for Graph Neural Networks
Jun 11, 2026James Flora, Mitchell Black, Weng-Keen Wong +1Graph Neural Networks Positional Encoding
School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, Oregon, USA · Halıcıoğlu Data Science Institute, University of California San Diego, San Diego, California, USA.
Positional Encoding in the Context of Memristor-Based Analog Computation for Automatic Speech Recognition
Jun 11, 2026Benedikt Hilmes, Nick Rossenbach, Ralf SchlüterIn-Memory Computing Reram Crossbar
Machine Learning and Human Language Technology Group, Faculty of Computer Science, RWTH Aachen University, Aachen, Germany
LazyAttention: Efficient Retrieval-Augmented Generation with Deferred Positional Encoding
Jun 3, 2026Haocheng Xia, Mihir Pamnani, Hanxi Fang +2Positional Encoding
Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign · Nexla · Amazon +1
Rethinking the Role of Positional Encoding: Sliding-Window Transformers without PE Remain Turing Complete
Jun 1, 2026Qian Li, Xinyu Mao, Shang-Hua TengPositional Encoding Transformer Architectures
Shenzhen Research Institute of Big Data · University of Southern California
Beyond Sinusoids: A Morlet Wavelet Framework for Transformer Positional Encoding
May 31, 2026Athanasios ZerisPositional Encoding Rotary Position Embedding
Independent Researcher, Athens, Greece.
Harmonic: Hierarchical State Space Models for Efficient Long-Context Language Modeling
May 30, 2026Petr NyomaLanguage Modeling Efficient Long-Context Inference
Independent Researcher
Anti Mode-Collapse in Mean-Field Transformer via Auxiliary Variables
May 28, 2026Masaaki Imaizumi, Masanori Koyama, Noboru Isobe +1Transformer Attention Transformer Architectures
The University of Tokyo, Tokyo, Japan · RIKEN Advanced Intelligene Project, Tokyo, Japan · Kyoto University, Kyoto, Japan
Give it Space! Explicit Disentangling of Positional and Semantic Representations in Encoders
May 28, 2026Pierre-Antoine Lequeu, Camille Barboule, Benjamin PiwowarskiPositional Encoding Transformer Encoder
Sorbonne Université, CNRS, ISIR, Paris, France · Orange Innovation
Positional Encodings Anchor Spatial Structure in Vision Transformers: A Geometric Perspective on Robustness
May 28, 2026Mahmoud MannesPositional Encoding Vision Transformer
ESSTHS
Benchmarking Positional Encoding Strategies for Transformer-Based EEG Foundation Models
May 28, 2026Ayse Betul Yuce, Sebastian StoberElectroencephalography Foundation Models Electroencephalography
Department of Computer Science, Otto von Guericke University, Magdeburg, Germany
Periodic RoPE for Infinite Context LLMs
May 27, 2026Simin HuoEfficient Long-Context Inference Long-Context Reasoning
Shanghai Jiao Tong University
LaneRoPE: Positional Encoding for Collaborative Parallel Reasoning and Generation
May 26, 2026Gabriele Cesa, Thomas Hehn, Aleix Torres-Camps +4LLM Inference Optimization Positional Encoding
Qualcomm AI Research∗
Energy-Gated Attention and Wavelet Positional Encoding: Complementary Inductive Biases for Transformer Attention
May 25, 2026Athanasios ZerisTransformer Attention Positional Encoding
Independent Researcher, Athens, Greece.
Building an Adversarial Malware Dataset by Family and Type: Generation, Evasion, and Poisoning Evaluation
May 25, 2026David Košťál, Martin JurečekMalware Poisoning
Department of Information Security, Faculty of Information Technology, Czech Technical University in Prague
Weierstrass Positional Encoding for Vision Transformers
May 20, 2026Zhihang Xin, Rui Wang, Xitong Hu +1Positional Encoding Vision Transformer
School of Mathematics and Data Science, Jiangnan University, Wuxi 214122, China · School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi 214122, China
MetaBackdoor: Exploiting Positional Encoding as a Backdoor Attack Surface in LLMs
May 14, 2026Rui Wen, Mark Russinovich, Andrew Paverd +2Fewer Surprising Backdoors Attacker Large Language Model
Institute of Science Tokyo · 2Microsoft Azure · 3Microsoft Security Response Center
CRePE: Curved Ray Expectation Positional Encoding for Unified-Camera-Controlled Video Generation
May 13, 2026Seonghyun Jin, Youngmin Kim, Sunwoo Park +1Camera Control Camera-Conditioned Video Generation
KAIST · Graduate School of AI Korea University · Graduate School of AI KAIST

Oct 1, 2026, cs.LGJ/K move · Enter open · S save

Higher-Order Positional Encodings for Graph Representation Learning

Caleb Stam, Aagrim Hoysal, Sanjukta Krishnagopal

University of California, Santa Barbara

Graph Representation Learning Positional Encoding Higher-Order Structures Neural Network Representations

Open paper AskPDF arXiv

Many real-world systems exhibit higher-order interactions among groups of entities that cannot be captured by pairwise relationships alone. Graph Transformers and Graph Neural Networks increasingly rely on positional encodings to enrich graph representations, yet existing positional encodings are computed solely from the original graph and therefore cannot directly capture observed higher-order interactions. Topological Deep Learning addresses this limitation by lifting graphs to simplicial complexes, but typically requires performing message passing or attention on higher-order neural network representations. We introduce a representation learning paradigm that enriches graph representations with higher-order topology through positional encodings, enabling standard graph learning models to exploit lifted incidence structure without modifying the backbone. We derive a theoretical characterization of the expressivity of higher-order positional encodings, proving that node-level operators induced by higher-order lifts can mix graph Laplacian frequencies in ways that scalar graph spectral filters cannot. Guided by this theory, we instantiate higher-order positional encodings using Hodge Laplacians derived from clique complexes. Experiments with Graph Transformers on ZINC and controlled synthetic benchmarks demonstrate improvements in predictive performance, while a fixed-1-skeleton experiment shows that the pipeline can transmit higher-order information when cells are supplied independently of the graph. Together, our results establish higher-order positional encodings as a principled bridge between graph positional encodings and topological deep learning.

Momentum

Related topics

Latest papers 85