Low-Rank Attention

Ask about this topic

Momentum

2 papers in the last four weeks, down 50% on the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 15

All topics
CardsList

  1. Tucker Bottleneck Attention for Multi-Dimensional Sequence Modeling

    Oct 6, 2026Ryan Solgi, Parsa Madinei, Zheng ZhangAttention MechanismsLow-Rank Attention

    University of California-Santa Barbara Santa Barbara, CA, USA

  2. Prescriptive SVD-Inspired Attention via Spectral Energy Retention

    Sep 21, 2026Vasileios Arampatzakis, Vasileios Sevetlidis, George PavlidisTransformer InterpretabilityLow-Rank Attention

    Athena Research Center, Greece

  3. Nyström Attention Matches Full Attention for Cross-Sectional Stock Prediction

    Sep 8, 2026Kunhan GuoLow-Rank AttentionFinancial Forecasting

    The University of Hong Kong

  4. RoLA: Rotary-Positioned Low-Rank Linear Attention for Efficient Diffusion Transformers

    Sep 6, 2026Zekun Zhang, Yixiang Cai, Yuxi Liu +9Rotary Positional EmbeddingsLow-Rank Attention

    Peking University, Melon Group · Alibaba Group · University of Electronic Science and Technology of China +1

  5. Intrinsic Interaction Geometry Controls the Low-Rank Complexity of Softmax Attention

    Aug 28, 2026Yuhe Sui, Jianing Zhang, Yingzhi TangSoftmax AttentionLow-Rank Attention

    Nanyang Technological University, Singapore · Carnegie Mellon University

  6. FLARE++: Low-rank attention with dynamic attention routing

    Aug 12, 2026Vedant Puri, Yongjie Jessica Zhang, Levent Burak KaraLow-Rank AttentionPDE Surrogate Modeling

    Department of Mechanical Engineering Carnegie Mellon University Pittsburgh, Pennsylvania, USA

  7. Three Tokens Force Exponential Feature Rank in Nonnegative Kernel Attention

    Aug 11, 2026Vicente OpazoLow-Rank AttentionAttention Mechanisms

    CENIA

  8. SAKI: Score-Aware Low-Rank Key Indexing with Random-Matrix Noise Correction for KV Retrieval

    Aug 4, 2026Lin ZhangLow-Rank AttentionKV Caching

    Data Science and Platform, Cotality

  9. LOCKS: Page-Local Compact Key Summaries for Efficient Long-Context Decoding

    Jul 27, 2026Junsung HwangLow-Rank AttentionKV Caching

  10. Through the Bottleneck: How Multi-head Latent Attention Separates Content from Position in Language Models

    Jul 25, 2026Dhruvil S, Fenil Sojitra, Ravirajsinh ChauhanTransformer InterpretabilityLow-Rank Attention

    1Indian Institute of Technology Madras · 2P P Savani University

  11. ELSAA: Efficient Low-Rank and Sparse Attention Approximation for Training Transformers

    Jul 22, 2026Mahdi Heidari, Mohammad Mahdi Rahimi, Jaekyun MoonLow-Rank AttentionTransformer Attention

    School of Electrical Engineering, KAIST, Daejeon 34141, Republic of Korea · Department of Electrical Engineering and Computer Science, Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu 42988, Republic of Korea

  12. Low-Rank Attention Residuals

    Jun 19, 2026Jonathan SuLow-Rank AttentionDecoder-Only Language Models

    Independent Researcher

  13. Hierarchical Attention via Domain Decomposition

    Jun 16, 2026Stephan Köhler, Oliver RheinbachLow-Rank AttentionPDE Operator Learning

    Faculty of Mathematics and Computer Science Technische Universität Bergakademie Freiberg, 09596 Freiberg, Germany

  14. High-Fidelity Industrial Crash Dynamics Prediction via Geometry-Aware Operator Learning with Memory-Efficient Low-Rank Attention

    May 26, 2026Deepak Akhare, Mohammad Amin Nabian, Corey Adams +2Low-Rank AttentionSurrogate Modeling

    Department of Aerospace and Mechanical Engineering, University of Notre Dame, Notre Dame, IN · NVIDIA, San Jose, CA · General Motors, Detroit, MI

  15. RoPeSLR: 3D RoPE-driven Sparse-LowRank Attention for Efficient Diffusion Transformers

    May 20, 2026Yuxi Liu, Zekun Zhang, Yixiang Cai +3Rotary Positional EmbeddingsLow-Rank Attention

    Peking University · University of Electronic Science and Technology of China · Beijing University of Posts and Telecommunications