Discontinuous Gating

Follow this topic Ask about this topic

Momentum

5 papers in the last four weeks, up 67% on the four weeks before. 0.1% of all new papers.

Jul 6Week of Sep 21

Latest papers 36

All topics

MS-GLA: Multi-Scale Gated Linear Attention for Addressing Representational Bottlenecks via Multi-Temporal Resolution
Sep 28, 2026Prasoon Dev, Anirudh Sankar, Vasudeva VarmaLinear Attention Multi-Scale Attention
Language Technologies Research Center International Institute of Information Technology Hyderabad Hyderabad, Telangana, India
Task-Aware Discretization of Differentiable Logic Gate Networks
Sep 27, 2026Thore GerlachDifferentiable Logic Gate Networks Differentiable Optimization
Independent Researcher
On a Separate Note: Robust Score-Informed Note Separation with a Two-Stream TFC-TDF U-Net and Adaptive Set Ownership
Sep 24, 2026Benjamin Shiue-Hal Chou, Purvish Jajal, Nicholas John Eliopoulos +5Speech Separation Separation
Purdue University · Loyola University Chicago · University of Michigan
On the Importance of Gating: Memorization vs. In-Context Learning in State Space Models
Sep 15, 2026William L. Tong, Aryo Lotfi, Emmanuel Abbe +6State Space Models Sequence Modeling
Harvard University · Apple
Liquid Gated Attention
Aug 31, 2026Yiheng Jiang, Yuanbo Xu, Yongjian YangTemporal Attention Discontinuous Gating
Mobile Intelligent Computing Laboratory, College of Computer Science and Technology, Jilin University, China
"More Is Different'' in Neural Circuits: Algebraic Emergence of Effective Theories in Canonical Recurrent Motifs of Biological Neuronal Networks
Aug 31, 2026Nima DehghaniNeural Circuits Neurons
McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA · The NSF AI Institute for Artificial Intelligence and Fundamental Interactions (IAIFI), Cambridge, Massachusetts, USAa)
Hybrid Gated Attention
Aug 12, 2026Zekun Zhou, Ruobing Xie, Lanrui Wang +1Discontinuous Gating Representational Capacity
Tencent Hunyuan · Peking University
Advantage-Guided Gate: Reshaping Open-Ended Reasoning for Vision-Based Spatial Intelligence
Aug 8, 2026Ling Lin, Yang Bai, Congcong Zhu +6Spatial Reasoning Multimodal Large Language Models
School of Artificial Intelligence and Data Science, University of Science and Technology of China, Hefei 230000, China · Suzhou Institute for Advanced Research, USTC, Suzhou 215123, China · Institute of Advanced Intelligence and Computing (IAIC), Agency for Science, Technology and Research (A*STAR), Singapore +4
Hidden Gauge Controls Feature Specialization in ReLU Networks
Aug 7, 2026Tongxi WangFeature Learning Rectified Linear Unit Networks
School of Future Technology Southeast University Nanjing, China
Gated Q-learning: Add Off-Policy Bias to Taste
Jul 31, 2026Brett DaleyQ-Learning Off-Policy Learning
Meta
TPCD: Tone-Pressure Contrastive Decoding and the Label-Free Gating Bottleneck in Vision-Language Models
Jul 29, 2026Jinkun Zhao, Kui Zhang, Wenjun WuContrastive Decoding Discontinuous Gating
Beihang University
Zero-Observation User Reactivation with Gap-Driven Dimensional Gating
Jul 22, 2026Jiandong Ding, Tianying Liu, Fuyuan Liu +2Sequential Recommendation Discontinuous Gating
Fudan University Shanghai, China · Huawei Technologies Shanghai, China
Exact Network Surgery: Functional Invariance and Gradient Plasticity in Reactive Computational Graphs
Jul 18, 2026Abdallah KhemaisDiscontinuous Gating Gradient Descent
ISITCOM, University of Sousse
What Does Model Growth Really Add? Functional Capacity Beyond Parameter Count
Jul 16, 2026Dante LokCatastrophic Forgetting Continual Learning
Votee AI · Beever AI
Fully Trainable Deep Differentiable Logic Gate Networks and Lookup Table Networks
Jul 10, 2026Wout Mommen, Lars Keuninckx, Matthias Hartmann +2Differentiable Logic Gate Networks Mnist
imec, Kapeldreef 75, Leuven, 3001, , Belgium · Vrije Universiteit Brussel, Peinlaan 2, Elsene, 1050, , Belgium
A Structural Interpretation of GELU and Threshold-Transmission Activations via the First-Order Loss Function
Jul 4, 2026Roberto RossiDiscontinuous Gating Loss Function
Business School, University of Edinburgh, Edinburgh, UK
Rethinking Neural Nonlinearity as Gating
Jul 3, 2026Muhammad Sabih, Frank Hannig, Jürgen TeichActivation Functions Discontinuous Gating
Department of Computer Science, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Germany
Gated MLPs as Symmetry-Broken Rank-1 Bilinear Attention
Jun 20, 2026Nathan BreslowDiscontinuous Gating Multilayer Perceptrons
Learning Hybrid Biophysical Neuron Models with Neural ODEs
Jun 15, 2026Jonas Beck, Michael Deistler, Dóra Viktória Molnár +2Neural Ordinary Differential Equations Neurons
Hertie Institute for AI in Brain Health, Tübingen, Germany · Tübingen AI Center, University of Tübingen, Tübingen, Germany · Machine Learning in Science, Excellence Cluster Machine Learning, University of Tübingen, Tübingen, Germany +2
Provenance-Grounded Gating and Adaptive Recovery in Synthetic Post-Training Data Curation
Jun 9, 2026Soham Bhattacharjee, Karun Sharma, Vinay Kumar Sankarapu +1Post-Training Data-Curation
Lexsi Labs
Lane Change Trajectory Planning for Personalized Driving Comfort and Mobility Efficiency
Jun 5, 2026Haoxuan Dong, Dongjun Li, Ziyou SongMobility Single Trajectory
Department of Mechanical Engineering National University of Singapore Singapore 117575, Singapore · Department of Electrical Engineering and Computer Science University of Michigan Ann Arbor, MI 48109, USA
BRIDGE: Biological Evidence Refinement and Heterogeneous Dynamic Gating for Gene Regulatory Networks
Jun 2, 2026Ziyang Dong, Shanwen Tan, Hengchuang Yin +5Regulatory Networks Single-Cell Rna
College of Computer Science, Sichuan University, Chengdu, 610065, China · Xinjiang Technical Institute of Physics and Chemistry, Chinese Academy of Sciences, Urumqi, 830011, China · School of Mathematics, Sichuan University, Chengdu, 610065, China +2
Composing Non-Conjugate Factor Graphs with Closed-Form Variational Inference
May 28, 2026Mykola Lukashchuk, Kyrylo Yemets, Wouter M. Kouw +4Variational Inference Factor Graphs
Eindhoven University of Technology, the Netherlands · Lviv Polytechnic National University, Lviv, Ukraine · Lazy Dynamics, Utrecht, the Netherlands +1
RotMoLE: Enhancing Mixture of Low-Rank Experts through Rotational Gating Mechanism
May 25, 2026Mengyang Sun, Maochuan Dou, Tao Feng +5Mixture-Of-Experts Large Language Models Mixture-Of-Experts
Tsinghua University · Beijing Information Science and Technology University · National University of Singapore +2
OSDN: Improving Delta Rule with Provable Online Preconditioning in Linear Attention
May 13, 2026Chenyu Zhou, Hongpei Li, Yuerou Liu +3Kimi Delta Attention Linear Attention
Shanghai Jiao Tong University · Northwestern University · Huazhong University of Science and Technology +1
A Random-Matrix Criterion for Initializing Gated Recurrent Neural Networks
May 11, 2026Tommaso Fioratti, Riccardo Marcaccioli, Francesco CasolaRecurrent Neural Networks Discontinuous Gating
Capital Fund Management, 75007 Paris, France · Institute of Mathematics, EPFL, 1015 Lausanne, Switzerland
Fitting Multilinear Polynomials for Logic Gate Networks
May 9, 2026Youngsung KimDifferentiable Logic Gate Networks Polynomials
Department of AI; and Department of ECE, Inha University
Inductive Power Grid Cascading Failure Analysis with GRU-Gated Graph Attention
May 7, 2026Tianxin Zhou, Xiang Li, Haibing LuPower Systems Interleaved Graph Attention
Department of Computer Science & Engineering, Santa Clara University, Santa Clara, USA · Dept. of Information Systems & Analytics, Santa Clara University, Santa Clara, USA
MILE: Mixture of Incremental LoRA Experts for Continual Semantic Segmentation across Domains and Modalities
May 5, 2026Shishir Muralidhara, Didier Stricker, René SchusterEfficient Semantic Segmentation Semantic Segmentation
German Research Center for Artificial Intelligence (DFKI), Trippstadter Str. 122, 67663 Kaiserslautern, Germany · University of Kaiserslautern-Landau (RPTU), Gottlieb-Daimler-Str. 47, 67663 Kaiserslautern, Germany
Where to Focus: Query-Modulated Multimodal Keyframe Selection for Long Video Understanding
Apr 19, 2026Shaoguang Wang, Weiyu Guo, Ziyang Chen +2Video Understanding Keyframe Selection
Thrust of Artificial Intelligence, HKUST (Guangzhou) Guangzhou, China · Department of CSE, HKUST Hong Kong SAR, China
Revisiting Auxiliary Losses for Conditional Depth Routing: An Empirical Study
Apr 19, 2026Qingwei LinDiscontinuous Gating Empirical Study
Independent Researcher
Gating Enables Curvature: A Geometric Expressivity Gap in Attention
Apr 16, 2026Satwik Bathula, Anand A. JoshiDiscontinuous Gating Local Curvature
Department of Electrical and Computer Engineering University of Southern California
TacVLA: Contact-Aware Tactile Fusion for Robust Vision-Language-Action Manipulation
Mar 13, 2026Kaidi Zhang, Heng Zhang, Zhengtong Xu +7Diffusion-Based Vision-Language-Actions Tactile
Edwardson School of Industrial Engineering, Purdue University, West Lafayette, IN 47907, USA · Human-Robot Interfaces and Interaction Lab, Istituto Italiano di Tecnologia, Genova, Italy · Ph.D. program of national interest in Robotics and Intelligent Machines (DRIM) and Universit`a di Genova, Genoa, Italy
Geometric Observability Index: An Operator-Theoretic Framework for Per-Feature Sensitivity, Weak Observability, and Dynamic Effects in SE(3) Pose Estimation
Feb 5, 2026Joe-Mei Feng, Sheng-Wei Yu, Hsin-Hsiung KaoRobust Geometric Model Estimation Partial Observability
Department of Mathematics, Tamkang University, New Taipei, Taiwan · Department of Information Management, Central Police University, Taoyuan, Taiwan
A Systematic Analysis of Hybrid Linear Attention
Jul 8, 2025Dustin Wang, Rui-Jie Zhu, Steven Abreu +9Kimi Delta Attention Transformer Architectures
1UC Santa Cruz · 2ByteDance Seed · University of Groningen +3
Regularity and Stability Properties of Selective SSMs with Discontinuous Gating
May 16, 2025Nikola Zubić, Davide ScaramuzzaState Space Models Discontinuous Gating
Robotics and Perception Group University of Zurich

Sep 28, 2026, cs.CLJ/K move · Enter open · S save

MS-GLA: Multi-Scale Gated Linear Attention for Addressing Representational Bottlenecks via Multi-Temporal Resolution

Prasoon Dev, Anirudh Sankar, Vasudeva Varma

Language Technologies Research Center International Institute of Information Technology Hyderabad Hyderabad, Telangana, India

Linear Attention Multi-Scale Attention Discontinuous Gating Recurrent Model Bottlenecks

Open paper AskPDF arXiv

Gated Linear Attention (GLA) Transformers advance linear recurrent models through data-dependent gating, but face a core limitation: the fixed-capacity memory matrices across all heads operate at a single temporal resolution, where each token is processed individually, forcing them to simultaneously encode local syntactic patterns and long-range semantic structure, creating a representational bottleneck that gating alone is insufficient to resolve. We introduce Multi-Scale Gated Linear Attention (MS-GLA), which addresses this by distributing attention heads across multiple temporal resolutions. Coarser resolutions pool longer token spans naturally specializing toward long-range dependencies, while finer head groups retain sensitivity to local syntactic structure. A learnable, input-dependent fusion layer dynamically recombines head group outputs at each timestep, expanding effective memory capacity without increasing per-head state size. This multi-resolution decomposition draws on principles from Multi-Scale State-Space Models (MS-SSM), adapting them to the gated linear attention setting. We evaluate MS-GLA on language modeling, recall-intensive tasks, and long-context generalization. Across all settings, MS-GLA consistently achieves higher accuracy and lower perplexity than GLA at matched parameter counts, with up to 18.9% improvement on recall-intensive tasks and 9.5% lower average perplexity on language modeling benchmarks, validating multi-temporal resolution decomposition as a principled and effective extension of Gated Linear Attention.

Momentum

Related topics

Latest papers 36