Attention Mechanisms

Follow this topic Ask about this topic

Momentum

42 papers in the last four weeks, up 100% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 378

All topics

Look Beyond Saliency: Low-Attention Guided Dual Encoding for Video Semantic Search
May 7, 2026Faisal Aljehrai, Mohammed A. Alkhrashi, Alreem Almuhrij +6Visual Representation Learning Information Retrieval
Saudi Data And Artificial Intelligence Authority (SDAIA)
Neuromorphic visual attention for Sign-language recognition on SpiNNaker
May 7, 2026Sarka Liskova, Olha Vedmedenko, Mazdak Fatahi +3Event-Based Vision Visual Attention
Dept. of Cybernetics, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic · Faculty of Information Technology, Czech Technical University in Prague, Czech Republic
Retrieval from Within: An Intrinsic Capability of Attention-Based Models
May 7, 2026Elad Hoffer, Yochai Blau, Edan Kinderman +3Retrieval-Augmented Generation Question Answering
NVIDIA · Department of Electrical Engineering, Technion, Haifa, Israel
Large Vision-Language Models Get Lost in Attention
May 7, 2026Gongli Xi, Ye Tian, Mengyu Yang +5Transformer FFNs Vision-Language Models
School of Cyberspace Security, Beijing University of Posts and Telecommunications, Beijing, China · State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China · School of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications, Beijing, China +3
Ensuring Reliability in Programming Knowledge Tracing: A Re-evaluation of Attention-augmented Models and Experimental Protocols
May 6, 2026Jaewook Kim, Hyeoncheol KimProgramming Education Temporal Data Leakage
Korea University, Seoul, Republic of Korea
Average Attention Transformers and Arithmetic Circuits
May 6, 2026Lena Ehrmuth, Laura StriekerTransformer Expressivity Transformer
Institute for Theoretical Computer Science, Leibniz University Hanover, Hanover, Germany
Angle-I2P: Angle-Consistent-Aware Hierarchical Attention for Cross-Modality Outlier Rejection
May 6, 2026Muyao Peng, Shun Zou, Pei An +2Point Cloud Registration Cross-Modal Feature Matching
School of Electronic Information and Communications, Huazhong University of Science and Technology
FLUID: Continuous-Time Hyperconnected Sparse Transformer for Sink-Free Learning
May 6, 2026Waleed Razzaq, Yun-Bo ZhaoTransformer Transformer Attention
School of Automation University of Science and Technology China Hefei, Anhui
Where Reliability Lives in Vision-Language Models: A Mechanistic Study of Attention, Hidden States, and Causal Circuits
May 5, 2026Logan Mann, Ajit Saravanan, Ishan Dave +4VLM Evaluation VLM Interpretability
UC Santa Barbara · UC Berkeley · NVIDIA +2
Gradient Flow Structure and Quantitative Dynamics of Multi-Head Self-Attention
May 5, 2026Ayan PendharkarSoftmax Attention Attention Head Analysis
How Language Models Process Negation
May 4, 2026Zhejian Zhou, Tianyi Zhou, Robin Jia +1LLM Interpretability Language Modeling
Information Sciences Institute, University of Southern California · Thomas Lord Department of Computer Science, University of Southern California
When Attention Collapses: Residual Evidence Modeling for Compositional Inference
May 4, 2026Niklas HoubaGravitational-Wave Astronomy Self-Attention
Institute for Particle Physics and Astrophysics ETH Zurich
Projection-Free Transformers via Gaussian Kernel Attention
May 4, 2026Debarshi Kundu, Archisman Ghosh, Swaroop Ghosh +1Transformer Self-Attention
The Pennsylvania State University, University Park, USA
Characterizing the Expressivity of Local Attention in Transformers
May 1, 2026Jiaoda Li, Ryan CotterellTransformer Expressivity Transformer
How Language Models Process Out-of-Distribution Inputs: A Two-Pathway Framework
Apr 30, 2026Hamidreza SaghirLLM Interpretability Attention Mechanisms
Independent Researcher
Attention Is Where You Attack
Apr 30, 2026Aviral Srivastava, Sourav PandaSelf-Attention Adversarial Attacks
Amazon · The Pennsylvania State University
Better Models, Faster Training: Sigmoid Attention for single-cell Foundation Models
Apr 29, 2026Vijay Sadashivaiah, Georgios Dasoulas, Judith Mueller +1Softmax Attention Self-Attention
Merck & Co., Inc., Cambridge, MA, USA
From Architecture to Output: Structural Origins of Hallucination in Large Language Models and the Amplifying Role of Data
Apr 29, 2026Md. Rejaul Korim Sadi, Toufiqur Rahman Tasin, Golam Mostofa NaeemHallucination in Language Models LLM Reliability
*Dept. of Computer Science and Engineering, Metropolitan University, Sylhet, Bangladesh
GateMOT: Q-Gated Attention for Dense Object Tracking
Apr 29, 2026Mingjin Lv, Zelin Liu, Feifei Shao +4Visual Object Tracking Gated Attention
Huazhong University of Science and Technology, Wuhan, China · Zhejiang University, Hangzhou, China · La Trobe University, Melbourne, Australia
Comparative Analysis of AutoML and BiLSTM Models for Cyberbullying Detection on Indonesian Instagram Comments
Apr 29, 2026Raihana Adelia Putri, Aisyah Musfirah, Anggi Puspita Ningrum +3Long Short-Term Memory Networks Attention Mechanisms
Institut Teknologi Sumatera
AMMA: A Multi-Chiplet Memory-Centric Architecture for Low-Latency 1M Context Attention Serving
Apr 28, 2026Zhongkai Yu, Haotian Ye, Chenyang Zhou +9Compute-in-Memory LLM Inference Acceleration
University of California, San Diego · Columbia University · Yonsei University +2
Block-Wise Differentiable Sinkhorn Attention: Tail-Refinement Gradients with a Gap-Aware Dustbin Bridge
Apr 28, 2026Dylan FordeAutomatic Differentiation Transformer Attention
Independent Researcher
QAROO: AI-Driven Online Task Offloading for Energy-Efficient and Sustainable MEC Networks
Apr 28, 2026Yongtao Yao, Yao Yang, Haorui Shi +3Edge Computing Wireless Resource Allocation
Emergent Self-Attention from Astrocyte-Gated Associative Memory Dynamics
Apr 28, 2026Arnau Vivet, Alex ArenasSoftmax Attention Dynamical Systems
1Departament d’Enginyeria Inform`atica i Matem`atiques, Universitat Rovira i Virgili, 43007 Tarragona, Spain · 2Complexity Science Hub Vienna, Metternichgasse 8, 1030 Vienna, Austria
QFlash: Bridging Quantization and Memory Efficiency in Vision Transformer Attention
Apr 28, 2026Sehyeon Oh, Yongin Kwon, Jemin LeeSoftmax Attention GPU Acceleration
University of Science and Technology, Daejeon, Republic of Korea · Electronics and Telecommunications Research Institute, Daejeon, Republic of Korea · Pusan National University, Busan, Republic of Korea +1
Image Classification via Random Dilated Convolution with Multi-Branch Feature Extraction and Context Excitation
Apr 28, 2026Wentao Jiang, Yuanchan Xu, Heng YuanMulti-Scale Feature Fusion Convolutional Neural Networks
Sichuan University
Transformer Approximations from ReLUs
Apr 27, 2026Jerry Yao-Chieh Hu, Mingcheng Lu, Yi-Chen Lee +1Softmax Attention Neural Network Approximation Theory
Center for Foundation Models and Generative AI, Northwestern University, Evanston, IL 60208, USA · Department of Computer Science, Northwestern University, Evanston, IL 60208, USA · Department of Physics, National Taiwan University, Taipei 10617, Taiwan +1
Learning to Rotate: Temporal and Semantic Rotary Encoding for Sequential Modeling
Apr 27, 2026Hailing Cheng, Daqi Sun, Xinyu LuRotary Positional Embeddings Self-Attention
LinkedIn Inc. Mountain View, CA 94043
Learning to Route Queries to Heads for Attention-based Re-ranking with Large Language Models
Apr 27, 2026Yuxing Tian, Fengran Mo, Zhiqi Huang +2Attention Head Analysis Learning to Rank
Université de Montréal · Capital One · McGill University & MILA
Beyond the Attention Stability Boundary: Agentic Self-Synthesizing Reasoning Protocols
Apr 27, 2026Dahlia Shehata, Ming LiInstruction Following Attention Mechanisms
University of Waterloo · Canada

May 7, 2026, cs.CVJ/K move · Enter open · S save

Look Beyond Saliency: Low-Attention Guided Dual Encoding for Video Semantic Search

Faisal Aljehrai, Mohammed A. Alkhrashi, Alreem Almuhrij, Sarah Abuhimed+5

Saudi Data And Artificial Intelligence Authority (SDAIA)

Visual Representation Learning Information Retrieval Attention Mechanisms Video Retrieval

Open paper AskPDF arXiv

Video semantic search in densely crowded scenes remains a challenging task due to visual encoders tendency to prioritize salient foreground regions while neglecting contextually important, background areas. We propose an Inverse Attention Embedding mechanism that explicitly captures and highlights these overlooked regions. By combining inverse attention embeddings with traditional visual embeddings, our method significantly enhances semantic retrieval performance without additional training. Initial experiments and ablation studies demonstrate promising improvements over existing approaches in recall for video semantic search in crowded environments.

Momentum

Related topics

Latest papers 378