AFA-Net: A Differential Attention Approach for Auditory Attention Detection

Philip H. Lee; Shreeram Suresh Chandra; Karan Thakkar; John H. L. Hansen

cs.SDSep 25, 2026

AFA-Net: A Differential Attention Approach for Auditory Attention Detection

Authors: Philip H. Lee, Shreeram Suresh Chandra, Karan Thakkar, John H. L. Hansen

Organizations: Independent Researcher, USA · Center for Language and Speech Processing (CLSP), Johns Hopkins University, USA · Laboratory for Computational Audio Perception (LCAP), Johns Hopkins University, USA · Center for Robust Speech Systems (CRSS), University of Texas at Dallas, USA

Abstract

Auditory Attention Detection (AAD) utilizes electroencephalographic (EEG) signals to identify a target speaker in a multi-speaker environment. Despite considerable progress, existing deep learning architectures often lack explicit mechanisms for handling noisy EEG data. To address this limitation, we propose Auditory Focus Attention Networks (AFA-Net), a machine learning framework that replaces vanilla attention with a simple yet flexible differential attention mechanism to help focus on task-relevant neural activity. AFA-Net achieves an upward accuracy of 96.8% at the 2s decision window, while using substantially fewer parameters than most existing methods. To the best of our knowledge, AFA-Net is among the first frameworks to explicitly try to combat EEG noise to improve AAD.

Figures & tables

Figure 1: Proposed architecture of AFA-Net.

Figure 2: Ablation study comparing frequency band removal for differential vs. vanilla attention at the 1s decision window.

		KUL			DTU
Model	Params (M)	0.1s	1s	2s	0.1s	1s	2s
SSF-CNN [ 4 ]	4.21	$76.3\pm 8.47$	$84.4\pm 8.67$	$87.8\pm 7.87$	$62.5\pm 3.40$	$69.8\pm 5.12$	$73.3\pm 6.21$
STANet [ 22 ]	0.04	$80.8\pm 6.43$	$89.8\pm 7.11$	$91.7\pm 5.89$	$64.8\pm 5.11$	$72.1\pm 6.98$	$73.5\pm 6.44$
DenseNet-3D [ 26 ]	0.76	$-$	$93.7\pm 4.32$	$95.3\pm 4.44$	$-$	$-$	$-$
DBPNet [ 18 ]	0.91	$85.3\pm 6.22$	$94.4\pm 4.62$	$95.3\pm 4.63$	$74.0\pm 5.20$	$79.8\pm 6.91$	$80.2\pm 6.79$
DARNet [ 20 ]	0.08	$89.2\pm 5.50$	$94.8\pm 4.53$	$95.5\pm 4.89$	$74.6\pm 6.09$	$80.1\pm 6.85$	$81.2\pm 6.34$

Table 1: Comparison of auditory attention detection accuracy (%) across different models on the KUL and DTU datasets using various decision windows. The * symbol denotes significant differences over the strongest baseline, MHANet (two-tailed paired t-test,

p<0.05

Differential Attention

See all 10 papers

FAConformer: Frequency-Aware Convolutional Transformer for Auditory Attention Decoding
Jun 12, 2026Ziwei Wang, Xingyi He, Tianwang Jia +2Hybrid CNN-Transformer Architectures Neural Decoding
Hubei Key Laboratory of Brain-inspired Intelligent Systems, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China
CA-TCN: A Causal-Anticausal Temporal Convolutional Network for Direct Auditory Attention Decoding
Mar 27, 2026Iñigo García-Ugarte, Rubén Eguinoa, Ricardo San Martín +2Neural Decoding Temporal Convolutional Networks
Department of Science, Universidad Pública de Navarra (UPNA), 31006 Pamplona, Spain · BCBL, Basque Center on Cognition Brain and Language, 20009 San Sebastián, Spain · Department of Statistics, Computer Sciences and Mathematics, Universidad Pública de Navarra (UPNA), 31006 Pamplona, Spain +2
End-to-End Markov State Sequence Learning for Auditory Attention Decoding
Jul 21, 2026Yushan Yashengjiang, Jie Zhang, Miao Sun +3Markov Models Neural Decoding
NERC-SLIP, University of Science and Technology of China (USTC), Hefei 230027, China · School of Information and Communication Engineering, Guangzhou Maritime University, Guangzhou 510725, China · Artificial Intelligence Research Institute, iFLYTEK Company, Ltd., Hefei 230088, China +1

Jun 12, 2026J/K move · Enter open · S save

FAConformer: Frequency-Aware Convolutional Transformer for Auditory Attention Decoding

Ziwei Wang, Xingyi He, Tianwang Jia, Hongbin Wang+1

Hubei Key Laboratory of Brain-inspired Intelligent Systems, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China

Hybrid CNN-Transformer Architectures Neural Decoding EEG Decoding Frequency-Domain Feature Learning Transformer Attention

Open paper AskPDF arXiv

Auditory attention decoding (AAD) aims to infer the attended speaker from neural responses in multi-speaker acoustic environments and is a key problem for neuro-steered hearing systems. Although recent studies have achieved encouraging progress, existing AAD models still do not fully exploit frequency domain electroencephalography (EEG) information. In particular, most approaches introduce multi-band information through handcrafted feature extraction or direct cross-band feature concatenation, which mainly exploit frequency information at a shallow level and may overlook band-specific patterns and cross-band interactions. To address these limitations, this paper proposes FAConformer, a frequency-aware CNN-Transformer framework for AAD that explicitly integrates band-specific encoding and adaptive cross-band interaction. Specifically, FAConformer first decomposes EEG signals into multiple frequency bands and assigns each band to an independent CNN-Transformer encoder for band-specific modeling. The resulting band-wise features are then adaptively fused by a carefully designed frequency-aware attention (FAA) module that models cross-band dependencies by treating band-wise features as tokens. Further, band-wise auxiliary supervision (BAS) is introduced to prevent weakly contributing branches from being under-optimized during joint training. In this way, FAConformer performs frequency-aware modeling that more effectively exploits frequency domain information. Extensive experiments on two public AAD datasets with three decision-window lengths demonstrated that FAConformer consistently outperformed 12 competitive baselines, surpassing the current state-of-the-art model by 4.9%. Further analyses of band importance, ablation, and parameter sensitivity verify the effectiveness, robustness, and interpretability of the proposed framework. Code is available at https://github.com/wzwvv/FAConformer.

Abstract

Differential Attention

Explore similar work

FAConformer: Frequency-Aware Convolutional Transformer for Auditory Attention Decoding

CA-TCN: A Causal-Anticausal Temporal Convolutional Network for Direct Auditory Attention Decoding

End-to-End Markov State Sequence Learning for Auditory Attention Decoding