cs.SDNov 1, 2026

Acoustic and Attentional Challenges Shape the Balance Between Attended and Competing Speech Tracking Differently

Authors: Johanna Wilroth, Martin A. Skoglund, Emina Alickovic, Martin Enqvist

Organizations: Dept. of Electrical Engineering Linköping University Sweden. · Eriksholm Research Centre, Oticon A/S Denmark.

Abstract

Selective attention enhances cortical tracking of attended speech, but the role of ignored speech remains less clear. In particular, it is unknown whether attended and ignored speech contribute to cortical tracking in a fixed balance or whether this balance changes with listening conditions. We addressed this question across six electroencephalography (EEG) datasets spanning acoustic and attentional challenges, hearing impairment, and hearing-aid processing. We systematically varied the relative weighting of attended and competing speech and studied how their combined contribution affected cortical speech tracking. Across most listening conditions, cortical tracking was improved when we included competing speech, compared with attended speech alone. Importantly, the balance between attended and competing speech changed across conditions. Increasing acoustic difficulty increased both the benefit and the optimal contribution of competing speech, with the same general pattern observed in normal-hearing and hearing-impaired listeners. In the hearing-aid datasets examining noise-reduction conditions, the contribution of competing speech was more limited. Attentional challenges also changed the contribution of competing speech, but differently from acoustic challenges. These findings show that attended and competing speech do not contribute to cortical tracking in a fixed balance. Rather, their balance depends on the listening challenge. Although these results do not define a single optimal ratio, they provide a step toward hearing devices that adapt not only to the attended talker, but also to how competing speech is balanced in complex listening environments.

Figures & tables

Explore similar work

Oct 7, 2026

A multi-scenario EEG dataset for auditory attention decoding in naturalistic multi-talker environments

Understanding how the brain selectively follows relevant speech amid competing voices is a central challenge in auditory neuroscience and a key step toward neuro-steered hearing technologies. However, most open-source Electroencephalography (EEG) datasets for Auditory Attention Decoding (AAD) use idealized single-competing-talker paradigms that oversimplify the acoustic, spatial, and semantic structure of everyday communication. To capture this ecological complexity, we introduce the SoundBubble-EEG dataset: a high-density 128-channel EEG resource comprising more than 25 hours of recordings from 30 participants. The paradigm requires listeners to selectively attend to a dynamic target speaker group, a designated "sound bubble", amid competing multi-speaker distractor bubbles across three realistic scenarios: a restaurant, a home TV viewing, and a meeting discussion. By bridging the gap between constrained laboratory protocols and real-world auditory scenes, this dataset enables investigations of multi-talker speech comprehension, neural speech tracking, and cross-scenario generalization. It also provides a benchmark for AAD algorithms under realistic acoustic and semantic variability and may support auditory neuroscience and the development of neuro-steered hearing technologies.
Sep 30, 2026

NEUROTOKEN: Joint Source and Directional AAD with Envelope Decoding via Conditional Flow Matching

Identifying which speaker a listener is attending to in a noisy room -- the cocktail-party problem -- is the missing ingredient for next-generation hearing aids and brain-computer interfaces: it tells the device whose voice to amplify. Auditory attention decoding (AAD) reads this answer from EEG, but the literature splits into disconnected pieces: directional-AAD classifies side but does not map side to stream; regression-based source-AAD ranks candidate streams by a single Pearson correlation that is intrinsically noisy at the 1-5 s windows real devices need; and envelope reconstruction has no native AAD rule. We argue the right object is not any single statistic but the conditional likelihood of the attended envelope given EEG, and we make this practical with NEUROTOKEN: a single network whose three heads share one EEG front-end, with a conditional flow-matching head (ATTUNEFLOW) that scores candidates by an integrated velocity-residual likelihood ratio. Two inference-time ensembles -- QUADTRACK (four complementary statistics) and ENV-FLOW (z-normalised QUADTRACK+ATTUNEFLOW) -- absorb per-statistic failure modes for free. On KU Leuven, DTU, and NJU at 5 s, ATTUNEFLOW lifts per-segment source-AAD by 9%-16% over the strongest non-generative baseline and shrinks across-subject variance by ~3x; trial-level fusion exceeds 93% on two of three datasets. In parallel reproductions we show that canonical 95-97% direction-AAD numbers collapse by 17%-45% under a strict trial-disjoint protocol, clarifying both the true ceiling and why a likelihood-based formulation is needed.
Sep 25, 2026

MAESTRO: a Multimodal Auditory-attention Egocentric Speech-TRacking Open corpus

Humans rely on gaze, head movements, and visual cues to attend to speakers in noisy environments, yet auditory attention decoding (AAD) has been studied primarily using electroencephalography (EEG). We introduce the Multimodal Auditory-attention Egocentric Speech-TRacking Open (MAESTRO) corpus, the first AAD dataset to simultaneously record EEG, eye gaze, pupillometry, egocentric video, and head inertial measurement unit (IMU) data. MAESTRO includes four competing speakers and background noise across multiple signal-to-noise ratio (SNR) conditions, enabling attention decoding under realistic listening scenarios. Through a four-speaker attention decoding benchmark, we show that combining behavioral and physiological signals improves decoding performance over EEG-only approaches, enabling future advances in multimodal auditory attention decoding. These findings open the door to new applications, analyses, and methodological advances in multimodal AAD. The complete dataset is publicly available at https://huggingface.co/datasets/aspire-osu/maestro-eeg-dataset . The official code repository is available at https://github.com/ASPIRE-OSU/MAESTRO .