Multi-Channel Replay Speech Detection using Acoustic Maps

Michael Neri; Tuomas Virtanen

eess.ASFeb 18, 2026

Multi-Channel Replay Speech Detection using Acoustic Maps

Authors: Michael Neri, Tuomas Virtanen

Organizations: Faculty of Information Technology and Commmunication Sciences, Tampere University, Tampere, Finland

Abstract

Replay attacks remain a critical vulnerability for automatic speaker verification systems, particularly in real-time voice assistant applications. In this work, we propose acoustic maps as a novel spatial feature representation for replay speech detection from multi-channel recordings. Derived from classical beamforming over discrete azimuth and elevation grids, acoustic maps encode directional energy distributions that reflect physical differences between human speech radiation and loudspeaker-based replay. A lightweight convolutional neural network is designed to operate on this representation, achieving competitive performance on the ReMASC dataset with approximately 6k trainable parameters. Experimental results show that acoustic maps provide a compact and physically interpretable feature space for replay attack detection across different devices and acoustic environments.

Figures & tables

Fig. 1: Spatial distribution of acoustic maps from delay-and-sum beamformer across azimuth and elevation angles for two utterances from the ReMASC dataset, recorded using device $\mathrm{D}3$ with $6$ microphones arranged in a hexagonal shape. The top row corresponds to the genuine sample 1264222.wav , and the bottom row to the replay sample 1380536001.wav from the same indoor environment and positions. Each column represents a distinct frequency band: Low ( $100–500$ Hz), Mid ( $500–3000$ Hz), High ( $3000–8000$ Hz), and Super-High ( $8000–22050$ Hz). The color scale indicates normalized acoustic intensity.

Fig. 2: Microphone-wise performance in both generalization scenarios.

Domain-Incremental Learning for Multi-Channel Replay Speech Detection
Sep 10, 2026Michael NeriSpeech Processing Domain-Incremental Learning
Faculty of Information Technology and Communication Sciences, Tampere University, Tampere, Finland
Spectro-Temporal Modulation Representation Framework for Human-Imitated Speech Detection
Apr 25, 2026Khalid Zaman, Masashi UnokiAudio Deepfake Detection Speech Processing
Graduate School of Advanced Science and Technology, Japan Advanced Institute of Science and Technology, Nomi, Ishikawa 923-1292, Japan
Bypassing Direct Reconstruction: Speech Detection from MEG via Large-Scale Audio Retrieval
May 13, 2026Boda Xiao, Bo Wang, Heping ChengNeural Decoding Neuroimaging
Center for BioMed-X Research, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China · Speech and Hearing Research Center, School of Intelligence Science and Technology, Peking University, Beijing, China · National Biomedical Imaging Center, State Key Laboratory of Membrane Biology, Institute of Molecular Medicine, Peking-Tsinghua Center for Life Sciences, College of Future Technology, Peking University, Beijing, China

Sep 10, 2026J/K move · Enter open · S save

Domain-Incremental Learning for Multi-Channel Replay Speech Detection

Michael Neri

Faculty of Information Technology and Communication Sciences, Tampere University, Tampere, Finland

Speech Processing Domain-Incremental Learning

Open paper AskPDF arXiv

Replay attacks are the most accessible threat to voice-controlled systems, and the acoustic cues that expose them are strongly modulated by the environment in which the attack is mounted. A detector deployed in the field therefore has to absorb new acoustic conditions over time, ideally without revisiting past recordings, since retaining speech indefinitely is both expensive and legally constrained. We frame this as Domain-Incremental Learning (DIL) over acoustic environments and present the first continual learning benchmark for multi-channel replay speech detection, evaluating a state-of-the-art beamformer-based detector over all 24 environment orderings of the ReMASC corpus with five seeds. Sequential fine-tuning forgets severely, raising the error rate on previously learned environments by 18.8 points. Elastic weight consolidation (EWC) halves forgetting but loses plasticity, gradient projection memory (GPM) is statistically indistinguishable from naive fine-tuning, and the proposed task-specific beamformer (TSB) that keeps one spatial front-end per environment significantly improves final and incremental accuracy. We further show that the last environment of the sequence dominates final performance. Code, results, and analysis are available at https://github.com/michaelneri/replay-speech-continual.

Abstract

Explore similar work

Domain-Incremental Learning for Multi-Channel Replay Speech Detection

Spectro-Temporal Modulation Representation Framework for Human-Imitated Speech Detection

Bypassing Direct Reconstruction: Speech Detection from MEG via Large-Scale Audio Retrieval