eess.ASFeb 18, 2026

SIRUP: A diffusion-based virtual upmixer of steering vectors for highly-directive spatialization with first-order ambisonics

Authors: Emilio Picard, Diego Di Carlo, Aditya Arie Nugraha, Mathieu Fontaine, Kazuyoshi Yoshii

Organizations: RIKEN AIP, UP1 EMS · Center for Advanced Intelligence Project, RIKEN, Japan · Sorbonne University, France · RIKEN AIP, IP Paris · RIKEN AIP · LTCI, IP Paris · LTCI, Télécom Paris, Institut Polytechnique de Paris, France · Graduate School of Engineering, Kyoto University, Japan

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

This paper presents virtual upmixing of steering vectors captured by a fewer-channel spherical microphone array. This challenge has conventionally been addressed by recovering the directions and signals of sound sources from first-order ambisonics (FOA) data, and then rendering the higher-order ambisonics (HOA) data using a physics-based acoustic simulator. This approach, however, struggles to handle the mutual dependency between the spatial directivity of source estimation and the spatial resolution of FOA ambisonics data. Our method, named SIRUP, employs a latent diffusion model architecture. Specifically, a variational autoencoder (VAE) is used to learn a compact encoding of the HOA data in a latent space and a diffusion model is then trained to generate the HOA embeddings, conditioned by the FOA data. Experimental results showed that SIRUP achieved a significant improvement compared to FOA systems for steering vector upmixing, source localization, and speech denoising.

Figures & tables

Figure 1: The SIRUP upmixer for downstream tasks.
 DI [dB] ↑\uparrow3-dB BW [ \SIUnitSymbolDegree\mathrm{\SIUnitSymbolDegree} ] ↓\downarrowSL [dB] ↓\downarrow
 FOAUpmixedHOAFOAUpmixedHOAFOAUpmixedHOA
DRT60\mathcal{D}_{\text{RT60}}10±±2.610\pm\pm 2.619.8±±2.319.8\pm\pm 2.320±±2.220\pm\pm 2.230±±630\pm\pm 624±±3.324\pm\pm 3.324±±224\pm\pm 2−0.9±±0.7-0.9\pm\pm 0.7−9.5±±3.1-9.5\pm\pm 3.1−11.2±±2.8-11.2\pm\pm 2.8
DSNR\mathcal{D}_{\text{SNR}}8.1±±2.78.1\pm\pm 2.717.1±±2.117.1\pm\pm 2.117.7±±2.017.7\pm\pm 2.048±±6.748\pm\pm 6.727±±3.527\pm\pm 3.526±±2.226\pm\pm 2.2−1.2±±0.9-1.2\pm\pm 0.9−9.6±±3.4-9.6\pm\pm 3.4−11.7±±2.7-11.7\pm\pm 2.7
Table 1: Performances for different spatial representations, averaged over 30 simulated rooms. Ground-truth HOA values (gray) are provided for reference. Beamwidth is measured at -3dB (lower is better).
 Measured SVsAlgebraic SVs following SSL
MetricSV-FOASV-SIRUP-MSV-alg FOASV-alg SIRUP
SDR [dB]17.2 ± 3.217.4 ± 3.112.6 ± 7.413.0 ± 7.2
SIR [dB]38.8 ± 3.638.8 ± 3.333.5 ± 7.834.0 ± 7.5
SAR [dB]17.3 ± 3.217.4 ± 3.112.6 ± 7.313.0 ± 7.2
Table 2: Source synthesis enhancement from mixture across varying for mixtures of two speech sources.
Figure 2: Average angular errors across conditions using different localization methods and SV models, using SRP (Eq. ( 5 )).
Figure 3: 2D Heatmap comparaison of estimated SV for FOA and ground truth HOA setup.

Explore similar work

CardsList

  1. DiffAU: Diffusion-Based Ambisonics Upscaling

    Sep 30, 2025Amit Milstein, Nir Shlezinger, Boaz RafaelyAudio Diffusion ModelsSpatial Audio

    ECE School Ben-Gurion University of the Negev, Be’er-Sheva, Israel

  2. Ambisonics Encoding of Room Impulse Responses using a Device-Agnostic Diffusion Model

    Aug 14, 2026Eloi Moliner, Christoph Hold, Juan Azcarreta Ortiz +5Spatial AudioRoom Acoustics

    Reality Labs Research, Meta · Acoustics Lab, DICE, Aalto University, Espoo, Finland

  3. Flow-HOA: Generative Joint Optimization for Ambisonics Encoding via Flow Matching

    Jun 3, 2026Yuhuan You, Yufan Qian, Tianshu Qu +2Spatial Audio

    State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University, Beijing, China · 2Beijing Xiaomi Mobile Software Co., Ltd · 3Xiaomi Communications Co., Ltd