Supervised Source Separation of Electric Vehicle Interior Noise Using a Hybrid Transformer Architecture
Authors: Philipp Diel, Zhuo Cao, Arne Hildenbrand, Marco Günther, Stefan Pischinger, Hanno Scharr, Steffen Marburg
Organizations: Chair of Thermodynamics of Mobile Energy Conversion Systems, RWTH Aachen University, Aachen, Germany · IAS-8, Forschungszentrum Jülich, Jülich, Germany · Chair of Vibroacoustics of Vehicles and Machines, Technical University of Munich, Garching, Germany
The interior sound of an electric vehicle is a superposition of several distinct noise contributions - wind noise, tire noise, powertrain (electric drive unit and inverter) noise, and auxiliary-drive noise - whose relative balance governs both the overall loudness and the perceived sound quality in the vehicle interior. Decomposing a recorded vehicle interior signal into these source contributions is a prerequisite for targeted acoustic engineering, but requires effortful measurement campaigns as well as expertise in the field of transfer path analysis. This work formulates the decomposition as a supervised, binaural source-separation problem with four target source contributions and adapts a hybrid time-spectrogram transformer architecture, originally developed for music demixing, to the domain of sound source separation for technical systems. To disambiguate source contributions, conditioning is introduced in the form of operating-state information, including electric drive speed, electric drive load, vehicle speed, and auxiliary-drive speed. Since isolated real-world reference stems are costly to obtain, the model is trained on a physics-based parametric data generator that synthesizes labeled component sounds from randomized virtual-vehicle hardware configurations and driving scenarios, so that the ground-truth contribution of each source is known by construction. Validation on both synthetic and measured vehicle interior sounds probes the achieved separation performance and underscores the feasibility of applying the proposed method for acoustic contribution analysis in technical systems.
Figures & tables
Abbrev.
Conditioning
Loss Norm.
LSTFT
Description
UC-BASE
–
–
–
Baseline: unconditioned, unnormalized waveform loss, no spectrogram loss
UC-N
–
✓
–
Unconditioned with normalized waveform loss
UC-STFT
–
–
✓
Unconditioned, unnormalized waveform loss, with auxiliary spectrogram loss
UC-N-STFT
–
✓
✓
Unconditioned, normalized waveform loss, with auxiliary spectrogram loss
FiLM
FiLM
–
–
FiLM-conditioned, unnormalized waveform loss
FiLM-N
FiLM
✓
–
FiLM-conditioned with normalized waveform loss
Table 1: Overview of model variants differing in conditioning, waveform-loss normalization, and use of an auxiliary spectrogram loss.
Figure 1: SDR separation results for single virtual vehicle.
Figure 2: Spectrograms for separation of source contributions of a single virtual vehicle. Experiment UC-STFT.
Figure 3: Spectrograms for conditioned separation of source contributions of a single virtual vehicle. Experiment FiLM-N-STFT.
Figure 4: Spectrograms for separation of source contributions for one of multiple virtual vehicles. Experiment UC-STFT.
Figure 5: Source contribution spectrograms for real vehicle interior noise measurements recorded in [ 43 ] . As the mixtures are normalized before inference, the magnitude scale is relative and does not reflect measured decibel values.
Property
Scope
Value Range
Distribution
Tire exterior diameter [m]
per vehicle
[0.60,0.75]
Uniform
Gear ratio
per vehicle
[7.5,11.5]
Uniform
Pole pairs p
per vehicle
{3,4,5}
Discrete uniform
Stator slots Nslots
per vehicle
{36,48,72}
Discrete uniform
Gear teeth count Nteeth
per vehicle
[80,100]
Uniform, integer
Inverter switching freq. fsw [Hz]
per vehicle
[7000,14000]
Uniform
Table 2: Randomly sampled properties, their value ranges and underlying distributions.
Operation Parameter
Affected Source(s)
Mechanism / Effect
Vehicle speed v(t) [km/h]
Powertrain, Inverter
Sets drive unit rotational speed (see below)
Tire, Wind (broadband)
Broadband noise (sum of tire and wind) amplitude scales linearly with vehicle speed [ 4 ]
Wind only
Low-speed roll-off via sigmoid 1/(1+e−0.2(v−20)) gradually suppresses wind noise below ∼ 50 km/h
Cavity tone (tire sub-component)
Resonance frequency shifts with speed: fcav=fbase+0.1(v−60 km/h ) ; amplitude scales relative to tire RMS
Load factor L(t)∈[0,1] (derived)
Powertrain
Time-varying, constructed with speed-dependence and acceleration-dependence. Modulates amplitude of all powertrain orders.
Drive unit rotational speed fmech (derived from v via gear ratio and wheel diameter)
Powertrain, Inverter
Determines exact excitation frequencies: mechanical orders fm = nm⋅fmech ; electrical orders fe = ne⋅fmech⋅p ( p = pole pairs); slot order fslot = fmech⋅Nslots ; gear mesh order fgear = fmech⋅Nteeth ; inverter sidebands fsb = fsw±j⋅fmech⋅p (sidebands j=1,…,Nsb ); all order amplitudes increase with higher rotational speeds
Table 3: Influence of vehicle operation parameters on the synthesized sound sources.
Audio source separation models must preserve fine spectral structure while keeping the cost of the separator tractable. Band-split architectures reduce this cost by manually partitioning the spectrum into predefined frequency bands and encoding each band as a token, but the same boundaries are used for every mixture. Because informative spectral structure varies with source content, fixed boundaries can merge discriminative components or spend tokens where little resolution is needed. We introduce Dynamic Band Splitting (DBS), an end-to-end framework that adapts boundary locations and band widths to each mixture under a fixed band-token budget. DBS predicts content-conditioned boundary scores within frequency blocks, selects a prescribed number of boundaries using hard Top-K, and applies content-dependent pooling to obtain compact band tokens. After separation, it smooths and expands these tokens to the original STFT grid, refines them along frequency, and decodes complex masks with a frequency-shared head. DBS improved separation over the corresponding static models on DnR v2, MUSDB18-HQ, and EchoSet, with DBS-BSRNN achieving relative gain of 29.0% in average SI-SDR on DnR v2. These results and boundary analyses on DnR v2 show that content-adaptive frequency partitioning benefits multiple separator backbones under a fixed band-token budget. Code and audio demos are available at https://cslikai.cn/Dynamic-Band-Splitting
Bingzhe Zhang, Kai Li, Xiaolin Hu
Department of Computer Science and Technology, Institute for AI, BNRist, Tsinghua University, Beijing 100084, China · IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing 100084, China · Chinese Institute for Brain Research (CIBR), Beijing 100010, China
Isolating a target singing voice from a music video remains challenging, particularly in the presence of multiple vocalists and dense instrumental accompaniment. We propose MambaVoice, a lightweight audiovisual framework that leverages a hybrid Mamba--Transformer architecture for targeted singing voice separation. The model jointly encodes audio and visual streams using an attention-based band-split audio encoder and a spatio-temporal graph convolutional network (ST-GCN) for facial motion features. These modalities are fused through a multiplicative gating mechanism, enabling visual cues to selectively modulate audio representations. The fused features are processed by a hybrid backbone that combines Transformer self-attention with Selective State Space Models (SSMs), achieving efficient long-range temporal modeling with linear complexity. We evaluated MambaVoice on the Acappella and URSing datasets under challenging conditions, including mixtures with interfering singers. At 16.2 million parameters, the model demonstrates comparable performance, achieving 14.18 dB SDR on Acappella and strong cross-dataset performance on URSing, comparable to larger models at a fraction of the parameter count. These findings highlight the effectiveness of hybrid SSM--attention architectures for scalable, efficient audiovisual source separation, suggesting they are well-suited as lightweight components within larger pipelines. We conduct a perceptual study that further supports our improvements in objective metrics. We provide our implementation online.
The rapid advancement of artificial intelligence has observed increased application in predicting vehicle interior noise levels within the automotive industry. However, the collection of labeled data for training models in this context involves significant costs. Previous studies in semi-supervised regression (SSR) have effectively mitigated the reliance on labeled data by incorporating unlabeled data. Nonetheless, these approaches often introduce a high computational cost due to the training of multiple models and data sampling. This study introduces SpecRegMatch, a novel SSR method aimed at addressing the computational cost associated with training by leveraging a single model, thus eliminating the need for multiple data samplings. SpecRegMatch integrates consistency regularization and information maximization to robustly train the model, achieved through various augmentations applied to both the embedding vectors and predicted values. Experimental results demonstrate that SpecRegMatch achieves state-of-the-art performance across various scenarios, even when using a single model. It attains a remarkable performance, as indicated by an R^2 score of 0.434. This is especially noteworthy in scenarios where labeled data is scarce. You can access the code for our proposed method at https://github.com/sejin-sim/SpecRegMatch.
Sejin Sim, Jinsoo Bae, Seoung Bum Kim
School of Industrial and Management Engineering, Korea University, Seoul 02841, South Korea