PhysicsSliders: Learning Physical Controls for Audio Foley Generation
Authors: Peiwei Chang, Yongqiang Chen, Yuelan Cheng, Liting Gao, Yishun Lu, Li Zhang, Mark D. Plumbley, Philip J. B. Jackson, Wenwu Wang
Organizations: Centre for Vision, Speech and Signal Processing, University of Surrey, UK · Department of Engineering Science, University of Oxford, UK · Department of Computer Science, Royal Holloway, University of London, UK · Department of Informatics, King’s College London, UK
Physical properties provide intuitive controls for sounds made by interacting objects. The challenge is to make a generative model respond predictably to these controls, both individually and in combination. We present \method\ (PXS), trained on simulated contact sounds with 15 material, geometry and contact parameters. Two separately trained variants accept direct parameter inputs or numerical captions. PXS controls hardness, size, speed and damping, supports room descriptions and produces spectrally distinct outputs at fixed targets. When controlling damping and size together, specifying thickness improves frequency accuracy and joint target delivery. Measurements from real recordings define effective-damping targets, which simulator inversion maps to generation inputs without real-audio fine-tuning. Across 18 impact classes from five real-audio datasets, PXS-Physical follows these targets with mean grouped Spearman correlation ρ=0.976 and class-median error 0.094 dex (a factor of 1.24).
Figures & tables
Figure 1: PXS variants train separately with a shared architecture and frozen decoder. Paired labels show PXS-Physical first. Objects and latents are schematic. Upper-right brass impacts vary one input at fixed sampling noise. Spectra (attack 30 ms, resonance 500 ms) and RMS envelopes are peak-normalised. Low/mid/high use E={0.05,2.24,100} GPa, L={0.126,0.200,0.316} m and η={1.71,4.30,10.8}×10−3 . Lower curves show five-level impact sweeps with paired noise. PXS curves geometrically average valid-reading medians (32 samples) across three held-out materials and three runs. Simulator curves average over materials. q is measured effective damping.
Impulsive ( n=18 )
Sustained ( n=12 )
Generator
ρ↑
g→1
Error ↓
ρ↑
g→1
Error ↓
PXS-Physical
.976
.77
.094
.829
.19
.293
PXS-Text
.954
.78
.058
.844
.27
.360
Table 1: Effective-damping control. (a) Tracking ( n : classes; error: dex). (b) Impact requests with two or more of four outputs on target (%).
Table 4: Metrics: VGGish Fréchet audio distance (FAD) [ 23 ] ; Kullback–Leibler (KL) divergence of pretrained audio neural networks (PANNs) [ 24 ] softmax outputs (reference to generated, nats); contrastive language–audio pretraining (CLAP) [ 25 ] audio–text cosine for material-and-strike descriptions. References are 16 independent renders per θ∗ or real target recordings. SAO captions specify material, plate and strike.
Figure 2: Damping–size grid (log axes). Open points geometrically average PXS-Physical four-output medians over three runs. Grey points are eight-output simulator medians. Arrows increase damping (red) or size (blue). Bold lines fix the other control at its midpoint; faint lines show full grids.