cs.SDNov 8, 2026

Music Inpainting With Similarity-Guided Latent Diffusion

Authors: Sean Turland, Eloi Moliner, Vesa Välimäki

Organizations: Acoustic Lab, Dept. Information and Communications Engineering, Aalto University, Finland · Acoustics and Audio Group, University of Edinburgh, United Kingdom

Abstract

We propose a music inpainting technique combining similarity matching and a diffusion-based generative model to replace large segments of audio recordings with suitable alternatives. The proposed hybrid method, Similarity-Guided Diffusion Posterior Sampling (SimDPS), retrieves candidate replacement segments from the wider signal based on contextual similarity and uses them to guide the diffusion process toward coherent reconstructions. Being model-agnostic, the framework is compatible with any pre-trained diffusion model operating in either the waveform or latent domain. Objective evaluations demonstrate that SimDPS outperforms widely adopted RePaint and DPS methods across acoustic features and neural-embedding distance metrics. Subjective evaluations on multi-genre music datasets confirm that SimDPS enhances perceptual plausibility compared to previous diffusion-based methods and demonstrates robustness to poorly matched similarity candidates. These findings pave the way for improved music restoration and context-aware audio editing.

Figures & tables

Explore similar work

CardsList
  1. Music Restoration via Latent Operator Optimization and Diffusion Model Priors

    Aug 3, 2026Michal Švento, Eloi Moliner, Valtteri Kallinen +3

  2. Stable Audio 3

    May 18, 2026Zach Evans, Julian D. Parker, Matthew Rice +4Text-to-Audio GenerationLatent Diffusion Models

  3. Pitch-class Steering for Diffusion-based Music Generation via Latent-space Probes

    Sep 3, 2026Yushi Ye, Wilson Zheng, Yongyi ZangControllable Music GenerationDiffusion Model Guidance