PLAUD (Performative Latents and Unsupervised DDSP) is a neural synthesizer and Max for Live instrument for live electronic music, built on NoiseBandNet and trained on small personal sound corpora. We present its architecture, combining a variational DDSP synthesis model, latent smoothing, multi-scale spectral and adversarial losses, and an optional transformer prior, alongside a set of bending operations that intervene directly in the synthesis chain: component limiting, waveshaping, and prior feedback. The Max for Live interface exposes control generation, trajectory sampling, and modulation as primary modes of interaction. Throughout, we thread an affordance analysis arguing that the system's performative character follows from architectural decisions rather than being designed on top of them. The paper contributes both a technical account of the system and a situated affordance analysis of its role in live electronic music performance.
Figures & tables
Figure 1: PLAUD synthesis model architecture. During training, mel-band features feed the encoder (producing latent z ), while raw audio feed a parallel feature extraction path; both are smoothed before concatenation and decoding into per-band amplitudes, which are multiplied with pre-rendered noise bands. Training combines multi-scale spectral and adversarial losses. In inference, only the decoder is used. The prior network is not shown.
Figure 2: Latent space projections from models trained on the same corpus with β=10−3 (left) and β=10−2 (right). Stronger regularisation yields a denser, more uniformly populated manifold, reducing the unpopulated gaps that make continuous navigation unpredictable.
Figure 3: Log-magnitude spectrograms of training audio (top), reconstruction without adversarial training (middle), and reconstruction with adversarial training (bottom). The adversarially trained model recovers finer spectral detail, visible as sharper horizontal structures in the upper frequency range and increased contrast in transient regions, compared to the smoother, more diffuse reconstruction produced by MSS loss alone.
Figure 4: Control trajectories over a 1 second excerpt, without smoothing (top) and with smoothing at kernel size K=257 (bottom). Each coloured line represents one control dimension; the blue waveform shows the corresponding audio. Without smoothing, the trajectories are highly dynamic and tightly coupled to the audio’s micro-temporal structure. After smoothing, the same dimensions trace slower, more gestural contours — delegating short-term continuity to the GRU and making the latent space navigable through simple modulation.
Figure 5: Prior network architecture. Continuous control signals are mu-law quantized into discrete tokens, embedded with positional encoding, and processed by an encoder-only transformer. A dense layer produces logits over the quantization classes, trained with cross-entropy loss; at inference, softmax sampling and mu-law dequantization recover continuous control trajectories.
Figure 6: Prior temporal context TRF (solid) and internal control rate fint (dashed) as functions of resampling rate W , for a fixed prior sequence length L=512 at Fs=44100 Hz. Their product is constant at L .
Figure 7: The PLAUD Max for Live interface. From left: model loading, presets and interpolation controls; loudness, highs, and latent dimension control knobs; autoregressive prior controls, latent trajectory and sampling; posterior trajectory modifiers and synthesis bending parameters; bending visualizer.
Figure 8: The first author performing with PLAUD in a live club setting. The setup includes a laptop running Ableton Live with the PLAUD Max for Live device, alongside external Push controller.
Audio generation creates speech, music, and sound from text, media, symbolic controls, or reference audio, with growing focus on controllability and realism.
Music and Audio Research Group (MARG) · Department of Intelligence and Information, Seoul National University · Interdisciplinary Program in Artificial Intelligence, Seoul National University