cs.CVJul 6, 2026

Steering Optimisation Trajectories in Diffusion Representation Learning

Authors: Rajat Rasal, Avinash Kori, Tian Xia, Ben Glocker

Organizations: Imperial College London

Abstract

We study why diffusion autoencoders can achieve similar image quality while learning substantially different latent structures. We trace this behaviour to optimisation dynamics; we analyse curves of image reconstruction against latent representation quality, revealing trajectories that organise around two distinct regimes early in training. Models in the reconstruction regime prioritise image fidelity early, whereas those in the disentanglement regime improve reconstruction and disentanglement more gradually. We hypothesise that this behaviour can be influenced by targeting shortcut pathways in the diffusion U-Net and controlling early noise-level exposure, thereby shaping the reconstruction-disentanglement trade-off during training. To steer optimisation toward stronger representations, we introduce SteeringDRL, combining gated residual U-Nets with a simple noise-level exposure curriculum for training. Across disentanglement benchmarks, SteeringDRL improves representation quality and reduces seed sensitivity. Our method further extends to spatial disentanglement in object-centric learning, improving segmentation quality on synthetic and real-world datasets.

Explore similar work

CardsList
  1. On the Diffusibility of High-Dimensional Latents

    Sep 23, 2026Chao Feng, Zhiyang Xu, Bowei Chen +7

  2. Improved Baselines with Representation Autoencoders

    May 18, 2026Jaskirat Singh, Boyang Zheng, Zongze Wu +3Masked AutoencodersImagenet