cs.CVNov 13, 2026

Role-Separated Diffusion Transformers: Global Rule Isolation and Deterministic Sampling Manifold Compression

Authors: Zhiguang Liu, Yi Shang

Organizations: Department of Electrical Engineering and Computer Science, University of Missouri

Abstract

We present Role-Separated Diffusion Transformers (RoSeDiT), a new diffusion transformer architecture that isolates the global rule from the image tokens. RoSeViT showed in visual reasoning that a vanilla ViT spreads a shared rule over all LL tokens at an entropy of order LdLd, whereas an enforced controller route reduces it to order dd. In class-conditional DiT and SiT, the class and the noise level are explicitly conditioned by adaLN, so a RoSeViT-style controller channel on the ViT side is insufficient to establish the full role separation. By introducing the adaLN listening mechanism, which makes adaLN compute under the controller's command, RoSeDiT brings the entropy theorem and role separation into the latent diffusion domain. Under the same SiT training and evaluation protocol at the same level of parameter count, RoSeDiT surpasses SiT at all four model scales at 400K steps, and RoSeDiT-L/2 even surpasses SiT-XL/2's 17.2 FID with 15.34 and 32% fewer parameters. RoSeDiT-XL/2 surpasses SiT-XL/2's 7M-step FID of 8.3 at 1.4M steps with 7.98, using one-fifth as many training steps. We further observe that models with the same denoising loss can have a large FID gap, which indicates a misalignment between the denoising objective and the image manifold. We formalize this phenomenon into Deterministic Sampling Manifold Compression (DSMC), a two-stage training schedule with which RoSeDiT-XL/2 surpasses SiT + REPA's 4M-step FID of 5.9 at 2.4M steps with 5.84.

Figures & tables

Explore similar work

CardsList
  1. Improved Distributional Diffusion Models

    Sep 29, 2026Tommaso Martorella, Alexandre Galashov, Felix Krause +4Few-Step Diffusion SamplingDiffusion-Based Image Generation

  2. TamedFlow: Trajectory-Aware Distillation for Teacher-Assisted Diffusion Sampling

    Oct 19, 2026Shuaiqi Cheng, Jiahao Zhang, Boyuan Gu +5Diffusion SamplingDiffusion Model Inference Acceleration

  3. Taming Outlier Tokens in Diffusion Transformers

    May 6, 2026Xiaoyu Wu, Yifei Wang, Tsu-Jui Fu +3Diffusion TransformerActivation Outliers