Representation-alignment methods effectively accelerate the diffusion transformer training. REPA, for example, uses a pretrained vision encoder to supervise an intermediate layer through an MLP projection head which is discarded after training. It aims to import semantics from the vision encoder to the diffusion backbone. While this objective is achieved at early layers by back-propagating the alignment loss, there is no guarantee that later layers, supervised by the velocity prediction loss alone, can well access such semantics. We propose a simple solution, representation conditioning (RECON), to improve semantics in later layers. During training, we feed the pretrained vision encoder tokens into later layers; during inference, because we no longer have the clean image, we reuse the MLP head (as a proxy to the vision encoder) and project its output tokens to the later layers. Our method adds no parameters and leads to much faster convergence: RECON is 63× faster than SiT and 25× faster than REPA. Code is available at https://github.com/saeidrazavi/RECON
Figures & tables
(a) REPA
Figure 2: Alignment with the vision encoder, measured by CKNNA against DINOv2-B. (a) gFID against CKNNA over training checkpoints. (b) CKNNA of backbone features at each layer. Dashed line is CKNNA between the projector output gϕ(h) and the vision encoder, which is stronger than any layer. (c) CKNNA of every layer for REPA and RECON at t=0.25 .
Figure 3: Representation conditioning improves visual scaling. RECON produces higher-quality images at different steps compared with vanilla REPA. Results for both methods are sampled using the same seed, noise and class label. We use no classifier-free guidance.
Figure 4: Qualitative results on ImageNet 256×256 from SiT-XL/2 + RECON, using classifier-free guidance with w=4 .
Table 5Table 6
Figure 5: Qualitative comparison on text-to-image generation (BLIP3-o). Curated qualitative samples from REPA and RECON trained for 100K iterations. Prompts are shown on top.
Figure 6: Linear probing vs. layers. ImageNet-1K validation top-1, at t=0.25 .
Figure 7: RECON settles the structure of a sample earlier in generation. (a) measures how close the running estimate x^0(t) is, in semantic space, to the image that the same run finally produces. It is measured for a single sample, where RECON is above REPA at every step (both methods start from the same noise). In (b), we generate 10K images. At each diffusion timestep, we measure the percentage of samples whose running estimate x^0(t) similarity with the final image is greater than 0.8. RECON is above REPA.
NFE
50
100
250
REPA
10.8
8.4
7.9
RECON
3.3
3.2
3.2
Table 5: Sampling steps. gFID-50K; SiT-XL/2, 400K iter., no CFG.
Figure 8: RECON vs . REPA across model scale, tokenizer and target encoder. gFID-50K of REPA and RECON. (a) SiT-B/2, SiT-L/2 and SiT-XL/2 at 20 epochs. (b) Three tokenizers at 80 epochs. (c) Five target vision encoders, SiT-B/2 at 20 epochs.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Table 12
REPA
RECON
FID ↓
IS ↑
Pre. ↑
Rec. ↑
FID ↓
IS ↑
Pre. ↑
Rec. ↑
Model scale : 20 epochs, SD-VAE latents
SiT-B/2
46.35
30.53
0.477
0.608
26.08
54.63
0.620
0.577
SiT-L/2
22.95
59.17
0.618
0.610
11.24
104.44
0.742
0.550
SiT-XL/2
18.88
69.34
0.647
0.608
9.25
118.80
0.760
0.543
Tokenizer : SiT-XL/2, 80 epochs
Appendix
Table 11: Full metrics for Fig. 8 . ImageNet 256×256 , no CFG.
Figure 9: Denoising heavily corrupted images. We take three clean images (left) and corrupt each with heavy noise (80% noise and 20% image). Then let REPA and RECON denoise them; each frame shows the clean-image estimate x^0(t) . RECON recovers the identity and layout of the original image better than REPA, which shows its strength in retaining semantics under heavy noise.
Figure 10: Uncurated generation results of SiT-XL/2 + RECON. We use classifier-free guidance with w=4.0 .
Figure 11: Uncurated generation results of SiT-XL/2 + RECON. We use classifier-free guidance with w=4.0 .
Figure 12: Uncurated generation results of SiT-XL/2 + RECON. We use classifier-free guidance with w=4.0 .
Figure 13: Uncurated generation results of SiT-XL/2 + RECON. We use classifier-free guidance with w=4.0 .
Recent representation alignment (REPA) methods accelerate diffusion transformer training by aligning projections of the transformer's hidden states with representations from pretrained visual encoders. In this work, we explore a reverse and complementary direction to REPA: rather than projecting diffusion representations into the encoder's space, we inject encoder representations into the diffusion transformer, allowing them to actively participate in the denoising process. To this end, we introduce REPresentation Injection (REPI), a training framework based on a scaffold-to-internalization strategy, in which projected encoder representations initially serve as a temporary scaffold and are then progressively internalized by the diffusion transformer. REPI outperforms REPA across a wide range of backbones and is highly complementary to it: combining the two yields substantial gains over either alone. Notably, with only 160K training steps, REPI + REPA matches vanilla SiT trained for 7M steps, a speedup of over 43.5×. Code will be available at https://jeneveuxpas.github.io/REPI
Han Fu, Jiacheng Chen, Baoquan Zhao +4
Sun Yat-sen University · Video Rebirth · The Hong Kong Polytechnic University
Recent advances in Diffusion Transformers (DiTs) demonstrate that aligning noisy latent states with well-trained semantic features-as pioneered by Representation Alignment (REPA)-can substantially accelerate training and improve generation fidelity. Subsequent analysis(e.g., iREPA) suggests that these gains arise primarily from transferring spatial structure contained in pre-trained vision representations. However, mostly existing alignment methods employ point-wise matching objectives or rely on implicit architectural tweaks, which fail to explicitly model the spatial relational geometry inherent in vision foundation models. We argue that such element-wise supervision is insufficient to capture the rich spatial topology of visual representations, and that effective alignment for generation should instead be formulated as an explicit structural constraint. To this end, we propose sREPA, a structural REPresentation Alignment framework to enforce consistency in the relational geometry of feature maps, rather than merely matching individual feature points. By encouraging the model to internalize holistic spatial layouts and structural correlations from pre-trained features, sREPA achieves faster and more stable convergence, along with improved sample quality, compared to state-of-the-art alignment strategies. Our code and models will be released.
While representation alignment with self-supervised models has been shown to improve diffusion model training, its potential for enhancing inference-time conditioning remains largely unexplored. We introduce Representation-Aligned Guidance (REPA-G), a framework that leverages these aligned representations, with rich semantic properties, to enable test-time conditioning from features, in generation. By optimizing a similarity objective (the potential) at inference, we steer the denoising process toward a conditioned representation extracted from a pre-trained feature extractor. Our method provides versatile control at multiple levels of granularity, ranging from patch level matching via single patches to broad semantic guidance using global image feature tokens. We further extend this to multi-concept composition, allowing for the faithful combination of distinct concepts. REPA-G operates entirely at inference time with no additional training required, offering a flexible and precise alternative to often ambiguous text prompts or coarse class labels. Our approach achieves high-quality, diverse generations on ImageNet and COCO. Code is available at https://github.com/valeoai/REPA-G
Nicolas Sereyjol-Garros, Ellington Kirby, Victor Letzelter +2
Valeo.ai, Paris, France · LTCI, Télécom Paris, Institut Polytechnique de Paris, France