Representation alignment (REPA) is widely used to accelerate the training of latent-space diffusion transformers (DiTs). However, applying it to pixel-space models such as Just Image Transformers (JiT) is challenging. Our analysis reveals a pronounced representation conflict between discriminative alignment and pixel-space generation. Compared with its latent counterpart, JiT learns representations that are substantially less aligned with DINOv2. Applying REPA to JiT consistently reduces the reconstructability of its intermediate representations, whereas this effect is not observed in Latent JiT. We attribute this conflict to JiT directly reconstructing pixels, which requires its representations to hierarchically encode discriminative semantics and reconstruction-relevant visual information across network depths. To address these challenges, we propose Routed Representation Alignment (RouteREPA), which uses lightweight, representation-dependent routers to adaptively route task-oriented representations for alignment and generation. We further apply routing across multiple network depths to accommodate their depth-dependent representation requirements. Extensive experiments on ImageNet-1K at 256x256 resolution show that RouteREPA consistently improves JiT across model scales and training budgets, outperforming standard REPA. On JiT-H/16, RouteREPA further surpasses PixelREPA using only one-third of its training epochs without classifier-free guidance.
Figures & tables
Figure 1: Overview of RouteREPA . (a) Architecture of RouteREPA . Task-specific channel-wise routers direct intermediate JiT representations to the generation and representation alignment branches according to their representation preference. (b) FID-50K during ImageNet-1K training at 256×256 resolution without CFG. RouteREPA requires fewer than 100 and 200 training epochs to match JiT-H/16 and PixelREPA-H/16 trained for 600 epochs, respectively.
Figure 2: Pixel-space generation of JiT introduces a pronounced representation conflict : (a) JiT aligns less closely with DINOv2 than Latent JiT. (b) REPA lowers the reconstruction capability of JiT intermediate representations. (c) In contrast, no such degradation is observed in Latent JiT.
Table 3
Figure 3: Images generated by JiT-H/16 + RouteREPA . Additional samples appear in Appendix D .
Method
Routing Strategy
Routing config.
Layers
Params
FID ↓
IS ↑
JiT-B/16
–
–
–
131M
32.2
50.1
+ REPA
–
–
4
131M
27.5
58.4
+ REPA
Orthogonal
–
4
131M
27.4
57.9
+ RouteREPA
Gated
Shared
4
132M
24.2
63.4
+ RouteREPA
Gated
Gen. only
4
132M
27.3
58.6
+ RouteREPA
Gated
Align. only
4
131M
28.0
56.5
Table 3: Ablations of RouteREPA on JiT-B/16. We compare different routing strategies, routing configurations, and routing locations. The compared design choices are highlighted in blue, and the best result within each comparison is shown in bold.
Figure 4: Analysis of generation and alignment representations. (a) Alignment-routed representations align with DINOv2 even better than REPA. (b, c) Generation-routed representations better preserve decodable information for image structure and local appearance, while such reconstruction-related information is suppressed in alignment-routed representations.
Figure 5: Learned routing dynamics at layers 4 and 6. Blue denotes routing for generation and green for alignment; darker shades indicate later epochs. Channels are sorted by routing weight.
Method
Params (M)
Train GFLOPs
Infer. GFLOPs
Peak mem. (GiB)
Train step (ms)
Infer. forward (ms)
REPA
953
1,146
364
113
583
48.6
RouteREPA
956 (+0.3%)
1,146 ( <0.1% )
364 ( <0.1% )
114 (+0.8%)
592 (+1.6%)
49.5 (+1.7%)
Table 4: Computational cost on JiT-H/16. RouteREPA adds negligible computational overhead compared with REPA.
Method
FID ↓
IS ↑
SiT-B/2
33.00
–
+ REPA
24.40
59.90
+ RouteREPA
21.85
67.04
Table 5: RouteREPA improves SiT-B/2 at 400k iterations (CFG 1).
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
JiT-B
JiT-L
JiT-H
Architecture
Depth
12
24
32
Hidden dimension
768
1,024
1,280
Attention heads
12
16
16
Image size
256×256
Patch size
16×16
Appendix
Table 6: Architecture, training, sampling, and alignment configurations. The JiT settings follow the public recipe ( Li & He, 2025 ) ; the final block lists the RouteREPA and standard REPA configurations.
Figure 6: Feature-to-pixel reconstructions at three flow timesteps. Lower t corresponds to a more heavily corrupted input. As more clean-image signal becomes available, the generation route retains more object structure and local appearance than the alignment route.
Figure 7: Single-layer routing dynamics in JiT-B/16 models. (a,b) L4-only routing; (c,d) L6-only routing. Blue denotes generation and green alignment.
CFG
Method
Nearest 100
Farthest 100
1.0
JiT
34.7
23.9
REPA
24.5
20.2
PixelREPA
27.5
20.6
RouteREPA
23.1
18.6
1.5
JiT
20.0
18.0
REPA
15.2
15.9
Appendix
Table 8: FID on DINOv2-neighborhood subsets across guidance scales. We report subset-reference FID after perturbing images at t=0.2 and denoising with 50 Heun intervals. Lower is better.
Figure 8: Representative images from the 100 most similar and 100 least similar samples in the DINOv2 feature space for the ImageNet tabby class.
Figure 9: Uncurated samples from JiT-H/16 + RouteREPA on ImageNet 256×256 .
Figure 10: Uncurated samples from JiT-H/16 + RouteREPA on ImageNet 256×256 .
Figure 11: Uncurated samples from JiT-H/16 + RouteREPA on ImageNet 256×256 .
Figure 12: Uncurated samples from JiT-H/16 + RouteREPA on ImageNet 256×256 .
Figure 13: Uncurated samples from JiT-H/16 + RouteREPA on ImageNet 256×256 .
Figure 14: Uncurated samples from JiT-H/16 + RouteREPA on ImageNet 256×256 .
Figure 15: Uncurated samples from JiT-H/16 + RouteREPA on ImageNet 256×256 .
Representation Alignment (REPA) has emerged as a simple way to accelerate Diffusion Transformers training in latent space. At the same time, pixel-space diffusion transformers such as Just image Transformers (JiT) have attracted growing attention because they remove a dependency on a pretrained tokenizer, and then avoid the reconstruction bottleneck of latent diffusion. This paper shows that the REPA can fail for JiT. REPA yields worse FID for JiT as training proceeds and collapses diversity on image subsets that are tightly clustered in the representation space of pretrained semantic encoder on ImageNet. We trace the failure to an information asymmetry: denoising occurs in the high dimensional image space, while the semantic target is strongly compressed, making direct regression a shortcut objective. We propose PixelREPA, which transforms the alignment target and constrains alignment with a Masked Transformer Adapter that combines a shallow transformer adapter with partial token masking. PixelREPA improves both training convergence and final quality. PixelREPA reduces FID from 3.66 to 3.17 for JiT-B/16 and improves Inception Score (IS) from 275.1 to 284.6 on ImageNet 256×256, while achieving >2× faster convergence. Finally, PixelREPA-H/16 achieves FID=1.81 and IS=317.2. Our code is available at https://github.com/kaist-cvml/PixelREPA.
Jaeyo Shin, Jiwook Kim, Hyunjung Shim
KAIST AI, 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Republic of Korea
Recent work on latent-free diffusion transformers has demonstrated the feasibility of training image generators without a latent VAE. This avoids compression artifacts from the VAE and simplifies the training pipeline. In particular, JiT showed that even a standard transformer can be trained to generate images when the model is trained to predict the final image (x-prediction). Even though JiT unlocks training efficiency comparable to latent models, it still suffers from reduced image quality likely arising from modeling the image as a collection of large patches. In this work, we revisit JiT and show that adding simple convolutional pixel-pathway layers in a post-training phase significantly improves generation quality. Moreover, we show that the added pixel-pathway does not need to be active in the early denoising stages, and can be activated as late as in the last 20% of the denoising process. Our experiments on ImageNet@256 show improvements compared to the original JiT results, achieving an FID of 1.78 (from 2.34 with corrected sampling), without using additional losses.
Recent advances in Diffusion Transformers (DiTs) demonstrate that aligning noisy latent states with well-trained semantic features-as pioneered by Representation Alignment (REPA)-can substantially accelerate training and improve generation fidelity. Subsequent analysis(e.g., iREPA) suggests that these gains arise primarily from transferring spatial structure contained in pre-trained vision representations. However, mostly existing alignment methods employ point-wise matching objectives or rely on implicit architectural tweaks, which fail to explicitly model the spatial relational geometry inherent in vision foundation models. We argue that such element-wise supervision is insufficient to capture the rich spatial topology of visual representations, and that effective alignment for generation should instead be formulated as an explicit structural constraint. To this end, we propose sREPA, a structural REPresentation Alignment framework to enforce consistency in the relational geometry of feature maps, rather than merely matching individual feature points. By encouraging the model to internalize holistic spatial layouts and structural correlations from pre-trained features, sREPA achieves faster and more stable convergence, along with improved sample quality, compared to state-of-the-art alignment strategies. Our code and models will be released.