Recent work on latent-free diffusion transformers has demonstrated the feasibility of training image generators without a latent VAE. This avoids compression artifacts from the VAE and simplifies the training pipeline. In particular, JiT showed that even a standard transformer can be trained to generate images when the model is trained to predict the final image (x-prediction). Even though JiT unlocks training efficiency comparable to latent models, it still suffers from reduced image quality likely arising from modeling the image as a collection of large patches. In this work, we revisit JiT and show that adding simple convolutional pixel-pathway layers in a post-training phase significantly improves generation quality. Moreover, we show that the added pixel-pathway does not need to be active in the early denoising stages, and can be activated as late as in the last 20% of the denoising process. Our experiments on ImageNet@256 show improvements compared to the original JiT results, achieving an FID of 1.78 (from 2.34 with corrected sampling), without using additional losses.
Figures & tables
Figure 1
Figure 3: The architecture used by Li and He (2026) is an image transformer with an input bottleneck.
Figure 4: Our model architecture (left) and ConvNeXt-inspired pixel-head block architecture (right). Note the similarity of the ConvNeXt block to a transformer block.
Figure 5: The predictions of the pretrained JiT-B/16 model at different timesteps t . Notice the smoothness of predictions at t≤0.5 .
Figure 6: Effect of denominator clamping during 50-step Heun sampling. (a) Quantitative results for pretrained JiT models with the original clamp ( τclip=0.05 ), unclamped sampling, and matched-noise controls. CFG and EMA settings follow Li and He (2026) . (b–d) JiT-L/16 samples with original clamp, no clamp, and no clamp followed by Gaussian noise with σm=0.018 , respectively.
Method
Params.
GFLOPs
Time ↓
gFID ↓
sFID ↓
IS ↑
(M)
(per FE)
(s/img)
JiT-B
Pre-trained ( ϵ=0.05 )
131.3
50.4
1.44
3.64
7.21
269.0
Pre-trained ( ϵ=0.0 )
131.3
50.4
1.44
4.61
8.68
282.4
Fine-tuned (100 epochs)
131.3
50.4
1.44
4.29
7.21
275.7
Conv. pixel head
132.9
60.9
1.74
3.42
5.49
264.3
Table 1: Comparison of pixel-head variants for JiT-B and JiT-L. For Dual Head variants, GFLOPs are reported with the linear head and the convolutional head separately. Further details and breakdowns are provided in Appendix D .
Method
REPA
Epochs
Params
NFE
gFID ↓
IS ↑
Simple Diffusion
( Hoogeboom et al., 2023 )
–
800
2.0B
–
2.44
256.3
PixelDiT-XL
( Yu et al., 2026 )
✓
320
797M
100×2
1.61
292.7
DeCo-XL/16
( Ma et al., 2026b )
✓
600
682M
100×2
1.69
304.0
DiP-XL/16
( Chen et al., 2026c )
✓
600
631M
100×2
1.79
281.9
PixelU-H/16
( Guo et al., 2026 )
✓
600
1.17B
100×2
1.63
305.9
JiT-H/16
( Li and He, 2026 )
–
600
953M
100×2
1.86
303.4
Table 2: Comparison on ImageNet @256 for recent pixel-space generative models. For full comparison, see Tab. 5 in Appendix A . For the H-model, we use AdaLN-based conditioning and wider pixel head (BW-Cond variant; see Appendix C ) and t∼LogitNormal(0,1) . The convolutional head is activated at t=0.5 .
Gradual
Hard
Method
.50 – .60
.60 – .70
.70 – .80
.80 – .90
.90 – .99
.60
.70
.80
.90
.95
JiT-B + Dual Pixel Head
3.39
3.41
3.41
3.46
5.39
3.40
3.41
3.42
3.58
4.98
JiT-L + Dual Pixel Head
2.24
2.23
2.21
2.20
3.55
2.24
2.22
2.20
2.27
3.24
Table 3: Transition ablation for JiT-B and JiT-L with Dual Pixel Heads on ImageNet 256×256 , measured using gFID with 50k samples. Smooth transitions blend the linear and pixel heads over the indicated interval; hard switches activate the pixel head at the indicated timestep.
JiT-B + Dual Pixel Head
gFID ↓
IS ↑
Baseline
3.41
269.3
No skip from xt
3.91 (+0.50)
277.9
Table 4: Ablation results on xt skip connection (left) and residual pathway (right).
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Epochs
Params
NFE
gFID ↓
IS ↑
Latent Generative Models
DiT-XL
( Peebles and Xie, 2023 )
1400
675M + 49M
250×2
2.27
278.2
SiT-XL
( Ma et al., 2024 )
1400
675M + 49M
250×2
2.06
277.5
MaskDiT
( Zheng et al., 2024 )
1600
675M + 49M
79×2
2.28
276.6
REPA
( Yu et al., 2025 )
800
675M + 49M
250×2
1.42
305.7
LightningDiT
( Yao et al., 2025 )
800
675M + 41M
250×2
1.35
295.3
Appendix
Table 5: Full comparison on 256×256 generation on ImageNet. NFE values with ×2 denote nominal counts with classifier-free guidance; guidance intervals may reduce the actual count. RAE uses AutoGuidance with an additional guiding model.
Figure 7: Weights applied to the x -prediction MSE: normalized soft-min-SNR ( γ=4 ) and v -loss. The v -loss curve is clipped at the plot boundary.
Figure 8: Timestep sampling densities for logit-normal distributions with (μ,σ)=(−0.8,0.8) and (0,1) , and uniform sampling on [0,1] .
Timestep distribution
Loss weighting
FID ↓
IS ↑
Logit-normal ( μ=−0.8 , σ=0.8 )
v -loss
3.65
282.86
Logit-normal ( μ=0 , σ=1 )
v -loss
3.41
274.04
Uniform ∗
v -loss
4.01
285.48
Uniform
Soft-min-SNR ( γ=4 )
3.41
269.34
Appendix
Table 6: Comparison of timestep sampling and loss weighting on ImageNet 256×256 using the dual-head JiT-B/16 model with the B head. We select the lowest FID from final-epoch CFG/EMA sweeps from JiT’s implementation, evaluated on 50,000 images and report IS from the same evaluation. All selected results use 50-step Heun sampling and decoder-transition start 0.5 and end 0.7.
Backbone
Head
Width
Depth
Head params (M)
Head GFLOPs
Share (%)
FID ↓
IS ↑
B/16
T
64
2
3.36
6.22
10.9
3.71
274.15
B/16
B
64
4
3.53
10.79
17.5
3.41
269.34
B/16
B-Cond
64
4
6.77
15.62
23.5
3.46
273.43
B/16
BW
128
4
7.71
39.83
43.9
3.28
272.28
B/16
BW-Cond
128
4
14.30
55.94
52.4
3.37
281.83
B/16
L
128
6
8.37
58.08
53.3
3.51
282.14
Appendix
Table 7: Head configuration ablation on ImageNet 256×256 . We report FID and Inception Score (IS) on 50,000 generated images using 50-step Heun sampling. For each configuration, we select the lowest-FID evaluation from CFG/EMA sweeps using JiT’s evaluator, and report its corresponding IS. Width and depth specify the head channel width and number of blocks. Head parameters (millions) include all weights and biases in the convolutional or ResNet head, excluding the backbone and overlap head. Head GFLOPs count convolution and matrix-multiplication operations per image for one forward pass. Head share is relative to the full forward pass. All runs use uniform timestep sampling and soft-min-SNR loss ( γ=4 ).
Model
Active heads
Total
Backbone
Heads
B/16
Linear
50.82
50.14 (98.66%)
0.68 (1.34%)
Conv
60.93
50.14 (82.29%)
10.79 (17.71%)
Both
61.61
50.14 (81.38%)
11.47 (18.62%)
L/16
Linear
176.39
175.48 (99.48%)
0.91 (0.52%)
Conv
186.81
175.48 (93.94%)
11.32 (6.06%)
Both
187.72
175.48 (93.48%)
12.24 (6.52%)
Appendix
Table 8: Computational cost per network forward pass. Values are GFLOPs; percentages indicate the fraction of the corresponding row’s total. “Heads” includes all active output heads. One CFG-guided evaluation requires two network forwards and therefore doubles all listed costs.
Model
Head schedule
Total
Backbone
Heads
B/16
Conv throughout
12.1
9.9 (82.3%)
2.1 (17.7%)
Head: B
Blend 0.5 – 0.7
11.1
9.9 (89.8%)
1.1 (10.2%)
Switch at 0.8
10.5
9.9 (94.8%)
0.5 (5.2%)
L/16
Conv throughout
37.0
34.7 (93.9%)
2.2 (6.1%)
Head: B
Blend 0.5 – 0.7
36.0
34.7 (96.6%)
1.2 (3.4%)
Switch at 0.8
35.3
34.7 (98.3%)
0.6 (1.7%)
Appendix
Table 9: Estimated computation per generated image using 50 sampling steps with Heun, a final Euler step, and CFG. The convolutional head variant is shown below each model. Values are TFLOPs; percentages give each component’s share of the total. Head costs aggregate all active head evaluations over the sampling trajectory.
Figure 9: Qualitative samples on ImageNet 256×256 , conditioned on the indicated classes, generated with JiT-H/16 + Dual Head and CFG of 2.8.
Figure 10: Qualitative samples on ImageNet 256×256 , conditioned on the indicated classes, generated with JiT-H/16 + Dual Head and CFG of 2.8.
Figure 11: Qualitative samples on ImageNet 256×256 , conditioned on the indicated classes, generated with JiT-H/16 + Dual Head and CFG of 2.8.
Figure 12: Qualitative samples on ImageNet 256×256 , conditioned on the indicated classes, generated with JiT-H/16 + Dual Head and CFG of 2.8.
Figure 13: Qualitative samples on ImageNet 256×256 , conditioned on the indicated classes, generated with JiT-L/16 + Dual Head and CFG of 2.6.
Latent diffusion models rely on a decoder to map denoised latents to pixels, adding complexity and inference cost. We introduce Decoding Without a Decoder (DWD), which lets a finetuned diffusion transformer produce final pixels directly through a linear head. Unlike previous methods, where the denoising output leads to final pixels, DWD separates the two roles: the original denoising head continues to drive sampling, then the new DWD head predicts the pixels at the final step. We extend pretrained latent models with DWD efficiently by finetuning only the last few transformer blocks, training the DWD head with pixel quality losses while anchoring the denoising head to the pretrained model to preserve the sample trajectory. Across latent image and video generators, DWD removes the decoder and reduces inference cost while maintaining or even improving pixel quality. On Wan video generation, it visibly reduces detail distortion and flickering, lowering warping error by 23%. Remarkably, the same separation also improves pixel diffusion models that do not even use a decoder: a final-step DWD head produces substantially better texture detail in text-to-image generation, and improves ImageNet FID from 1.55 to 1.43, a new state of the art among pixel flow models.
Hansheng Chen, Shengqu Cai, Jan Ackermann +4
Stanford University · Luma AI · California Institute of Technology
Representation Alignment (REPA) has emerged as a simple way to accelerate Diffusion Transformers training in latent space. At the same time, pixel-space diffusion transformers such as Just image Transformers (JiT) have attracted growing attention because they remove a dependency on a pretrained tokenizer, and then avoid the reconstruction bottleneck of latent diffusion. This paper shows that the REPA can fail for JiT. REPA yields worse FID for JiT as training proceeds and collapses diversity on image subsets that are tightly clustered in the representation space of pretrained semantic encoder on ImageNet. We trace the failure to an information asymmetry: denoising occurs in the high dimensional image space, while the semantic target is strongly compressed, making direct regression a shortcut objective. We propose PixelREPA, which transforms the alignment target and constrains alignment with a Masked Transformer Adapter that combines a shallow transformer adapter with partial token masking. PixelREPA improves both training convergence and final quality. PixelREPA reduces FID from 3.66 to 3.17 for JiT-B/16 and improves Inception Score (IS) from 275.1 to 284.6 on ImageNet 256×256, while achieving >2× faster convergence. Finally, PixelREPA-H/16 achieves FID=1.81 and IS=317.2. Our code is available at https://github.com/kaist-cvml/PixelREPA.
Jaeyo Shin, Jiwook Kim, Hyunjung Shim
KAIST AI, 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Republic of Korea
Pixel-space diffusion learns visual representations and image generation jointly in a single end-to-end training stage, without a separately pretrained visual encoder. However, plain pixel-space Transformers still trail leading two-stage latent diffusion models in generation quality and training efficiency. We hypothesize that part of this gap arises from retaining one token per patch throughout the backbone, which does not explicitly encourage a visual hierarchy that summarizes related content at progressively coarser levels. To address this, we introduce Hierarchical Diffusion Transformers (HiT), which progressively group related tokens and represent each group with a single token. The main Transformer bottleneck processes these compact group tokens, and ungrouping restores features to their original image positions. A compression budget regulates the average compression ratio across the batch, allowing group memberships and counts to vary across images and diffusion timesteps during both training and generation. The model exhibits emergent semantic structure, learning to group coherent objects and object parts without segmentation labels. In text-to-image generation, HiT further learns to group caption words with image patches depicting the same concepts, without region annotations. HiT achieves 1.48 FID on ImageNet 256×256 and reaches comparable generation quality to JiT using approximately 40% of its cumulative training FLOPs. For text-to-image generation, HiT exceeds UniLiP's average score across five benchmarks with roughly 6× fewer denoiser parameters and 3× fewer forward FLOPs. These results suggest that learning a visual hierarchy within the generator breaks down the barrier to stronger image generation without external visual pretraining.