cs.CVNov 9, 2026

Just Image Transformers with Late-Stage Pixel Refinement

Authors: Denis Lukovnikov, Andreas Müller, Asja Fischer

Organizations: Ruhr University Bochum

Abstract

Recent work on latent-free diffusion transformers has demonstrated the feasibility of training image generators without a latent VAE. This avoids compression artifacts from the VAE and simplifies the training pipeline. In particular, JiT showed that even a standard transformer can be trained to generate images when the model is trained to predict the final image (xx-prediction). Even though JiT unlocks training efficiency comparable to latent models, it still suffers from reduced image quality likely arising from modeling the image as a collection of large patches. In this work, we revisit JiT and show that adding simple convolutional pixel-pathway layers in a post-training phase significantly improves generation quality. Moreover, we show that the added pixel-pathway does not need to be active in the early denoising stages, and can be activated as late as in the last 20%20\% of the denoising process. Our experiments on ImageNet@256 show improvements compared to the original JiT results, achieving an FID of 1.781.78 (from 2.342.34 with corrected sampling), without using additional losses.

Figures & tables

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Decoding Without a Decoder: Direct Pixel Prediction from Diffusion Transformers

    Nov 17, 2026Hansheng Chen, Shengqu Cai, Jan Ackermann +4Diffusion ModelsDiffusion Transformer

  2. Representation Alignment for Just Image Transformers is not Easier than You Think

    Mar 15, 2026Jaeyo Shin, Jiwook Kim, Hyunjung ShimDiffusion ModelsDiffusion Model Alignment

  3. Hierarchical Diffusion Transformer for End-to-End Image Generation

    Nov 17, 2026Xiaoyu Wu, Yifei Wang, Weihang Guo +4Diffusion-Based Image GenerationDiffusion Models