Persistence Forcing: Exploiting Feature Specialization in Pixel-Space Diffusion

Chong Wang; Zixuan Fu; Shiqi Huang; Siyuan Yang; Hao Cheng; Bihan Wen

cs.CVSep 28, 2026

Persistence Forcing: Exploiting Feature Specialization in Pixel-Space Diffusion

Authors: Chong Wang, Zixuan Fu, Shiqi Huang, Siyuan Yang, Hao Cheng, Bihan Wen

Organizations: Nanyang Technological University · KTH Royal Institute of Technology · Hebei University of Technology

Abstract

Pixel-space diffusion Transformers (DiTs) directly operate on high-dimensional visual data, yet their hidden representations typically undergo uniform refinement across depth. Natural images, however, are inherently organized at different levels of granularity. Global structure can often be represented compactly, whereas local textures and fine details require richer representations. Motivated by this, we introduce heterogeneous refinement in pixel-space DiTs, assigning different feature groups distinct refinement budgets across depth. Consequently, an ordered feature specialization emerges: sparsely refined features predominantly encode global visual structure, whereas more frequently refined features increasingly specialize toward localized, high-frequency details. We refer to these two groups as persistent and active features, respectively. Building on this emergent specialization, we introduce Persistence Forcing (PerF), which explicitly exploits this persistent--active feature organization for pixel-space image generation. This enables persistent features to continuously condition actively refined features, allowing stable global information to guide the ongoing refinement of finer visual details. During generative sampling, this interaction further induces a meaningful guidance direction that promotes coherent global structure and naturally complements classifier-free guidance. On ImageNet $256\times256$ , PerF-L achieves FID of $1.91$ , approaching $1.86$ of JiT-H with only half the parameters, while PerF-H further achieves FID of $1.63$ and $1.76$ on ImageNet $256\times256$ and $512\times512$ , respectively.

Figures & tables

Figure 1: Heterogeneous refinement induces ordered feature specialization. We visualize the contributions of different feature groups to the final RGB prediction. Under uniform refinement, feature groups show no systematic ordering. Under heterogeneous refinement, increasing refinement budget $R$ organizes their contributions from global structure toward fine, high-frequency details.

Figure 2: From uniform to heterogeneous refinement. (a) In vanilla JiT ( Li and He, 2026 ) , every hidden dimension is updated at every block, yielding the same refinement budget $R_{i}=L$ . (b) Our design updates only the first $d_{\ell}$ dimensions at layer $\ell$ while bypassing the remaining. Varying $d_{\ell}$ across depth assigns different refinement budgets to different feature dimensions.

Model	Params.	100 Epoch		200 Epoch
Model	Params.	FID $\downarrow$	IS $\uparrow$	FID $\downarrow$	IS $\uparrow$
Uni.	131M	42.1	37.6	33.1	49.8
Hetero.	131M	42.7	36.2	33.8	47.1

Table 1: Heterogeneous refinement alone does not readily achieve better generation.

Figure 3: Overview of Persistence Forcing. (a) Persistent-to-active conditioning explicitly uses preserved persistent features as structural context for actively refined features. (b) The resulting controllable interaction is further exploited to construct Persistence Guidance.

Latent-space Diffusion Models
Model	Params.	Epochs	FID $\downarrow$	IS $\uparrow$	Prec. $\uparrow$	Rec. $\uparrow$
DiT-XL/2 ( Peebles and Xie, 2023 )	675M	1400	2.27	278.2	0.83	0.57
SiT-XL/2 ( Ma et al., 2024 )	675M	1400	2.06	277.5	0.83	0.59
REPA-XL/2 ( Yu et al., 2025 )	675M	800	1.42	305.7	0.80	0.64
DDT-XL/2 ( Wang et al., 2026b )	675M	400	1.26	310.6	0.79	0.65
RAE-XL/2 ( Zheng et al., 2026 )	839M	800	1.13	262.6	0.78	0.67

Table 2: Comparison with latent-space and pixel-space diffusion models on ImageNet

256\times 256

Latent-space Diffusion Models
Model	Params.	Epochs	FID $\downarrow$	IS $\uparrow$	Prec. $\uparrow$	Rec. $\uparrow$
SiT-XL/2 ( Ma et al., 2024 )	675M	600	2.62	252.2	0.84	0.57
REPA-XL/2 ( Yu et al., 2025 )	675M	800	2.08	274.6	0.83	0.58
RAE-XL/2 ( Zheng et al., 2026 )	839M	800	1.13	259.6	0.80	0.63
Pixel-space Diffusion Models
ADM-G ( Dhariwal and Nichol, 2021 )	554M	400	7.72	172.7	0.84	0.53

Table 3: Comparison with latent-space and pixel-space diffusion models on ImageNet

512\times 512

Figure 4: Qualitative results on ImageNet $512\times 512$ and $256\times 256$ using PerF-H.

Table 9

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Depth-wise evolution of persistent and active features. We visualize the intermediate RGB readouts and feature-space PCA representations of a persistent feature group and the corresponding actively refined features. The persistent features are updated at Blocks 1 and 12 and bypass the intermediate blocks, whereas the active features are updated at every block. After only the first update, persistent features already exhibit coherent global object structure, which remains stable throughout the bypass trajectory before being further refined upon re-entry at Block 12. In contrast, active features continue to evolve across depth and predominantly exhibit residual and higher-frequency structures at intermediate blocks.

Figure 8: Frequency specialization across refinement budgets. We measure the normalized spectral centroid of the RGB contribution from feature groups with different refinement budgets

R

, using equal-sized 64-channel subsets for a controlled comparison. Larger refinement budgets generally correspond to higher spectral centroids across diffusion timesteps, quantitatively confirming the coarse-to-fine feature organization observed in Figure 1 .

Model	Depth	$\bm{d_{\max}}$	$\bm{d_{\min}}$	Width schedule $\{d_{\ell}\}$
PerF-B/16	12	1024	512	[1024, 896, 832, 768, 640, 576, 512, 512, 576, 704, 896, 1024]
PerF-L/16	24	1600	512	[1600, 1472, 1408, 1280, 1152, 1088, 1088, 960, 896, 832, 832, 768, 704, 640, 640, 576, 576, 512, 640, 768, 896, 1088, 1280, 1600]
PerF-H/16	32	2000	640	[2000, 1920, 1840, 1760, 1600, 1520, 1440, 1360, 1360, 1280, 1200, 1120, 1040, 1040, 1040, 960, 880, 800, 800, 800, 720, 720, 720, 640, 800, 880, 1040, 1120, 1280, 1520, 1680, 2000]

Appendix

Table 7: Layer-wise width schedules used in PerF. For each model scale, the variable-width backbone is approximately parameter-matched to the corresponding JiT backbone by preserving the attention and MLP parameter budget,

\sum_{\ell=1}^{L}d_{\ell}^{2}\approx Ld^{2}

Architecture
	PerF-B/16	PerF-L/16	PerF-H/16	PerF-H/32
Image size	256	256	256	512
Depth	12	24	32	32
Patch size	16	16	16	32
Hidden dim $d_{\max}$	1024	1600	2000	2000
Hidden dim $d_{\min}$	512	512	640	640

Appendix

Table 8: Configurations of PerF experiments.

Model	Params.	Epochs	FID $\downarrow$	IS $\uparrow$	Prec. $\uparrow$	Rec. $\uparrow$
ADM ( Dhariwal and Nichol, 2021 )	554M	400	10.94	–	0.69	0.63
PixelFlow-XL ( Chen et al., 2025 )	677M	320	12.23	103.3	0.63	0.66
PixNerd-XL ( Wang et al., 2026a )	700M	320	15.61	88.9	0.59	0.68
DeCo-XL/16 ( Ma et al., 2026a )	682M	320	14.88	88.2	0.60	0.68
PixelGen-XL/16 ( Ma et al., 2026b )	676M	80	5.11	159.2	0.72	0.63
JiT-B/16 ( Li and He, 2026 )	131M	600	25.42	63.2	0.54	0.66

Appendix

Table 9: Comparison of class-conditional generation on ImageNet at

256\times 256

resolution without CFG. All other settings and notation follow Table 2 .

Table 10: Interaction with representation alignment. Left: effect of different REPA schedules on PerF-B/16. Right: comparison with PixelREPA, which specifically adapts representation alignment to JiT.

Scale	JiT		PerF		Overhead
	Params.	GFLOPs	Params.	GFLOPs	Params.	GFLOPs
Base	131M	25.22	137M	26.51	+4.6%	+5.1%
Large	459M	87.94	471M	90.83	+2.6%	+3.3%
Huge	953M	181.95	987M	190.44	+3.6%	+4.7%

Appendix

Table 11: Per-forward computational cost on ImageNet

256\times 256

. PerF introduces modest parameter and FLOP overhead over the corresponding JiT backbones across model scales.

Panel of Figure 9 — Figure 9: Uncurated class-conditional samples on ImageNet $256\times 256$ using PerF-H/16

Panel of Figure 11 — Figure 11: Uncurated class-conditional samples on ImageNet $256\times 256$ using PerF-H/16.

Explore similar work

PixelDiT2: Representation-Grounded Pixel Diffusion Transformers
Sep 21, 2026Yongsheng Yu, Wei Xiong, Yichen Sheng +2Contextual Grounding
NVIDIA · University of Rochester
Advanced Pixel Diffusion Model with Guided Sparse Global Refinement
Sep 1, 2026Weiyi You, Jinhua Zhang, Xingyu Zhou +3Pixelloop Pixels
University of Electronic Science and Technology of China
DuSPiT: Dual-Branch Sub-Patch Pixel Diffusion Transformer
Jul 20, 2026Yunpeng Bai, Yossi Gandelsman, Michaël GharbiDiffusion Transformers Multi-Reference Image Generation
The University of Texas at Austin · Reve

Sep 21, 2026, cs.CVJ/K move · Enter open · S save

PixelDiT2: Representation-Grounded Pixel Diffusion Transformers

Yongsheng Yu, Wei Xiong, Yichen Sheng, Shiqiu Liu+1

NVIDIA · University of Rochester

Contextual Grounding

Open paper AskPDF arXiv

Recent advances in pixel-space diffusion models have narrowed the image quality gap with latent-space diffusion, but still converge more slowly and lag behind in final image quality. We argue that a key reason is the lack of an explicit representation prior: unlike latent diffusion, which usually denoises in a compact and structured latent space, pixel diffusion needs to learn denoising-friendly representations and pixel generation simultaneously from raw RGB space. To address this problem, we propose PixelDiT2, an end-to-end pixel-space diffusion model designed to decouple representation learning from pixel generation without introducing an autoencoder or latent reconstruction bottleneck. We propose representation grounding that uses a frozen pretrained vision foundation model to provide explicit per-patch representation guidance throughout denoising, allowing the pixel diffusion transformer to focus more on pixel generation. On ImageNet-256x256, PixelDiT2 achieves an FID of 1.46 after 600 epochs; at 512x512 resolution, PixelDiT2 achieves an FID of 1.48 after 680 epochs. Project page: https://pixeldit.github.io/pixeldit2/

Abstract

Explore similar work

PixelDiT2: Representation-Grounded Pixel Diffusion Transformers

Advanced Pixel Diffusion Model with Guided Sparse Global Refinement

DuSPiT: Dual-Branch Sub-Patch Pixel Diffusion Transformer