Beyond the Estimate: Wiener-Informed Decoding for Easier Training of Pixel-Space Flow Models

Ziqiao Wang; Yang You

cs.CVOct 26, 2026

Beyond the Estimate: Wiener-Informed Decoding for Easier Training of Pixel-Space Flow Models

Authors: Ziqiao Wang, Yang You

Organizations: NUS HPC-AI Lab

Abstract

Learning velocity fields directly in pixel space is difficult and slow. We show that coarse second-order image statistics can ease this task: frozen Wiener estimates improve velocity learning, with gains extending beyond the linear predictions they explicitly provide. Moreover, we observe similar but complementary gains from frozen bases and pixel decoding. These observations motivate Wiener-informed Decoding (WiD), which uses statistical knowledge both for direct prediction and to organize decoder computation without modifying native blocks. WiD combines a trainable Wiener prediction path with coefficient interfaces that inject features into the native decoder and return its predicted coefficient corrections, with these components initialized with data statistics. All introduced parameters, including the spectrum and coordinate transforms, are trained jointly under the original pixel-space velocity objective. Across model scales and decoder architectures, WiD accelerates training and improves generation quality with modest parameter overhead. It also yields smoother gradient trajectories and remains effective with representation alignment. WiD improves class-conditional and text-to-image generation across different resolutions. On ImageNet 256 x 256, PixelDiT-XL with WiD reaches FID 1.53 after 500 epochs, achieving comparable quality to PixelDiT with 37.5% fewer training epochs. Statistical denoisers contribute beyond the estimate: their computational structure can also ease pixel-space training.

Figures & tables

Panel of Figure 1 — Figure 1: Analytic Wiener bases ease pixel-space velocity learning. We train B-size models for 40 epochs on ImageNet $256{\times}256$ w/o REPA and report FID-10K results w/o CFG. (a) With the same DiT-B/16 setup, adding a DCT or PCA Wiener base rescues the velocity prediction baseline. (b) DiTs with analytic bases reach the performance range of several pixel-space methods. Each method uses its native training and evaluation recipe. (c) When added to native PixelDiT, our Wiener-informed Decoding provides stronger and more sustainable gains than frozen Wiener bases.

Panel of Figure 2 — Figure 2: Prediction gains and pathway alignment. (a, b) Using a well-trained official PixelDiT-XL checkpoint as the prediction reference, we fit affine maps to each model’s predictions and decompose the prediction error into linear and residual components. Relative to plain DiT, we compare how Wiener bases and pixel decoding reduce MSE in each component: (a) DiT with frozen PCA or DCT bases; (b) PixelDiT and PixelDiT+PCA base evaluated with patch-affine fits. (c) PixelDiT JVPs compared with Wiener responses, removing components parallel to perturbation.

Figure 3: Overview of our WiD framework. WiD integrates statistical operations as trainable components with the native decoder, and allows both branches to interact through coefficient interfaces.

Panel of Figure 4 — Figure 4: WiD improves training stability, remains compatible with REPA, and applies across decoder architectures. We compare B-size models trained for 40 epochs on ImageNet $256{\times}256$ . (a) PixelDiT+WiD exhibits smoother pre-clipping gradient norms than native PixelDiT with REPA. (b,c) WiD improves both PixelDiT and DeCo and gains persist with REPA.

Method	Epochs	#Params.	FID $\downarrow$	sFID $\downarrow$	IS $\uparrow$	Precision $\uparrow$	Recall $\uparrow$
REPA-XL ( Yu et al., 2025 )	800	675M	1.42	4.70	305.7	0.80	0.65
DDT-XL ( Wang et al., 2026b )	400	675M	1.26	–	310.6	0.79	0.65
RAE-XL ( Zheng et al., 2026 )	800	839M	1.13	–	262.6	0.78	0.67
ADM ( Dhariwal & Nichol, 2021 )	400	554M	4.59	5.25	186.7	0.82	0.52
RIN ( Jabri et al., 2023 )	480	410M	3.42	–	182.0	–	–
VDM++ ( Kingma & Gao, 2023 )	–	–	2.12	–	267.7	–	–

Table 1: Class-conditional generation on ImageNet

256\times 256

. Gray entries denote latent-space models.

\uparrow

and

\downarrow

denote higher and lower values are better, respectively. Our Wiener-informed Decoding consistently improves the generation quality of PixelDiT-XL during the training process.

Method	Epochs	#Params.	FID $\downarrow$	sFID $\downarrow$	IS $\uparrow$	Precision $\uparrow$	Recall $\uparrow$
REPA-XL ( Yu et al., 2025 )	200	675M	2.08	4.19	274.6	0.83	0.58
DDT-XL ( Wang et al., 2026b )	256 + 100	675M	1.28	4.22	305.1	0.80	0.63
RAE-XL ( Zheng et al., 2026 )	400	839M	1.13	–	259.6	0.80	0.63
PixNerd-XL/16 ( Wang et al., 2026a )	320 + 40	700M	2.45	5.19	275.8	0.79	0.63
JiT-H/32 ( Li & He, 2026 )	600	956M	1.94	–	309.1	–	–
PixelU-H/32 ( Guo et al., 2026 )	600	1.17B	1.92	5.98	322.1	0.80	0.58

Table 2: Class-conditional generation on ImageNet

512\times 512

. Gray entries denote latent-space models. Results demonstrate that WiD is also effective across different resolutions.

Method	Steps	GenEval							DPG-Bench
	Steps	Sin.Obj.	Two.Obj.	Counting	Colors	Pos.	Color Attr.	Overall $\uparrow$	Average $\uparrow$
DeCo-XXL/16	80K	0.85	0.32	0.27	0.74	0.12	0.24	0.43	73.1
DeCo-XXL/16 +WiD	80K	0.92	0.45	0.33	0.81	0.15	0.27	0.49	78.8

Table 3: Text-to-image generation at

256\times 256

after 80K training steps. We evaluate both methods on 553 GenEval prompts and 1,065 DPG-Bench prompts, with four images per prompt. Higher is better for all scores. WiD improves the performance of DeCo-XXL/16 on both benchmarks.

Method	GenEval							DPG-Bench
	Sin.Obj.	Two.Obj.	Counting	Colors	Pos.	Color Attr.	Overall $\uparrow$	Average $\uparrow$
PixelFlow ( Chen et al., 2025b )	–	–	–	–	–	–	0.60	77.9
PixNerd ( Wang et al., 2026a )	0.97	0.86	0.44	0.83	0.71	0.53	0.73	80.9
PixelDiT-T2I ( Yu et al., 2026 )	1.00	0.94	0.70	0.90	0.53	0.65	0.78	83.7
DeCo-XXL ( Ma et al., 2026 )	1.00	0.92	0.72	0.91	0.80	0.79	0.86	81.4
DeCo-XXL †	0.99	0.92	0.65	0.92	0.70	0.76	0.82	82.3

Table 4: Comparison of text-to-image generation at

512\times 512

on GenEval and DPG-Bench.

\dagger

marks results of our test on the public checkpoint of DeCo with its own evaluation protocol. WiD also improves the performance of DeCo-XXL across different resolutions.

Method	FID $\downarrow$	sFID $\downarrow$	IS $\uparrow$
PixelDiT-B/16	60.01	14.10	24.89
PixelDiT-B/16 +WiD	41.71	9.18	35.14
w/o Wiener prediction	49.54	10.88	29.93
w/o Statistics init.	49.02	9.85	30.35
w/o Coefficient interface	50.67	9.14	28.97

Table 5: Ablation experiments on WiD components and initialization on ImageNet

256\times 256

. Using PixelDiT-B as the backbone, we train each model for 40 epochs with REPA and report FID, sFID, and IS evaluated on 50K samples from EMA weights without CFG. Removing the Wiener prediction, coefficient interface or statistics initialization leads to the reduction of generation quality, demonstrating the effectiveness of components and initialization strategy in our WiD.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5: Illustration of decoder structures in PixelDiT and DeCo.

	DiT-B	PixelDiT-B	PixelDiT-XL	DeCo-B	DeCo-XXL
Architecture
DiT depth $N$	12	12	26	12	16
Decoder depth $M$	–	2	4	3	3
Hidden size $D$	768	768	1152	768	1536
Attention heads	12	12	16	12	24
Pixel hidden size	–	16	16	32	32

Appendix

Table 6: Detailed settings of the five model configurations used in this work. Parameter counts refer to the native denoising networks, excluding WiD, REPA projectors and external encoders.

ImageNet $\mathit{256}\times\mathit{256}$
Method	Epochs	CFG	Interval	FID $\downarrow$	sFID $\downarrow$	IS $\uparrow$	Precision $\uparrow$	Recall $\uparrow$
PixelDiT-XL/16 +WiD	80	2.90	[0.10, 1.00]	2.09	4.73	278.7	0.79	0.60
PixelDiT-XL/16 +WiD	160	2.60	[0.10, 1.00]	1.87	4.50	280.1	0.79	0.62
PixelDiT-XL/16 +WiD	200	2.75	[0.10, 0.90]	1.72	4.51	295.4	0.78	0.64
PixelDiT-XL/16 +WiD	260	2.75	[0.10, 0.90]	1.65	4.58	296.4	0.78	0.65
PixelDiT-XL/16 +WiD	260	3.00	[0.12, 0.90]	1.64	4.69	295.1	0.77	0.65

Appendix

Table 7: We provide more results of PixelDiT-XL/16+WiD with different CFG weights and intervals.

Epoch	$t=0.1$	$t=0.25$	$t=0.5$	$t=0.75$	$t=0.9$
DiT+PCA relative to plain DiT
10	6.519	9.084	15.104	22.932	19.754
20	5.585	8.244	13.326	19.678	15.777
30	5.874	8.368	12.840	18.729	15.142
40	5.718	8.211	12.547	18.246	14.880
DiT+DCT relative to plain DiT

Appendix

Table 8: Time-resolved remaining-error reductions, in units of

10^{-3}

. Positive values indicate closer agreement with the reference after removing the fitted affine responses. Each block specifies its comparison baseline; all use patch-affine fits except DiT+DCT.

Epoch	$t=0.1$	$t=0.25$	$t=0.5$	$t=0.75$	$t=0.9$
PCA reference
10	$-0.137$	0.200	0.405	0.316	0.292
20	$-0.101$	0.261	0.514	0.439	0.403
30	$-0.054$	0.307	0.548	0.458	0.430
40	0.048	0.400	0.591	0.481	0.431
DCT reference

Appendix

Table 9: Time-resolved differences in pathway alignment with Wiener responses. Entries are the mean cosine to the reference for the direct pixel pathway minus that for the contextual pathway.

Figure 6: Qualitative results of DeCo-XXL/16+WiD on $512\times 512$ text-to-image generation.

Figure 7: $512\times 512$ samples of PixelDiT-XL/16+WiD at 260 + 50 epochs. We use time-shift 2.0 and classifier-free guidance with $w=4.0$ .

Figure 8: $256\times 256$ samples of PixelDiT-XL/16+WiD at 260 epochs. We use classifier-free guidance with $w=4.0$ . Class label = “arctic wolf” (270).

Figure 9: $256\times 256$ samples of PixelDiT-XL/16+WiD at 260 epochs. We use classifier-free guidance with $w=4.0$ . Class label = “baseball” (429).

Figure 10: $256\times 256$ samples of PixelDiT-XL/16+WiD at 260 epochs. We use classifier-free guidance with $w=4.0$ . Class label = “cheeseburger” (933).

Figure 11: $256\times 256$ samples of PixelDiT-XL/16+WiD at 260 epochs. We use classifier-free guidance with $w=4.0$ . Class label = “cliff drop-off” (972).

Flow-Based Generative Modeling

Flow-based generative modeling learns probability transports from simple noise to data for fast sampling, density estimation, and controllable generation.

See all 563 papers

Asymmetric Flow Models
May 13, 2026Hansheng Chen, Jan Ackermann, Minseo Kim +2Flow Matching T2I Generation
Stanford University
WiT: Waypoint Diffusion Transformers for Alleviating Trajectory Conflict in Pixel-Space Image Generation
Mar 16, 2026Hainuo Wang, Mingjia Li, Xiaojie GuoFlow Matching Diffusion Transformer
College of Intelligence and Computing, Tianjin University, Tianjin 300350, China
Dino Forcing Flow Models: Do not denoise what you can predict
Oct 8, 2026Arijit Ghosh, Lucas Degeorge, Paul Couairon +3Self-Conditioning Flow-Based Generative Modeling
ENPC, IP Paris · Ecole Polytechnique, IP Paris · AMIAD +1

May 13, 2026J/K move · Enter open · S save

Asymmetric Flow Models

Hansheng Chen, Jan Ackermann, Minseo Kim, Gordon Wetzstein+1

Stanford University

Flow Matching T2I Generation Image Generation Pixel-Space Diffusion Models Generative Modeling Flow-Based Generative Modeling

Open paper AskPDF arXiv

Flow-based generation in high-dimensional pixel spaces is difficult because velocity prediction requires modeling high-dimensional noise, even when data has strong low-rank structure. We present Asymmetric Flow Modeling (AsymFlow), a rank-asymmetric velocity parameterization that restricts noise prediction to a low-rank subspace while keeping data prediction full-dimensional. From this asymmetric prediction, AsymFlow analytically recovers the full-dimensional velocity, leaving the network architecture and the standard flow-matching training and sampling procedures unchanged. On ImageNet 256

\times

256, AsymFlow achieves a leading 1.57 FID, outperforming prior DiT/JiT-like pixel diffusion models by a large margin. AsymFlow also provides the first-ever route for finetuning pretrained latent flow models into pixel-space models: aligning the low-rank pixel subspace to the latent space gives a seamless initialization that preserves the latent model's high-level semantics and structure, so finetuning mainly improves low-level mismatches rather than relearning pixel generation. We show that the pixel AsymFlow model finetuned from FLUX.2 klein 9B establishes a new state of the art for pixel-space text-to-image generation, beating its latent base on HPSv3, DPG-Bench, and GenEval while qualitatively showing substantially improved visual realism.

Abstract

Flow-Based Generative Modeling

Explore similar work

Asymmetric Flow Models

WiT: Waypoint Diffusion Transformers for Alleviating Trajectory Conflict in Pixel-Space Image Generation

Dino Forcing Flow Models: Do not denoise what you can predict