Organizations: East China Normal University · State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University
Pixel-space diffusion models directly model image distributions but remain difficult to optimize. Recent methods alleviate this challenge through target reparameterization, while still relying on a fixed clean-image target throughout denoising. Through empirical analysis, we identify a scale-time mismatch: image structures become predictable from coarse to fine as noise decreases, whereas existing models are forced to predict the full image even under high noise, resulting in low-SNR gradients that hinder optimization. To resolve this mismatch, we propose Observation Operator Diffusion, a unified framework that aligns both the supervision trajectory and feature refinement with the intrinsic recovery order of image structures. Specifically, we replace fixed full-image supervision along the standard flow path with a time-indexed observation trajectory that evolves from coarse structures to the full image during denoising. This trajectory is instantiated with a family of Gaussian-Lanczos operators at varying observation scales, yielding a path-consistent training objective. We further introduce GL-CoDA, a decoder that injects scale-specific Gaussian-Lanczos observations across decoding stages for coarse-to-fine feature refinement. Extensive experiments show that the proposed approach converges substantially faster while consistently improving generation quality, achieving an FID of 1.52 on ImageNet-256.
Figures & tables
Figure 1: Fixed full-image supervision vs. time-indexed observation trajectory. Existing pixel-space diffusion models use the full clean image x0 as the target throughout the denoising path. We instead introduce a time-indexed observation trajectory yt=At(x0) , whose image content evolves from coarse structure to the full image as denoising proceeds.
Figure 2: Scale-dependent predictability during denoising. We decompose the single-step prediction residual into multiscale structural bands and measure the prediction error at each scale. The heatmap (denoising time × spatial scale) and representative trajectories (inset) show that coarse structures become recoverable earlier than fine details, exposing the mismatch with fixed full-image supervision.
Figure 3: Optimization benefits of the observation trajectory. Left: Observation Operator Diffusion converges substantially faster than conventional full-image supervision while achieving a lower final FID. Right: Gradient signal-to-noise ratio (SNR) over diffusion time. By avoiding supervision of unresolved fine details, the observation trajectory maintains higher-SNR gradients in the high-noise regime, leading to more efficient optimization.
Figure 4: Overview of the proposed Observation Operator Diffusion framework. Top: Gaussian–Lanczos observation trajectory, where a family of operators At maps the clean image x0 to time-indexed observations yt=At(x0) , progressively transitioning from coarse structures to the full image along denoising. Bottom: two-stage Patch–Pixel backbone, comprising a Patch-DiT encoder and a GL-CoDA decoder. The GL structure branch injects scale-specific structural responses across decoding stages. The right panel details the construction of each Gaussian–Lanczos operator At .
Figure 5: Spatial and frequency responses of the Gaussian–Lanczos kernel. (a) Spatial response. Gaussian smoothing attenuates the signed sidelobes of the Lanczos component, reducing potential ringing. (b) Frequency response. At a matched −3dB cutoff, the Gaussian–Lanczos kernel exhibits a narrower transition band than the Gaussian-only kernel, yielding sharper scale selectivity and stronger attenuation of high-frequency detail.
Space
Method
Epochs
#params
gFID ↓
sFID ↓
IS ↑
Precision ↑
Recall ↑
Latent
REPA ( Yu et al. 2024 )
800
675M
1.42
4.70
305.7
0.80
0.65
SVG-XL ( Shi et al. 2025 )
1400
675M
1.92
–
264.9
–
–
DDT-XL ( Wang et al. 2025 )
400
675M
1.26
–
310.6
0.79
0.65
RAE-XL ( Zheng et al. 2025 )
800
839M
1.13
–
262.6
0.78
0.67
Pixel
FractalMAR-H ( Li et al. 2025 )
600
844M
6.15
–
348.9
0.81
0.46
FARMER ( Zheng et al. 2026 )
320
1.9B
3.60
–
269.2
0.81
0.51
Table 1: Quantitative results on ImageNet 256×256 for class-conditioned generation.
Figure 6: Class-conditional samples on ImageNet. Top: samples at 512×512 . Bottom: samples at 256×256 .
Space
Method
gFID ↓
sFID ↓
IS ↑
Precision ↑
Recall ↑
Latent
DiT-XL ( Peebles and Xie 2023a )
3.04
5.02
240.8
0.84
0.54
SiT-XL ( Ma et al. 2024 )
2.62
4.18
252.2
0.84
0.57
MaskDiT ( Zheng et al. 2023 )
2.50
5.10
256.2
0.83
0.56
U-ViT-H ( Bao et al. 2023 )
4.05
–
263.8
0.84
0.48
REPA ( Yu et al. 2024 )
2.08
4.19
274.6
0.83
0.58
RAE-XL ( Zheng et al. 2025 )
1.13
–
259.6
0.80
0.63
Table 2: Quantitative comparison on ImageNet 512×512 .
Architecture
#params
Full-image path
G–L path
JiT
2B
4.11
3.79
PixelDiT
797M
2.36
2.14
Ours
798M
2.22
1.95
Table 3: Observation-Path Ablation across Architectures, measured by FID.
(a) Kernel
(b) Depth assignment
Kernel
FID ↓
Assignment
FID ↓
Identity
2.22
Coarse only
2.28
Gaussian
2.10
Fine only
2.12
Lanczos
2.08
Fine → Coarse
2.30
G–L
1.95
Coarse → Fine
1.95
Table 4: Kernel-Component and Depth-Assignment Ablations.
Initial kernel scales
Schedule
FID ↓
Lanczos f0
Gaussian σ0
4
1
Linear
2.00
2
2
Linear
2.06
2
1
Linear
1.95
2
1
Cosine
2.08
Table 5: Kernel-Scale and Temporal-Schedule Sensitivity.
Figure 7: Depth-wise feature visualization at t=0.25 . The first column shows x0 (top) and z0.25 (bottom); B1–B4 show block input features (top) and our injected Gaussian–Lanczos structural responses (bottom).