Natural images are often conjectured to lie on a low-dimensional manifold within the high-dimensional pixel space. Recent work leverages this to build self-contained pixel-space diffusion models without specialized machinery or pretrained components. A plain Transformer on raw image patches suffices, provided it predicts the clean image rather than the noise and compresses each patch through an aggressive bottleneck. The mixing operator, however, has remained softmax attention, inherited from sequence modeling rather than derived from denoising. For Gaussian noise, the optimal denoiser admits a canonical expansion in polynomial functionals of the observation, the Wiener--Volterra series. We prove that softmax attention, and any normalized mixing rule (including linear attention), cannot realize any polynomial map beyond an affine one. In contrast, Volterra layers realize this class directly and, on a data manifold, achieve approximation rates governed by the manifold dimension rather than the ambient pixel dimension. We therefore propose the architecture these results prescribe; raw image patches passed through a bottleneck embedding, followed by cascaded low-rank Volterra layers, which carry no activation function because each layer is nonlinear in its own right. To our knowledge this is the first pixel-space diffusion architecture whose mixing operators are Volterra layers throughout, with no attention, no convolution, no pretrained components, and no activation function on the residual path. On ImageNet 256×256 and 512×512, the resulting model is competitive with self-contained pixel-space diffusion at matched parameter and FLOP budgets.
Figures & tables
Figure 1: The PoD architecture. (a) pixel pipeline; (b) block, both Transformer sublayers replaced by Volterra mixers; (c) the primitive they share, equation 6 .
Figure 2: The manifold picture in a controlled setting. (a) Samples from a dM=1 spiral in RD (grey) for three targets and three architectures. (b) Mean distance to the manifold d(M) against D .
Figure 3: Class-conditional samples from PoD-G/16 on ImageNet 256×256 .
pre-training
Method
token
perc.
self-sup.
params
GFLOPs
FID ↓
IS ↑
Latent-space diffusion
DiT-XL/2 ( Peebles and Xie, 2023 )
SD-VAE
VGG
–
675+49M
119
2.27
278.2
REPA, SiT-XL/2 ( Yu et al., 2025 )
SD-VAE
VGG
DINOv2
675+49M
119
1.42
305.7
LightningDiT-XL/2 ( Yao et al., 2025 )
VA-VAE
VGG
DINOv2
675+49M
119
1.35
295.3
DDT-XL/2 ( Wang et al., 2026b )
SD-VAE
VGG
DINOv2
675+49M
119
1.26
310.6
Table 1: ImageNet 256×256 , FID and IS on 50K samples. Pre-training columns list the external models each method needs, and parameters include the tokenizer decoder where used.
Figure 4: Quality against compute on ImageNet 256×256 . PoD and JiT at scales B–G, which differ only in the mixing operator, and other pixel-space diffusion models.
Configuration
Token mixer
Channel mixer
FID-50K ↓
JiT baseline
Self-attention
FFN
3.66
Polynomial token only
Volterra k=2
FFN
3.54
Full polynomial
Volterra k=2
Volterra k=2
3.41
Table 2: Architecture ablation on ImageNet 256×256 at B/16 scale.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Symbol
Meaning
Dimensions and indices
D=Dp
ambient dimension of the theory, the per-patch pixel dimension
Dp=3p2
ambient dimension of one p×p RGB patch ( Dp∈{768,3072} )
p
patch side length ( p=16 at 256×256 , p=32 at 512×512 )
dM
intrinsic dimension of the data manifold M , with dM≪D
d′
per-token bottleneck dimension ( d′∈{128,256} ); 2dM+1 suffices in Proposition 1 , and the trained value is deliberately larger (Appendix C )
Appendix
Table 3: Summary of notation.
Table 3 , continued.
Symbol
Meaning
Architecture
P
per-layer parameter budget in the function-class statements of Section 3 (distinct from the factor P )
VL,P
depth- L stacks of low-rank Volterra mixers with at most P parameters per layer (Proposition 2 ); P0 is the attainment threshold
Appendix
Table 8
Regime
σ/t
Why the bound is controlled
Constrains
Early
≫τ
target nearly affine
—
Intermediate
∼τ
tangent span; analyticity and depth
width, degree
Late
≪τ
posterior concentrated; compression cost vanishes
—
Appendix
Table 4: The three regimes of the sampling schedule, in the order sampling visits them.
At order k
Transformer block (one head), smooth φ
Low-rank Volterra layer
Kernel
ck[∑a(W2)ca(W1)a⋅⊗k]⊗Ai⋅⊗k
W(k) of equation 6 , rank at most hR
Slots
the same A in every slot; token rank one ( ≤nh with nh heads)
independent Qq(m) per slot
Across orders
one shared W1A ; ck fixed by φ
independent kernels
Wider sublayer
buys channel rank, never token rank
—
Appendix
Table 5: What each operator can parametrize at order k . The feed-forward sublayer does produce cross-token polynomial terms, but as a k -fold power of the attention matrix, scaled by a coefficient the architecture cannot choose. Shown for one head. With nh heads each slot draws from the same nh matrices.
Figure 5: Why a low-dimensional projection suffices (schematic). Noised inputs z lie in a tube of radius δ about the scaled data manifold tM . The clean signals consistent with z differ by moving along the manifold (tangent direction), not by leaving it, so the denoiser ft varies along the tangent direction and is nearly flat along the normal one. The bottleneck π keeps a d′ -dimensional image of the input. It sends z and x⋆ to the same point; this costs little, since ft takes nearly the same value at both. Further, it keeps x⋆ and x′ apart; this is essential, since ft takes different values at them.
pre-training
Method
token
perc.
self-sup.
params
GFLOPs
FID ↓
IS ↑
Latent-space diffusion
DiT-XL/2 ( Peebles and Xie, 2023 )
SD-VAE
VGG
–
675+49M
525
3.04
240.8
SiT-XL/2 ( Ma et al., 2024 )
SD-VAE
VGG
–
675+49M
525
2.62
252.2
REPA, SiT-XL/2 ( Yu et al., 2025 )
SD-VAE
VGG
DINOv2
675+49M
525
2.08
274.6
DDT-XL/2 ( Wang et al., 2026b )
SD-VAE
VGG
DINOv2
675+49M
525
1.28
305.1
Appendix
Table 6: Reference results on ImageNet 512×512 . Notation as in Table 1 . At this resolution the patch dimension is Dp=3072 , exceeding the hidden width of every model listed, so the bottleneck embedding is compressing by a larger factor than at 256 . Baseline numbers are as reported by Li and He (2026) .
Figure 6: Uncurated samples from PoD-G/16 on ImageNet 256×256 , mammals. Each row shows two classes, four samples each, with the ImageNet class index in parentheses. From the top, the rows show Chihuahua (151) and beagle (162); golden retriever (207) and cocker spaniel (219); Old English sheepdog (229) and boxer (242); Siberian husky (250) and Pomeranian (259); tabby (281) and Persian cat (283); Egyptian cat (285) and lion (291); tiger (292) and hyena (276); hare (331) and zebra (340).
Figure 7: Uncurated samples from PoD-G/16 on ImageNet 256×256 , birds, reptiles and other animals. Each row shows two classes, four samples each, with the ImageNet class index in parentheses. From the top, the rows show hen (8) and vulture (23); peacock (84) and macaw (88); goose (99) and black swan (100); flamingo (130) and pelican (144); vine snake (59) and boa constrictor (61); Indian cobra (63) and triceratops (51); jellyfish (107) and snail (113); king crab (121) and coral reef (973).
Figure 8: Uncurated samples from PoD-G/16 on ImageNet 256×256 , objects, food and nature. Each row shows two classes, four samples each, with the ImageNet class index in parentheses. From the top, the rows show castle (483) and dome (538); fountain (562) and space shuttle (812); cannon (471) and tractor (866); pizza (963) and bell pepper (945); espresso (967) and potpie (964); cliff (972) and rapeseed (984); yellow lady’s slipper (986) and mushroom (947); earthstar (995) and acorn (988).
Figure 9: Uncurated samples from PoD-G/32 on ImageNet 512×512 , animals. Each row shows two classes, four samples each, with the ImageNet class index in parentheses. From the top, the rows show goldfish (1) and cock (7); ostrich (9) and kite (21); vulture (23) and king snake (56); macaw (88) and flamingo (130); Sussex spaniel (220) and miniature poodle (266); jaguar (290) and cheetah (293); American black bear (295) and ice bear (296); hare (331) and marmoset (377).
Figure 10: Uncurated samples from PoD-G/32 on ImageNet 512×512 , objects, food and scenes. Each row shows two classes, four samples each, with the ImageNet class index in parentheses. From the top, the rows show cannon (471) and cliff dwelling (500); fountain (562) and fig (952); chocolate sauce (960) and pizza (963); burrito (965) and alp (970); coral reef (973) and valley (979); volcano (980) and daisy (985).
Recent advances in pixel-space diffusion models have narrowed the image quality gap with latent-space diffusion, but still converge more slowly and lag behind in final image quality. We argue that a key reason is the lack of an explicit representation prior: unlike latent diffusion, which usually denoises in a compact and structured latent space, pixel diffusion needs to learn denoising-friendly representations and pixel generation simultaneously from raw RGB space. To address this problem, we propose PixelDiT2, an end-to-end pixel-space diffusion model designed to decouple representation learning from pixel generation without introducing an autoencoder or latent reconstruction bottleneck. We propose representation grounding that uses a frozen pretrained vision foundation model to provide explicit per-patch representation guidance throughout denoising, allowing the pixel diffusion transformer to focus more on pixel generation. On ImageNet-256x256, PixelDiT2 achieves an FID of 1.46 after 600 epochs; at 512x512 resolution, PixelDiT2 achieves an FID of 1.48 after 680 epochs. Project page: https://pixeldit.github.io/pixeldit2/
End-to-end pixel-space diffusion models bypass the lossy compression of Latent Diffusion Models (LDMs) but struggle to jointly model low-frequency semantics and high-frequency signals in high-dimensional space. Existing works heavily rely on complex pixel decoders to alleviate this issue. In this paper, we challenge this trend by revealing that these decoders primarily compensate for the optimization difficulties inherent to velocity prediction (v-prediction). Under the clean data paradigm (x-prediction), they are redundant. Motivated by this insight, we advocate for simplicity over complexity and introduce PixelU, a minimalist, single-stage U-shaped Diffusion Transformer tailored for pixel space. PixelU abandons auxiliary decoders in favor of zero-cost skip connections, which provide an "information highway" that directly routes uncorrupted high-frequency spatial details from shallow to deep layers. To further enable the backbone to focus exclusively on modeling low-frequency semantics, we introduce a constant-channel spatial down-sampling mechanism as a natural low-pass filter, which compresses deep features into a compact, low-frequency semantic manifold. Extensive experiments demonstrate that this decoupling of frequencies could outperform the strong baseline (JiT-G) with only about 1/3 of its computation cost. On ImageNet 256×256 and 512×512, PixelU achieves FID of 1.63 and 1.92 respectively, surpassing recent pixel-space methods and establishing a simple yet powerful new paradigm for end-to-end diffusion models.
Denoising diffusion models are the dominant architecture for image generation, whereas most natural language generation and modeling are primarily handled by well-known transformer architectures employing attention mechanism. Here, we show that diffusion models also inherently use an attention mechanism very similar to that of transformers. Therefore, attention emerges as a universal machine learning principle, based on a general training objective. We also show similarities in basic functional principle of auto-encoders and attention-based models. These equivalences allows us to interchange these designs based on practical requirements. As an example, we can reformulate the diffusion framework to reduce the lengthy training process and computation-intensive image generation. Using this approach, a simplified algorithm is proposed for image generation which is based on attention mechanism. Results show that the attention-based implementation achieves comparable performance with significantly less effort and computational resources.
Farzan Haddadi, Leila Monfared, Ebrahim Rezaii +3
School of Electrical Engineering, Iran University of Science & Technology, Tehran, IRAN · Independent researcher