Reviving ConvNeXt for Efficient Convolutional Diffusion Models

Taesung Kwon; Lorenzo Bianchi; Lennart Wittke; Felix Watine; Fabio Carrara; Jong Chul Ye; Romann Weber; Vinicius Azevedo

cs.CVMar 10, 2026

Reviving ConvNeXt for Efficient Convolutional Diffusion Models

Authors: Taesung Kwon, Lorenzo Bianchi, Lennart Wittke, Felix Watine, Fabio Carrara, Jong Chul Ye, Romann Weber, Vinicius Azevedo

Organizations: KAIST · ETH Zürich · ISTI-CNR · University of Pisa ·

Abstract

Recent diffusion models increasingly favor Transformer backbones, motivated by the remarkable scalability of fully attentional architectures. Yet the locality bias, parameter efficiency, and hardware friendliness--the attributes that established ConvNets as the efficient vision backbone--have seen limited exploration in modern generative modeling. Here we introduce the fully convolutional diffusion model (FCDM), a model having a backbone similar to ConvNeXt, but designed for conditional diffusion modeling. We find that using only 50% of the FLOPs of DiT-XL/2, FCDM-XL achieves competitive performance with 7 $\times$ and 7.5 $\times$ fewer training steps at 256 $\times$ 256 and 512 $\times$ 512 resolutions, respectively. Remarkably, FCDM-XL can be trained on a 4-GPU system, highlighting the exceptional training efficiency of our architecture. Our results demonstrate that modern convolutional designs provide a competitive and highly efficient alternative for scaling diffusion models, reviving ConvNeXt as a simple yet powerful building block for efficient generative modeling.

Figures & tables

Figure 2: Is scalability exclusive to transformers? Our Fully Convolutional Diffusion Model (FCDM) exhibits clear scalability: it is more efficient and achieves better convergence than Diffusion Transformers (DiTs). Bubble size indicates the FLOPs of each diffusion model. Across all scales (ordered by parameter count).

Model	Train steps	FLOPs (G) $\downarrow$	Throughput (it/s) $\uparrow$	FID $\downarrow$
DiT-S/2	400K	6	1234	68.4
FCDM-S	400K	3	2687	48.5
DiT-B/2	400K	23	380.1	43.5
FCDM-B	400K	12	1002	26.2
DiT-L/2	400K	81	114.6	23.3
FCDM-L	400K	48	381.3	13.8

Table 1: FCDM consistently yields lower FLOPs, higher throughput, and converges faster to superior performance compared to DiT across all scales.

Figure 3: The Fully Convolutional Diffusion Model (FCDM) architecture. (a) Details of the ConvNeXt block. (b) Our FCDM block, which incorporates conditioning via adaptive layer normalization. (c) We train conditional latent FCDMs. The input latent is processed by multiple FCDM blocks arranged in an easily scalable U-shaped architecture.

Model (Params)	Blocks $L$	Channel $C$	FLOPs (G)	$\frac{\text{FLOPs (FCDM)}}{\text{FLOPs (DiT)}}$	$\frac{\text{FLOPs (FCDM)}}{\text{FLOPs (DiCo)}}$
FCDM-S (32.7M)	2	128	3.1	50.8 $\%$	72.9 $\%$
FCDM-B (127.7M)	2	256	12.2	53.0 $\%$	72.3 $\%$
FCDM-L (504.5M)	2	512	48.3	59.9 $\%$	80.2 $\%$
FCDM-XL (698.8M)	3	512	64.6	54.5 $\%$	74.0 $\%$

Table 2: Parameter counts are aligned with the Diffusion Transformer (DiT) configurations for the Small (S), Base (B), Large (L), and XLarge (XL) variants. FLOPs are computed at a 256

\times

256 resolution. Overall, FCDM uses roughly 50

\%

of the FLOPs of DiT and about 75

\%

of the FLOPs of DiCo.

Figure 4: Simple illustration of DiCo and FCDM block. Both architectures share a similar high-level structure, but FCDM adopts an inverted bottleneck that expands channels for richer representations while keeping the computational cost of depthwise convolution unchanged. DiCo employs CCA with an additional 1 $\times$ 1 convolution, whereas FCDM uses GRN, requiring no extra pointwise convolutions. FCDM also does not include DiCo’s feed-forward module, resulting in a simpler and more efficient block.

Model	Architecture Type	FLOPs (G) $\downarrow$	Throughput (it/s) $\uparrow$	FID $\downarrow$	IS $\uparrow$	Precision $\uparrow$	Recall $\uparrow$
DiT-S/2 (400K) (ICCV 2023)	Transformer	6.1	1234.0	68.40	-	-	-
DiC-S (400K) (CVPR 2025)	Conv	5.9	3148.8	58.68	25.82	-	-
DiG-S/2 (400K) (CVPR 2025)	Hybrid	4.3	961.2	62.06	22.81	0.39	0.56
DiCo-S (400K) (NeurIPS 2025)	Conv	4.3	1695.7	49.97	31.38	0.48	0.58
FCDM-S (400K)	Conv	3.1	2687.2	48.52	31.64	0.48	0.58
DiT-B/2 (400K)	Transformer	23.0	380.1	43.47	-	-	-

Table 3: Scalability comparisons on ImageNet 256

\times

256. For each model scale, we report FID, IS, Precision, and Recall (50K samples without guidance), and efficiency metrics (training iterations, FLOPs, throughput). FCDM-XL achieves superior convergence while using 50

\%

fewer FLOPs than DiT-XL/2. The best results are highlighted in bold . Evaluated methods operate in the latent space.

Figure 5: FCDM improves FID across all model scales. FID-50K over training iterations for both DiT and FCDM. Across all model scales, FCDM converges much faster.

GAN
Model	Training epochs	FLOPs (G) $\downarrow$	Throughput (it/s) $\uparrow$	FID $\downarrow$	IS $\uparrow$	Precision $\uparrow$	Recall $\uparrow$
BigGAN-deep	-	-	-	6.95	171.4	0.87	0.28
StyleGAN-XL	-	-	-	2.30	265.12	0.78	0.53
Pixel diffusion
ADM-U	400	742.0	-	3.94	215.8	0.83	0.53
VDM++	560	-	-	2.12	267.7	-	-

Table 4: Benchmarking class-conditional image generation on ImageNet 256

\times

256. We compare representative models in terms of FID, IS, Precision, Recall (with guidance), and efficiency metrics (training epochs, FLOPs, throughput). FCDM-XL achieves competitive performance with superior efficiency. The best results are highlighted in bold .

Figure 6: Benchmarking class-conditional image generation performance and efficiency on ImageNet 256 $\times$ 256. Left : FID versus total training cost. Right : FID versus throughput. One zettaFLOP corresponds to $10^{21}$ FLOPs ( $10^{12}$ GFLOPs). A training iteration is assumed to cost about $3\times$ one evaluation (forward + backward to inputs + backward to weights). Red denotes fully convolutional, Green denotes hybrid, and Blue denotes fully transformer-based models.

Model	Training iterations	FLOPs (G) $\downarrow$	Throughput (it/s) $\uparrow$	FID $\downarrow$	IS $\uparrow$	Precision $\uparrow$	Recall $\uparrow$
DiT-XL/2	400K	524.7	18.6	20.94	66.3	0.74	0.58
DiG-XL/2	400K	-	-	17.36	69.4	0.75	0.63
DiC-XL	400K	464.3	124.2	15.32	93.6	-	-
DiC-H	400K	817.2	68.6	12.89	101.8	-	-
FCDM-XL	400K	257.7	129.6	10.23	108.7	0.79	0.60
DiT-XL/2	1.3M	524.7	18.6	13.78	-	-	-

Table 5: Benchmarking class-conditional image generation on ImageNet 512

\times

512. We report FID, IS, Precision, Recall (without guidance), and efficiency metrics for representative models. Even at this resolution, FCDM surpasses models trained for 3M iterations with only 1M iterations and achieves the best efficiency in FLOPs and throughput. The best results are highlighted in bold .

Model	FLOPs (G) $\downarrow$	FID $\downarrow$	IS $\uparrow$
FCDM-L (Default: 7 $\times$ 7 DWConv)	48.3	19.97	69.19
$\to$ 5 $\times$ 5 DWConv	48.2	20.48	66.69
$\to$ 3 $\times$ 3 DWConv	48.1	21.28	64.11
FCDM-L (Default: GRN)	48.3	19.97	69.19
$\to$ CCA ∗ [ 1 ]	48.3	23.85	61.60
FCDM-L (Default: w/o Feedforward)	48.3	19.97	69.19

Table 6: Ablation study on FCDM design choices. We analyze the effects of kernel size, GRN, DiCo [ 1 ] design choices, and the FCDM block. Training iterations are fixed to 200K.

\ast

indicates that

C

is adjusted to match FLOPs to ensure a fair comparison.

Figure 7: Feature activation visualization. We visualize features before and after the GRN layer during sampling for each image on the left. The first 64 channels of the last block in the first stage are shown as 8 $\times$ 8 grids. GRN clearly reduces channel redundancy.

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

	FCDM-S	FCDM-B	FCDM-L	FCDM-XL	FCDM-XL
Resolution	256 $\times$ 256	256 $\times$ 256	256 $\times$ 256	256 $\times$ 256	512 $\times$ 512
Architecture
Input dim.	32 $\times$ 32 $\times$ 4	32 $\times$ 32 $\times$ 4	32 $\times$ 32 $\times$ 4	32 $\times$ 32 $\times$ 4	64 $\times$ 64 $\times$ 4
Num. blocks ( $L$ )	2	2	2	3	3
Hidden channels ( $C$ )	128	256	512	512	512
1 $\times$ 1 conv. expansion ratio ( $r$ )	3	3	3	3	3

Appendix

Table 7: Hyperparameter setup of FCDM model scales. For all scales of FCDM, we adopt the same experimental settings as DiT.

Model	FLOPs (G)	Training Steps	FID $\downarrow$	sFID $\downarrow$	IS $\uparrow$	Precision $\uparrow$	Recall $\uparrow$
FCDM-S	3.10	50K	103.93	15.03	12.01	0.3013	0.3513
		100K	77.17	12.15	17.11	0.3809	0.4473
		150K	66.85	11.02	20.68	0.4136	0.5106
		200K	60.33	10.70	23.71	0.4396	0.5537
		250K	56.08	10.54	26.30	0.4568	0.5609
		300K	53.01	10.59	28.10	0.4687	0.5806

Appendix

Table 8: Performance of FCDMs across scales and training steps on ImageNet 256

\times

256 (Diffusion). Scaling FCDMs consistently leads to improved generative performance when trained with the diffusion objective.

Model	FLOPs (G)	Training Steps	FID $\downarrow$	sFID $\downarrow$	IS $\uparrow$	Precision $\uparrow$	Recall $\uparrow$
FCDM-S	3.10	50K	103.10	13.10	12.23	0.2863	0.3111
		100K	76.95	11.12	16.96	0.3826	0.4547
		150K	66.97	10.18	20.56	0.4228	0.4953
		200K	60.53	9.62	23.90	0.4428	0.5372
		250K	55.94	9.53	26.16	0.4670	0.5515
		300K	52.43	9.22	28.78	0.4787	0.5632

Appendix

Table 9: Performance of FCDMs across scales and training steps on ImageNet 256

\times

256 (Flow-Matching). Scaling FCDMs also demonstrates consistent improvements in generative performance when trained with the flow-matching objective.

Figure 8: Spectral energy of predicted noise across diffusion steps. FCDM consistently exhibits higher spectral energy than DiT across the entire diffusion process, suggesting potential for better preservation of high-frequency components.

Model Configuration	FLOPs (G) $\downarrow$	FID $\downarrow$	IS $\uparrow$
FCDM-L (Default: 7 $\times$ 7 DWConv)	48.3	19.97	69.19
7 $\times$ 7 $\to$ 9 $\times$ 9 DWConv	48.4	23.85	59.68
7 $\times$ 7 $\to$ 11 $\times$ 11 DWConv	48.6	23.93	60.18

Appendix

Table 10: Ablation study on larger kernels. We analyze effects of larger kernel sizes. Training iterations are fixed to 200K.

SD-VAE
Model Configuration	Training iterations	FLOPs (G) $\downarrow$	FID $\downarrow$
DiT-XL/2	400K	118.6	19.47
FCDM-XL	400K	64.6	11.57
EQ-VAE
DiT-XL/2	400K	118.6	14.50
FCDM-XL	400K	64.6	10.72

Appendix

Table 11: Ablation study on autoencoders. Across different latent spaces (SD-VAE and EQ-VAE), FCDM consistently outperforms DiT.

Model Configuration	Params (M)	FID $\downarrow$
DiT-B/2	130	55.00
FCDM (Iso.)	62 ( $\sim$ 48%)	41.15

Appendix

Table 12: Ablation study on isotropic architecture. Comparison of our isotropic variant against DiT-B/2 [ 46 ] . All models are trained on ImageNet

256\times 256

for 200k iterations under identical settings.

Model Configuration	FLOPs (G) $\downarrow$	TP (it/s) $\uparrow$	FID $\downarrow$	IS $\uparrow$
FCDM-L (Default: DWConv)	48.3	381.3	19.97	69.19
DWConv $\to$ NA ∗ [ 14 ]	45.9	122.8	29.81	50.92

Appendix

Table 13: Ablation study on replacing depthwise convolution with local self-attention. Neighborhood Attention (NA) [ 14 ] with a 7

\times

7 window is used as a replacement for depthwise convolution. All models are trained for 200K iterations at matched FLOPs.

\ast

indicates that

C

is adjusted to match FLOPs to ensure a fair comparison.

Asymmetric U-Net Ablations
Model Configuration	Hidden channel $C$	Depths	Params (M)	FLOPs (G) $\downarrow$	TP (it/s) $\uparrow$	FID $\downarrow$
FCDM-L (Default: Sym. U-Net)	512	[2, 4, 8, 4, 2]	504.5	48.3	381.3	19.97
Asym. U-Net	512	[2, 3, 8, 5, 2]	504.5	48.3	381.3	19.55
FCDM-XL (Default: Sym. U-Net)	512	[3, 6, 12, 6, 3]	698.8	64.6	272.7	15.54
Asym. U-Net	512	[3, 5, 12, 7, 3]	698.8	64.6	272.7	15.54
Asym. U-Net	512	[3, 3, 12, 9, 3]	698.8	64.6	272.7	15.55

Appendix

Table 14: Ablation study on asymmetric block allocation. All models trained for 200K iterations under identical training settings.

Model Configuration	Hidden channel $C$	Depths	Params (M)	FLOPs (G) $\downarrow$	TP (it/s) $\uparrow$	FID $\downarrow$
FCDM-L (Default: Scaling)	512	[2, 4, 8, 4, 2]	504.5	48.3	381.3	19.97
Uniform	600	[4, 4, 4, 4, 4]	496.6	62.8	261.6	17.63
FCDM-XL (Default: Scaling)	512	[3, 6, 12, 6, 3]	698.8	64.6	272.7	15.54

Appendix

Table 15: Ablation study on block scaling strategies. All models are trained on ImageNet

256\times 256

for 200k iterations under identical settings. The channel size is adjusted to match the total number of parameters.

Figure 9: Conditioning modules for class and text in the FCDM architecture. (a) FCDM block with conditioning vector $c$ , (b) Conditioning module for class conditioning, (c) Conditioning module for text conditioning incorporating the CLIP text encoder.

Figure 10: Qualitative results on text-to-image generation (MS-COCO). We use classifier-free guidance with scale = 4.0.

Model	Architecture Type	FLOPs (G) $\downarrow$	Throughput (it/s) $\uparrow$	FID $\downarrow$	IS $\uparrow$	Precision $\uparrow$	Recall $\uparrow$
MDT-XL/2 (400K) [ 12 ]	Transformer	118.7	83.9	16.42	-	-	-
FasterDiT-XL/2 (400K) [ 67 ]	Transformer	118.6	80.5	11.90	-	-	-
FCDM-XL (400K)	Conv	64.6	272.7	10.72	108.0	0.69	0.63
MDT-XL/2 (1.3M)	Transformer	118.7	83.9	9.60	-	-	-
FasterDiT-XL/2 (1M)	Transformer	118.6	80.5	8.72	121.17	0.68	0.67
FCDM-XL (1M)	Conv	64.6	272.7	7.91	135.55	0.71	0.64

Appendix

Table 16: Additional comparisons on ImageNet 256

\times

256 without guidance. We report efficiency metrics (FLOPs, throughput) and performance metrics (FID, IS, Precision, Recall) using 50K samples. The best results are shown in bold .

Model	Training epochs	FLOPs (G) $\downarrow$	Training FLOPs (Z) $\downarrow$	Throughput (it/s) $\uparrow$	FID $\downarrow$	IS $\uparrow$	Precision $\uparrow$	Recall $\uparrow$
MDT-XL/2 [ 12 ]	500	118.7	0.228	83.9	2.15	249.3	0.82	0.58
FasterDiT-XL/2 [ 67 ]	400	118.6	0.182	80.5	2.03	270.0	0.81	0.60
FCDM-XL	400	64.6	0.099	272.7	2.03	285.7	0.81	0.59

Appendix

Table 17: Additional comparisons on ImageNet 256

\times

256 with guidance. We report efficiency metrics (FLOPs, training FLOPs, throughput) and performance metrics (FID, IS, Precision, Recall) using 50K samples. The best results are shown in bold .

Panel of Figure 11 — Figure 11: Uncurated 512 $\times$ 512 FCDM-XL samples. Classifier-free guidance scale = 4.0

Panel of Figure 12 — Figure 12: Uncurated 512 $\times$ 512 FCDM-XL samples. Classifier-free guidance scale = 4.0

Panel of Figure 13 — Figure 13: Uncurated 512 $\times$ 512 FCDM-XL samples. Classifier-free guidance scale = 4.0

Panel of Figure 14 — Figure 14: Uncurated 512 $\times$ 512 FCDM-XL samples. Classifier-free guidance scale = 4.0

Panel of Figure 15 — Figure 15: Uncurated 512 $\times$ 512 FCDM-XL samples. Classifier-free guidance scale = 4.0

Panel of Figure 16 — Figure 16: Uncurated 256 $\times$ 256 FCDM-XL samples. Classifier-free guidance scale = 4.0

Panel of Figure 17 — Figure 17: Uncurated 256 $\times$ 256 FCDM-XL samples. Classifier-free guidance scale = 4.0

Panel of Figure 18 — Figure 18: Uncurated 256 $\times$ 256 FCDM-XL samples. Classifier-free guidance scale = 4.0

Panel of Figure 19 — Figure 19: Uncurated 256 $\times$ 256 FCDM-XL samples. Classifier-free guidance scale = 4.0

Panel of Figure 20 — Figure 20: Uncurated 256 $\times$ 256 FCDM-XL samples. Classifier-free guidance scale = 4.0

Panel of Figure 21 — Figure 21: Uncurated 256 $\times$ 256 FCDM-XL samples. Classifier-free guidance scale = 4.0

Diffusion Models

Diffusion models learn to reverse gradual data corruption, enabling flexible generation, editing, forecasting, and inference.

See all 1,169 papers

Rethinking the design space of Vanilla Pixel Diffusion Transformers
Oct 16, 2026Soumik Mukhopadhyay, Long Mai, Abhinav Shrivastava +1Diffusion-Based Image Generation Diffusion Models
University of Maryland, College Park · Adobe Research
Structured Residual Connectivity Matters for Diffusion Transformers
Sep 27, 2026Yuhe Liu, Xinyin Ma, Gongfan Fang +2Diffusion Transformer Diffusion-Based Image Generation
National University of Singapore · Shanghai Jiao Tong University
Improved Distributional Diffusion Models
Sep 29, 2026Tommaso Martorella, Alexandre Galashov, Felix Krause +4Few-Step Diffusion Sampling Diffusion-Based Image Generation
Munich Center for Machine Learning (MCML) · Google DeepMind

Oct 16, 2026J/K move · Enter open · S save

Rethinking the design space of Vanilla Pixel Diffusion Transformers

Soumik Mukhopadhyay, Long Mai, Abhinav Shrivastava, Aniruddha Mahapatra

University of Maryland, College Park · Adobe Research

Diffusion-Based Image Generation Diffusion Models Diffusion Transformer Pixel-Space Diffusion Models

Open paper AskPDF arXiv

Pixel-space diffusion models have recently re-emerged as a compelling alternative to latent diffusion, avoiding the lossy latent bottleneck and separately trained autoencoder. JiT shows that a vanilla diffusion transformer can generate directly in pixel space by replacing the commonly used velocity prediction (

\boldsymbol{u}

-prediction) with clean image prediction (

\boldsymbol{x_0}

-prediction). In this work, we revisit this choice and introduce MFlow, an adaptive parameterization that learns to interpolate between

\boldsymbol{x_0}

- and

\boldsymbol{u}

-prediction across principal directions of the data, learned end-to-end with diffusion training. We find that the preferred parameterization varies systematically with the data representation and model capacity -- notably, increasing model capacity shifts preference toward velocity prediction. We evaluate MFlow across ImageNet, UCF-101 video, and large-scale text-to-image generation, demonstrating competitive or improved generative performance across diverse settings. In addition, we find that vanilla pixel diffusion transformers, irrespective of parameterization, retain a persistent high-frequency deficit relative to real images. To address this, we introduce ConvRefiner, a lightweight convolutional module trained post-hoc with an adversarial loss while keeping the diffusion backbone frozen. With only a small amount of additional training, ConvRefiner restores the high-frequency statistics of generated images while preserving FID. Project webpage: https://soumik-kanad.github.io/mflow-diffusion/.

Abstract

Diffusion Models

Explore similar work

Rethinking the design space of Vanilla Pixel Diffusion Transformers

Structured Residual Connectivity Matters for Diffusion Transformers

Improved Distributional Diffusion Models