Setting-Matched and Semantics-Scaled Benchmarking of One-Step Generative Models Against Multistep Diffusion and Flow Models

Advaith Ravishankar; Serena Liu; Mingyang Wang; Todd Zhou; Jeffrey Zhou; Arnav Sharma; Ziling Hu; Léopold Das

cs.CVMar 15, 2026

Setting-Matched and Semantics-Scaled Benchmarking of One-Step Generative Models Against Multistep Diffusion and Flow Models

Authors: Advaith Ravishankar, Serena Liu, Mingyang Wang, Todd Zhou, Jeffrey Zhou, Arnav Sharma, Ziling Hu, Léopold Das, +6 more

Organizations: Harvard AI and Robotics Lab, Harvard University

Abstract

State-of-the-art text-to-image models produce high-quality images, but inference remains expensive as generation requires several sequential ODE or denoising steps. Native one-step models aim to reduce this cost by mapping noise to an image in a single step, yet fair comparisons to multi-step systems are difficult because studies use mismatched sampling steps and different classifier-free guidance (CFG) settings, where CFG can shift FID, Inception Score, and CLIP-based alignment in opposing directions. It is also unclear how well one-step models scale to multi-step inference, and there is limited standardized out-of-distribution evaluation for label-ID-conditioned generators beyond ImageNet. To address this, we benchmark eight models spanning one-step flows (MeanFlow, Improved MeanFlow, SoFlow), multi-step baselines (RAE, Scale-RAE), and established systems (SiT, Stable Diffusion 3.5, FLUX.1) under a class-conditional protocol on ImageNet validation, ImageNetV2, and reLAIONet, our new proofread out-of-distribution dataset aligned to ImageNet label IDs. Using FID, Inception Score, CLIP Score, and Pick Score, we show that FID-focused model development and CFG selection can be misleading in few-step regimes, where guidance changes can improve FID while degrading text-image alignment and human preference signals, worsening visual quality. To make these tradeoffs explicit, we introduce CLIP-scaled and PickScore-scaled variants of FID (csFID, psFID) and Inception Score (csIS, psIS) to serve as a diagnostic for semantically aligned image generation.

Figures & tables

Figure 1: Benchmark overview. Left: Motivation for fair one-step vs. multi-step comparison. Middle: setting-matched and semantics-scaled protocol over models, datasets, steps, CFG, and metrics. Right: reLAIONet preserves the ImageNet label-ID interface under distribution shift, while qualitative examples show that low FID hides visual artifacts, motivating csFID, psFID, csIS, and psIS.

Figure 2: Within model family comparison for native-one step models. Images generated with classes Golden Retriever, GoldFish, Groom, Scuba Diver and Sports Car for Meanflow and iMF. Both iMF and Meanflow show clear distortions even with low FID, indicating that FID minimization alone does not imply better image generation

Figure 3: Samples for Golden Retriever, GoldFish, Groom, Scuba Diver, and Sports Car at 25 steps and CFG 7. One-step models show low-FID models but exhibit artifacts in faces, underwater scenes, and fine details, while higher-FID SOTA models (SD3.5 and Flux) produce coherent samples. This motivates reporting scaled metrics such as csFID, psFID, csIS, and psIS alongside FID.

Figure 4: Effect of step size and CFG on MeanFlow performance. Heatmaps show FID, CLIP Score, Inception Score, and PickScore across different step sizes and CFG values

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Comparison of generation quality across all models. Images generated with classes lighthouse, church, butterfly, pizza and cat. Models settings are 25 steps and CFG 7

Model	Reported Inference CFG	Reported Steps	Model Weights
MeanFlow	1.0	1	DiT-B/4
iMF	6.0	1	DiT-XL/2
SoFlow	2.0	1	DiT-X/2-cond
RAE	1.42	50	DiT-XL, DINOv2-B
Scale-RAE	1.42	50	Qwen1.5B_DiT2.4B
SiT (ODE)	1.5	Dynamic	SiT-XL/2

Appendix

Table 3: Models evaluated, the checkpoints used, and their reported guidance/step settings from original releases. The original release CFG values for SD3.5-Large and FLUX.1-dev were taken based on model cards from HuggingFace, and the original step count for FLUX.1-dev was also taken from the model card

Figure 6: Within model family comparison for FLUX.1-dev and Stable Diffusion 3.5 Large

Figure 7: Within model family comparison for Scale RAE and RAE

Figure 8: Within model family comparison for SoFlow and SiT ODE

Figure 9: Correlation matrix between base and scaled metrics. csFID and psFID remain strongly correlated with FID while incorporating negative dependence on CLIP and PickScore, whereas csIS and psIS track IS while preserving semantic and preference alignment. This shows that the scaled metrics retain the behavior of the base metrics while making alignment and preference tradeoffs explicit.

CFG Scale	Steps	FID $\downarrow$	IS Mean $\uparrow$	CLIP Score $\uparrow$	PickScore $\uparrow$	csFID $\downarrow$	psFID $\downarrow$	csIS $\uparrow$	psIS $\uparrow$
1	1	10.01	150.48	29.36	19.49	116.12	263.52	12.97	5.72
	5	7.74	198.98	29.84	19.75	86.92	198.43	17.72	7.76
	10	7.69	202.32	29.88	19.77	86.13	196.75	18.06	7.91
	15	7.70	203.87	29.89	19.78	86.19	196.81	18.21	7.98
	20	7.72	204.65	29.90	19.78	86.35	197.32	18.30	8.01
	25	7.74	205.41	29.90	19.79	86.58	197.63	18.36	8.04

Appendix

Table 9: Ablations for Meanflow across step sizes of 1, 5, 10, 15, 20, 25 and CFG scales of 1, 3, 6, 7, 9, 12, 15 on ImageNet, including CLIP and Pick Scaled FID/IS

CFG Scale	Steps	FID $\downarrow$	IS Mean $\uparrow$	CLIP Score $\uparrow$	PickScore $\uparrow$	csFID $\downarrow$	psFID $\downarrow$	csIS $\uparrow$	psIS $\uparrow$
1	1	9.93	123.37	21.34	18.27	218.05	297.49	5.62	4.12
	5	6.08	141.77	28.89	19.72	72.85	156.35	11.83	5.51
	10	5.89	142.62	28.90	19.73	70.52	151.31	11.91	5.55
	15	5.84	143.20	28.92	19.73	69.83	150.02	11.98	5.57
	20	5.82	143.56	28.93	19.74	69.54	149.36	12.02	5.59
	25	5.78	144.24	28.93	19.74	69.06	148.33	12.07	5.62

Appendix

Table 10: Ablations for Improved Meanflow across step sizes of 1, 5, 10, 15, 20, 25 and CFG scales of 1, 3, 6, 7, 9, 12, 15 on ImageNet, including CLIP and Pick Scaled FID/IS

Figure 10: Effect of step size and CFG on Improved MeanFlow performance. Heatmaps show FID, CLIP Score, Inception Score, and PickScore across different step sizes and CFG values

Figure 11: reLAIONet class distribution. Left: Ranked per-class image counts after manual filtering, showing that reLAIONet covers 757 ImageNet classes with 25,252 total images (x-axis is a reindexing and does not reflect class labels). Right: Histogram of class sizes, with mean 33 and median 37 images per class. Most classes contain multiple validated samples, while long-tail classes retain fewer images after proofreading.

Parameter	Value	Description
$\tau$	$0.82$	CLIP cosine similarity threshold
$N$	$1000$	Top- $N$ classes selected
$K$	$70$	Max images per class
$B$	$2{,}048$	CLIP encoding batch size
CLIP model	ViT-B/32	OpenAI pre-trained weights
Lemma filter	unique only	Ambiguous lemmas excluded

Appendix

Table 11: LAIONet construction hyperparameters.

Hyperparameter	Value
Model	nyu-visionx/Scale-RAE-Qwen1.5B_DiT2.4B
Decoder encoder	google/siglip2-so400m-patch14-224
Max new tokens	512
Conditioning	Text prompt: “Can you generate a photo of a {class}?”
Image size (metrics)	$256\times 256$
Resize filter	LANCZOS

Appendix

Table 12: Scale-RAE Evaluation Setup

Hyperparameter	Value
Model	SiT-XL/2
VAE	stabilityai/sd-vae-ft-mse
Image size	$256\times 256$
Latent size	$32\times 32$
Sampler mode	ODE (Euler)
Generation batch size	16

Appendix

Table 13: SiT Evaluation Setup

Hyperparameter	Value
Model	stabilityai/stable-diffusion-3.5-large
Conditioning	Text prompt: “a photo of a {class}”
Metric resolution	$256\times 256$ (LANCZOS resize)
Generation batch size	4
Seed	42

Appendix

Table 14: SD 3.5 Large Evaluation Setup

Hyperparameter	Value
Model	black-forest-labs/FLUX.1-dev
Conditioning	Text prompt: “a photo of a {class}”
Metric resolution	$256\times 256$ (LANCZOS resize)
Generation batch size	1
Seed	42

Appendix

Table 15: FLUX.1 [dev] Evaluation Setup

Hyperparameter	Value
Model variant	DiT_B_4
VAE	stabilityai/sd-vae-ft-mse
Image size (metrics)	$256\times 256$
Latent size	$32\times 32\times 4$
Sampler	ODE (Euler, fori_loop )
Per-device batch size	1

Appendix

Table 16: MeanFlow Evaluation Setup

Hyperparameter	Value
Model variant	MiT_XL_2
VAE	stabilityai/sd-vae-ft-mse
Image size (metrics)	$256\times 256$
Latent size	$32\times 32\times 4$
Sampler	ODE with CFG interval
Per-device batch size	1

Appendix

Table 17: Improved MeanFlow Evaluation Setup

Hyperparameter	Value
Model variant	DiT-XL-2-cond
VAE	stabilityai/sd-vae-ft-mse
Image size (metrics)	$256\times 256$
Latent size	$32\times 32\times 4$
Per-device batch size	32
Seed	42

Appendix

Table 18: SoFlow Evaluation Setup

Hyperparameter	Value
Architecture	Two-stage: Stage 1 RAE decoder + Stage 2 diffusion
Latent size	$(768,16,16)$ (default)
Image size (metrics)	$256\times 256$
Per-device batch size	25
Seed	42

Appendix

Table 19: RAE Evaluation Setup

Explore similar work

Are We Overthinking One-Step Generation?
Oct 14, 2026Woobin Park, Jaeyeon Kim, Yilun Du +2One-Step Generative Modeling Conditional Generative Modeling
Seoul National University · Harvard University
Self-Adversarial One Step Generation via Condition Shifting
Apr 14, 2026Deyuan Liu, Peng Sun, Yansen Han +3One-Step Generative Modeling T2I Diffusion Models
Westlake University · Zhejiang University · Shanghai Innovation Institute +1
SubFlow: Sub-mode Conditioned Flow Matching for Diverse One-Step Generation
Apr 14, 2026Yexiong Lin, Jia Shi, Shanshan Ye +3Flow Matching Flow-Based Generative Modeling
Sydney AI Centre, The University of Sydney, Sydney, NSW, Australia · School of Artificial Intelligence, Xidian University, Xi’an, Shaanxi, China · Australian Artificial Intelligence Institute, University of Technology Sydney, Sydney, NSW, Australia +1

Oct 14, 2026J/K move · Enter open · S save

Are We Overthinking One-Step Generation?

Woobin Park, Jaeyeon Kim, Yilun Du, Sitan Chen+1

Seoul National University · Harvard University

One-Step Generative Modeling Conditional Generative Modeling T2I Generation

Open paper AskPDF arXiv

Diffusion and flow matching are now dominant frameworks for generative modeling in continuous domains, but they typically require multiple network evaluations at inference. This has motivated extensive research on one-step generative modeling primarily built on diffusion and flow, though existing approaches often rely on complex training machinery. In this work, we introduce Priming Models, a simple two-stage framework for training one-step generators. Motivated by the difficulty of distributional training from scratch, Priming Models are first trained on a reconstruction objective, e.g., pixel-space denoising at a fixed noise level, before undergoing feature-space distributional post-training. Our central scientific finding is that this simple reconstruction stage is already enough to entirely alleviate the difficulty of distributional training. Empirically, across class-conditional image generation, text-to-image generation, and world modeling, Priming Models significantly advance the Pareto frontier of training compute vs. generation quality. On ImageNet at

256 \times 256

resolution, we achieve a state-of-the-art FID of

1.19

among one-step generators using approximately one-tenth the training compute of prior methods with comparable FID. We open-source our codebase at https://github.com/JaeyeonKim01/priming.

Abstract

Explore similar work

Are We Overthinking One-Step Generation?

Self-Adversarial One Step Generation via Condition Shifting

SubFlow: Sub-mode Conditioned Flow Matching for Diverse One-Step Generation