Test-Time Scaling of Text-to-Image Flow Matching Models with Noise-Rejuvenated Sequential Monte Carlo

Wenyi Lian; Zheng Zhao; Thomas B. Schön

cs.CVOct 18, 2026

Test-Time Scaling of Text-to-Image Flow Matching Models with Noise-Rejuvenated Sequential Monte Carlo

Authors: Wenyi Lian, Zheng Zhao, Thomas B. Schön

Organizations: Department of Information Technology Uppsala University, Sweden · Department of Computer and Information Science Linköping University, Sweden

Abstract

Test-time scaling of diffusion models with sequential Monte Carlo (SMC) has become a promising strategy for sampling under user-specified rewards, such as human preference scores. However, the standard SMC framework typically relies on stochastic proposals and therefore does not directly transfer to flow matching models, in which inference is governed by deterministic ODEs. In this work, we introduce noise-rejuvenated SMC, an inference-time scaling method tailored for text-to-image flow matching models. Our key idea is to perturb each resampled particle by changing its source noise and constructing a new intermediate state using its current endpoint estimate. This moves the particle to another state that remains representable under the flow-matching interpolation form, enabling particle exploration without converting the sampler to an SDE. We further introduce a look-ahead mechanism to refine resampled particles before weighting. Experiments on Stable Diffusion 3, FLUX.1-dev, and Z-Image demonstrate consistent improvements in reward-guided generation, while ablation studies validate the contributions of noise rejuvenation and look-ahead refinement. Code is available at https://github.com/shermanlian/NR-SMC.

Figures & tables

Figure 1: Qualitative comparison between vanilla sampling and our method on text-to-image generation using Stable Diffusion 3 (SD3) ( Esser et al., 2024 ) , FLUX.1 ( Labs et al., 2025 ) , and Z-Image ( Team, 2025 ) . Our method steers generation toward higher reward objectives, producing outputs that better align with the input prompts without additional training or gradient computation.

Figure 2: Illustration of flow matching using standard SMC (left) and our noise-rejuvenated SMC (right) during inference-time sampling. For clarity, we only visualize two particles (purple). When directly applying SMC to flow matching models, resampling leads to particle collapse, producing identical trajectories and final outputs (left). In contrast, noise rejuvenation injects controlled stochasticity during resampling, inducing occasional “jumps” that allow particles to escape collapsed trajectories and explore alternative paths throughout the sampling process (right).

Backbone	Method	GenEval	ImageReward	HPSv2	Correct Image
SD3	Vanilla	0.718	1.081	0.302	0.707
	FK-Diffusion	0.636	1.080	0.293	0.624
	Ours	0.748	1.268	0.309	0.738
FLUX.1-dev	Vanilla	0.663	0.986	0.306	0.646
	FK-Diffusion	0.467	0.953	0.288	0.450
	Ours	0.673	1.218	0.309	0.656

Table 1: Text-to-image comparison on the GenEval ( Ghosh et al., 2023 ) benchmark across different pretrained backbones. We report GenEval, ImageReward ( Xu et al., 2023 ) , HPSv2 ( Wu et al., 2023b ) , and Correct Image ( Ghosh et al., 2023 ) . Bold numbers indicate the best result under each backbone.

	Objects
Model	Overall	Single	Two	Counting	Colors	Position	Color Attr.
minDALL-E	0.23	0.73	0.11	0.12	0.37	0.02	0.01
SD v1.5	0.43	0.97	0.38	0.35	0.76	0.04	0.06
PixArt-alpha	0.48	0.98	0.50	0.44	0.80	0.08	0.07
SD v2.1	0.50	0.98	0.51	0.44	0.85	0.07	0.17
DALL-E 2	0.52	0.94	0.66	0.49	0.77	0.10	0.19

Table 2: Task-wise accuracy results on the GenEval ( Ghosh et al., 2023 ) benchmark.

Method	Color			Shape			Texture
Method	BVQA	ImageReward	HPSv2	BVQA	ImageReward	HPSv2	BVQA	ImageReward	HPSv2
SD3	0.812	1.443	0.314	0.590	0.977	0.282	0.734	1.110	0.280
+ Ours	0.820	1.563	0.318	0.641	1.280	0.290	0.760	1.327	0.288
FLUX.1-dev	0.768	1.342	0.312	0.497	0.691	0.274	0.635	0.856	0.286
+ Ours	0.915	1.465	0.310	0.536	0.975	0.280	0.650	1.164	0.290
Z-Image	0.821	0.925	0.291	0.606	0.665	0.256	0.761	0.580	0.254

Table 3: Comparison on T2I-CompBench ( Huang et al., 2023 ) benchmark.

Figure 3: Qualitative comparison across different inference pipelines. Results are shown for Stable Diffusion 3 (SD3) ( Esser et al., 2024 ) and FLUX.1-dev ( Labs et al., 2025 ) backbones under the same prompts. For each selected prompt, the displayed samples are uncurated outputs from the corresponding inference pipeline. Our method generates samples that more closely follow the prompts in terms of semantic attributes and composition, while preserving visual quality.

Figure 4: Qualitative comparison across different inference pipelines on T2I-CompBench. Our method generates samples that more closely follow the prompts in terms of semantic attributes and composition, while preserving visual quality.

Model	Method	NFE $\downarrow$	Avg. Time (s) $\downarrow$	GenEval $\uparrow$	ImageReward $\uparrow$	HPSv2 $\uparrow$
SD3	Ours(w/o LA)	200	35.10	0.742	1.261	0.308
SD3	Ours(w/ LA)	228	46.25	0.748	1.268	0.309
FLUX.1-dev	Ours(w/o LA)	200	95.96	0.667	1.192	0.307
FLUX.1-dev	Ours(w/ LA)	228	109.32	0.673	1.218	0.309

Table 5: Results of our method with and without look-ahead on SD3 and FLUX baseline models. We further report the number of function evaluations (NFE) and the average inference time per prompt for reference. Average inference times were measured on a single NVIDIA A100 GPU.

	With Look-ahead			Without Look-ahead
Number of Particles	GenEval	ImageReward	HPSv2	GenEval	ImageReward	HPSv2
$N=\phantom{0}2$	0.720	1.079	0.304	0.706	1.063	0.303
$N=\phantom{0}4$	0.748	1.268	0.309	0.742	1.261	0.308
$N=\phantom{0}8$	0.761	1.322	0.311	0.740	1.323	0.309
$N=16$	0.766	1.347	0.311	0.763	1.348	0.311

Table 6: Ablation experiment on the number of particles

N

with and without look-ahead.

Method	$N=2$	$N=4$	$N=8$
Best-of- $N$	1.227	1.369	1.438
Ours	1.255	1.374	1.448

Table 7: Comparison with Best-of-

N

Method	LPIPS $\uparrow$	IR $\uparrow$
SD3	0.688	1.081
SMC w/o rejuvenation	0.154	1.282
SMC w/ rejuvenation	0.367	1.268
FLUX	0.605	0.986
SMC w/o rejuvenation	0.187	1.243
SMC w/ rejuvenation	0.364	1.218

Table 8: Diversity analysis on GenEval.

Method	ImageReward $\uparrow$	HPSv2 $\uparrow$	NFE $\downarrow$	Wall-clock time (s) $\downarrow$	Peak GPU memory (GiB) $\downarrow$
Vanilla-50	1.0773	0.3023	50	3.73	20.46
Vanilla-200	1.0792	0.3032	200	13.84	20.46
Best-of- $4$	1.3696	0.3097	200	14.59	26.11
FK-Diffusion	1.2536	0.2965	200	18.53	27.78
Diffusion Tree Sampling	1.2344	0.2951	218	16.40	25.16
Ours (w/o Look-ahead)	1.3734	0.3103	200	18.18	27.78

Table 9: Compute and performance comparison on SD3 with GenEval. Quality metrics are reported as means over three random seeds. Ours (NFE-matched) reduces the number of sampling steps from 50 to 43, resulting in 200 NFEs and matching the NFE of Best-of-4, FK-Diffusion, and the other 200-NFE variants. Wall-clock time and peak GPU memory were measured on a single NVIDIA GH200 GPU.

Method	ImageReward $\uparrow$	HPSv2 $\uparrow$	NFE $\downarrow$	Wall-clock time (s) $\downarrow$	Peak GPU memory (GiB) $\downarrow$
Vanilla-50	1.1841	0.2925	50	3.75	20.46
Vanilla-200	1.2028	0.2937	200	13.98	20.46
Best-of- $4$	1.4826	0.2979	200	14.70	26.11
FK-Diffusion	1.2691	0.2853	200	17.82	27.78
Diffusion Tree Sampling	1.2608	0.2848	218	16.43	25.16
Ours (w/o Look-ahead)	1.4944	0.2995	200	17.54	27.78

Table 10: Compute and performance comparison on SD3 with T2I-CompBench. Quality metrics are reported as means over three random seeds. Ours (NFE-matched) reduces the number of sampling steps from 50 to 43, resulting in 200 NFEs and matching the NFE of Best-of-4, FK-Diffusion, and the other 200-NFE variants. Wall-clock time and peak GPU memory were measured on a single NVIDIA GH200 GPU.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Schedule	$f$	$t_{\text{start}}$	$t_{\text{end}}$	GenEval $\uparrow$	ImageReward $\uparrow$	HPSv2 $\uparrow$
Early	5	0	30	0.737	1.262 $\pm$ 0.108	0.309 $\pm$ 0.007
Mid (default)	5	5	35	0.748	1.268 $\pm$ 0.106	0.309 $\pm$ 0.007
Late	5	20	50	0.721	1.261 $\pm$ 0.123	0.305 $\pm$ 0.009
Uniform window	5	10	40	0.740	1.288 $\pm$ 0.102	0.308 $\pm$ 0.007
Full window	5	0	50	0.737	1.265 $\pm$ 0.113	0.309 $\pm$ 0.007
Narrow, dense	3	10	28	0.748	1.276 $\pm$ 0.106	0.308 $\pm$ 0.008

Appendix

Table 11: Results under different resampling stages in our noise rejuvenation inference framework. We vary the resampling time window (early, mid, late, uniform, and full) to analyze how resampling at different stages affects generation quality.

Number of Particles	NFE	Avg. Time (s)	GenEval $\uparrow$	ImageReward $\uparrow$	HPSv2 $\uparrow$
$N=\phantom{0}2$	114	0 18.11	0.720	1.079	0.304
$N=\phantom{0}4$	228	0 46.25	0.748	1.268	0.309
$N=\phantom{0}8$	456	0 68.14	0.761	1.322	0.311
$N=16$	912	135.59	0.766	1.347	0.311

Appendix

Table 12: Computational cost with increasing particle numbers on SD3. We report the number of function evaluations (NFE), average inference time per prompt, and generation performance. Average inference times were measured on a single NVIDIA A100 GPU.

Method	GenEval	ImageReward	HPSv2
SD3	0.718	1.081	0.302
Local perturb ( $\sigma_{t}$ = 0.1)	0.718	1.081	0.302
Local perturb ( $\sigma_{t}$ = 0.5)	0.729	1.267	0.310
Local perturb ( $\sigma_{t}$ = 1.0)	0.589	0.794	0.279
Noise rejuvenation (Ours)	0.748	1.268	0.309

Appendix

Table 13: Comparison of initial-noise rejuvenation with direct local perturbation in the SD3 inference pipeline.

Panel of Figure 10 — Figure 10: Additional qualitative results comparing the vanilla SD3 inference pipeline, FK-Diffusion, and our method on SD3. Please zoom in for details.

Panel of Figure 12 — Figure 12: Challenging cases for our method on SD3.

Panel of Figure 13 — Figure 13: Challenging cases for our method on SD3.

Panel of Figure 14 — Figure 14: Additional examples comparing SMC on SD3 with and without noise rejuvenation. Without noise rejuvenation, resampled particles collapse to nearly identical outputs, whereas noise rejuvenation restores particle diversity. Please zoom in for details.

Inference-Time Scaling

Inference-time scaling studies how extra computation during deployment can improve model performance through search, sampling, verification, or adaptive reasoning.

See all 115 papers

Multi-Resolution Flow Matching: Training-Free Diffusion Acceleration via Staged Sampling
Jul 2, 2026Xingyu Zheng, Xianglong Liu, Yifu Ding +4Diffusion Model Acceleration Diffusion Model Inference Acceleration
State Key Laboratory of Complex & Critical Software Environment, Beihang University · School of Computer Science and Engineering, Beihang University · Nanyang Technological University +5
Reflective Flow Sampling Enhancement
Mar 6, 2026Zikai Zhou, Muyao Wang, Shitong Shao +4T2I Diffusion Models Flow-Based Generative Modeling
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China. · The University of Tokyo, Japan. · Microsoft, Beijing. +1
Flow-OPD: On-Policy Distillation for Flow Matching Models
May 8, 2026Zhen Fang, Wenxuan Huang, Yu Zeng +8Diffusion Model Alignment Flow Matching
University of Science and Technology of China · University of California, Los Angeles · 3The Chinese University of Hong Kong +1

Jul 2, 2026J/K move · Enter open · S save

Multi-Resolution Flow Matching: Training-Free Diffusion Acceleration via Staged Sampling

Xingyu Zheng, Xianglong Liu, Yifu Ding, Weilun Feng+3

State Key Laboratory of Complex & Critical Software Environment, Beihang University · School of Computer Science and Engineering, Beihang University · Nanyang Technological University+5

Diffusion Model Acceleration Diffusion Model Inference Acceleration T2I Diffusion Models

Open paper AskPDF arXiv

Hardware-agnostic strategies for accelerating text-to-image diffusion, such as timestep distillation and feature caching, can reduce inference time without custom kernels or system-level optimization. Among them, multi-resolution generation strategies have recently received broad attention, attaining more than 5x speedup without any training. However, the design of performing upsampling in the latent space, together with the selective modification of partial regions, causes these methods to exhibit noticeable blurring or artifacts. To this end, we propose MrFlow, a training-free multi-resolution acceleration strategy for pretrained flow-matching models built upon a staged low-to-high-resolution pipeline. MrFlow first rapidly generates the main structure at low resolution, then performs super-resolution in the pixel space using a lightweight pretrained GAN-based model, subsequently injects low-strength noise to enable high-frequency resampling, and finally refines the details at high resolution. Quantitative and qualitative results on FLUX.1-dev and Qwen-Image show that MrFlow exploits the quadratic token reduction and reduced step requirement of low-resolution sampling to achieve 10x end-to-end acceleration while keeping OneIG within a 1% gap relative to that before acceleration, significantly surpassing other training-free acceleration strategies, and requiring no training or runtime dynamic identification whatsoever. MrFlow can further be directly combined orthogonally with pre-trained timestep distillation strategies, achieving even higher generation acceleration of up to 25x.

Abstract

Inference-Time Scaling

Explore similar work

Multi-Resolution Flow Matching: Training-Free Diffusion Acceleration via Staged Sampling

Reflective Flow Sampling Enhancement

Flow-OPD: On-Policy Distillation for Flow Matching Models