SwiftExplorer: Training-free Diffusion Model Alignment with Swift Diversity Exploration

Renye Yan; Jikang Cheng; You Wu; Bojin Huang; Wei Peng; Zongwei Wang; Ling Liang; Yimao Cai

cs.LGSep 6, 2026

SwiftExplorer: Training-free Diffusion Model Alignment with Swift Diversity Exploration

Authors: Renye Yan, Jikang Cheng, You Wu, Bojin Huang, Wei Peng, Zongwei Wang, Ling Liang, Yimao Cai

Organizations: Peking University Beijing, China · Nanjing University Nanjing, China · Huazhong Agricultural University Wuhan, China · Stanford University Stanford, California, USA

Abstract

Diffusion models have general generative abilities but struggle to align with specific objectives. Fine-tuning can improve alignment, yet its training cost is often prohibitive. This led to training-free methods that apply objective-guided terms in sampling to bias the generation distribution toward designated regions, e.g., high-reward areas. However, these methods face two issues: (1) the strong directional bias narrows the pretrained distribution and generation diversity, and (2) indiscriminate constant guidance fails to prune redundant signals, hurting both quality and efficiency. To address the above challenges, we propose SwiftExplorer, a plugin that mitigates distribution collapse caused by excessive diversity loss and reduces compute costs. First, we adopt an Inheritance-Restart exploration mechanism to avoid early convergence, while exploration also increases the likelihood of high-reward trajectories. Additionally, it balances diversity and fidelity, adding diversity without causing a distribution over-shift. Second, our Quality-Efficiency arbitration mechanism improves guidance by removing incorrect signals, and it reduces computation by dynamically stopping generation when completeness and marginal reward gain are optimal. In an extensive number of experiments and different types of evaluation metrics, the proposed SwiftExplorer achieves excellent performance on all metrics, including preference, fidelity, diversity, and richness.

Figures & tables

Figure 1. Feature visualization via t-SNE . The features are extracted by a general self-supervised decoder ( i.e. , DINOv2) from images generated by different methods. With the guidance targets of both Ours and DyMO having achieved comparable scores, the distribution results indicate that existing SOTA guidance methods ( i.e. , DyMO) yield more compact clustering, narrowing the generated distribution, which in turn reduces the diversity of the generated data. This observation is substantiated by the Silhouette Coefficient ( Rousseeuw, 1987 ) , a clustering evaluation metric, which demonstrates its convergence and limited diversity. In contrast, our method maintains a broader generative distribution while achieving comparable target reward scores (see Tab. 1 ).

Figure 2. Overall framework of the proposed plug-and-play SwiftExplorer .

Figure 3. Visual impression on the advanced flow-matching-based models . Our results exhibit finer details, more diverse colors, and improved aesthetic style.

Method	Preference				Fidelity			Diversity			Richness			#Top2
Method	PS $\uparrow$	AES $\uparrow$	IR $\uparrow$	HPS $\uparrow$	FID $\downarrow$	CLIP $\uparrow$	iFS $\uparrow$	LPIPS $\uparrow$	IS $\uparrow$	TCE $\uparrow$	BRI $\downarrow$	NIQE $\downarrow$	SE $\uparrow$	#Top2
SDv15	20.48	5.412	0.181	0.262	-	0.243	-	0.654	23.79	38.05	18.66	5.401	11.26	3
Diff-DPO	20.87	5.551	0.443	0.271	109.1	0.244	0.795	0.639	22.77	39.20	15.12	4.463	11.02	1
Diff-KTO	20.83	5.585	0.599	0.272	101.3	0.240	0.801	0.634	22.70	39.12	26.25	4.361	11.22	0
SPO	20.76	5.613	0.282	0.218	78.76	0.241	0.855	0.649	23.18	39.32	25.67	4.103	11.09	1
DRaFT	22.52	5.697	0.779	0.271	84.47	0.231	0.856	0.572	20.10	37.85	35.64	5.336	10.43	2

Table 1. Main comparisons with SoTA . All metrics are obtained with SDv15 as backbone, Pick-a-pic as prompt set, and PickScore as reward for guidance. Orange denotes gradient-based training methods. Green denotes reinforcement-learning-based training methods. Blue denotes training-free methods .

Figure 4. Wall-clock runtime comparison under consistent experimental conditions.

Method	IR	PS	CLIP	LPIPS	TCE	NIQE $\downarrow$
SDv15	0.231	21.57	0.249	0.643	39.62	4.656
Diff-DPO	0.571	22.03	0.245	0.619	37.97	4.471
SPO	0.300	21.58	0.249	0.595	39.37	4.172
DDPO	0.678	22.26	0.245	0.607	38.96	3.973
DNO	0.446	21.79	0.246	0.575	39.14	4.523
DAS	0.684	23.09	0.246	0.653	39.31	4.133

Table 2. Results on HPSv2 dataset.

Method	SD3					SD3.5
Method	PS	CLIP	LPIPS	NIQE $\downarrow$	Top1	PS	CLIP	LPIPS	NIQE $\downarrow$	Top1
Base	21.62	0.238	0.645	4.312	1	21.90	0.238	0.621	4.553	1
+Gdt	22.17	0.239	0.636	4.304	0	22.95	0.235	0.616	4.174	0
+ Ours	22.57	0.241	0.614	2.990	3	23.93	0.238	0.625	3.794	4

Table 3. Comparison results with advanced flow matching models . Gdt represents the baseline that directly applies the reward gradient to the latent code during inference. Ours is designed based on Gdt with the proposed plugin components.

Figure 5. Performance-Computation Trade-off.

Figure 6. User study for plug-and-play effectiveness.

Figure 7. Plug-and-Play effectiveness . While maintaining the Fidelity and Preference, our method significantly enhances Diversity, Richness, and Speed across all cases, such as the “background and pizza filling diversity” in the first row, and “improved Van Gogh style” in the second row.

Var	Dive		Fide	Effi		Qual	IR	PS	CLIP	LPIPS	TCE	NIQE $\downarrow$	Time $\downarrow$	#Top2
Var	ESV	M-ESV	L2	I&R	EarS	GuiS	IR	PS	CLIP	LPIPS	TCE	NIQE $\downarrow$	Time $\downarrow$	#Top2
Base							0.709	22.79	0.239	0.627	39.29	4.454	44.92	0
V1	✓						0.739	22.12	0.236	0.659	39.30	2.897	46.31	1
V2	✓		✓				0.734	22.46	0.240	0.636	39.30	3.379	51.24	0
V3		✓	✓				0.757	22.75	0.240	0.641	39.32	2.939	67.08	0
V4		✓	✓	✓			0.746	22.73	0.241	0.640	39.29	3.039	48.56	0

Table 4. Ablation study . For each component, ESV and M-ESV represent exploration supplement variable and with Monte Carlo sampling. I&R and EarS denote Inheritance-Restart technique and Early Stopping. GuiS denotes Guidance Screening.

Figure 8. Generative diversity examples with the same prompt and different seeds.

Figure 9. Diversity-fidelity adaptive trade-off (Ours, red dotted lines) with different fixed intensities.

Figure 10. Showcase of generated samples. The images from our model achieve state-of-the-art performance.

Figure 11. Results with Aesthetic Score as the optimization objective.

Name	Description	Value
$\eta$	eta parameter for the DDIM sampler	1.0
$w$	classifier-free guidance weight	5.0
$m_{dno}$	Iteration Number for DNO	20
$m_{dymo}$	Iteration Number for DyMO	adaptive
$n_{das}$	Particle number for DAS	4
$mp$	mixed precision	fp16

Table 5. Hyper-parameters and metrics in our experiment. Notably, only

n

and

\delta_{r}

are newly introduced by our method ; all other hyper-parameters follow DDIM or the baseline methods.

Avg. Value	Base	HPSv2		Pick-a-Pic
Avg. Value	Base	SDv15	SD-XL	SDv15	SD-XL
Restart Count	50	4.37	8.55	7.32	10.59
Guiding Step	50	37.35	44.21	40.67	44.30

Table 6. Results of Restart Count and Early Stop across HPSv2 and Pick-a-Pic with SDv15 and SD-XL backbones.

Figure 12. The visual comparisons of different methods on SDv15. For photorealistic prompts from HPSv2, our results contain richer details while maintaining image-prompt alignment. For surreal prompts from Pick-a-Pic, we exhibit much more diverse patterns and colors.

Method	Preference	Fidelity	Diversity	Richness
DNO	0.88	0.52	0.27	1.27
DAS	0.96	0.81	0.30	1.36
DyMO	0.62	0.86	0.28	1.15
Ours	0.65	0.79	0.28	1.41

Table 7. CoV (%) comparison of methods across four evaluation metrics.

Method	Total	SimpleAnimal	Pick-a-Pic	HPSv2	GenEval
SD15	50	37.3	40.9	39.8	38.1
SDXL	30	24.7	24.9	25.1	24.6

Table 8. Mean step numbers predicted by the control network across different datasets. The control network is trained on the simple animal and directly generalizes to other datasets.

Figure 13. Feature visualization comparison with DAS.

Figure 14. The proposed ESV facilitates exploration during the denoising process. Left : ESV enables our method to identify a broader range of patterns and adjust the trajectory toward the “teddy bear lantern” during intermediate stages. Right : Our method exhibits greater posture variation throughout denoising. While this variation is not directly responsible for the superior final generation, it reflects an increased level of exploration.

Table 23

Method	IR	PS	CLIP	LPIPS	TCE	NIQE $\downarrow$
SD-XL	0.715	21.45	0.236	0.675	40.18	4.553
Diff-DPO	1.010	22.05	0.236	0.651	39.45	4.440
SPO	1.179	22.81	0.232	0.556	40.42	3.900
SDv35	1.193	21.90	0.238	0.621	39.42	4.873
DNO	0.924	22.58	0.238	0.581	39.34	4.783
DAS	1.171	23.07	0.237	0.594	39.36	5.365

Table 11. Results with advanced models.

Image	Prompt
Fig. 1 , Row 1, Col 1	ballet dancer, insanely detailed, photorealistic, 8k, perfect composition, volumetric lighting, natural complexion, award-winning professional photography, taken with Canon EOS 5D Mark IV, 85mm, mindblowing, masterpiece
Fig. 1 , Row 1, Col 2	A fluffy bunny as Rapunzel, with long golden ears flowing down from a tall enchanted tower, glowing lanterns in the night sky, warm fairytale atmosphere
Fig. 1 , Row 1, Col 3	A dog in sportswear lifting tiny dumbbells at the gym, determined expression, humorous fitness illustration
Fig. 1 , Row 1, Col 4	A boy superhero landing on the ground in classic “hero pose,” debris and glowing sparks flying around, comic action shot
Fig. 1 , Row 1, Col 5	A phoenix rising up from ashes
Fig. 1 , Row 2, Col 1	A young girl standing on a rooftop, blowing dandelions that transform into glowing comets, shooting across the night sky, dreamy fantasy artwork

Table 12. Detailed prompts used for generated images in Fig. 10 - Part 1

Image	Prompt
Fig. 9 , Row 1	Heart shaped balloon
Fig. 9 , Row 2	An apple on a table
Fig. 9 , Row 3	A castle in the sky, clouds, sunset, explosion
Fig. 9 , Row 4	Harry potter as a cat, pixar style, octane render, HD, high-detail

Table 13. Detailed prompts used for generated images in Fig. 9

Image	Prompt
Fig. 1 , Row 3, Col 1	A warrior standing at the edge of a glowing crater, surrounded by swirling cosmic energy, their silhouette outlined against the birth of a new star, ultimate epic fantasy art
Fig. 1 , Row 3, Col 2	A boy lying on the grass in a field, listening to music with glowing headphones, fireflies surrounding him
Fig. 1 , Row 3, Col 3	A little girl painting a rainbow bridge from the classroom window into the sky, playful magical fairytale art, hopeful and inspiring
Fig. 1 , Row 3, Col 4	A group of playful penguins throwing glowing snowballs at each other, each snowball turning into sparkling stars when it explodes, magical fairytale scene
Fig. 1 , Row 3, Col 5	Giant rubber duck floating in the ocean with a small island on its back, surrounded by tropical palm trees and crystal clear water, bright and sunny day, calm seas, vivid colors, cinematic lighting, high detail
Fig. 1 , Row 3, Col 6	A cozy library built inside an ancient oak tree, warm lights glowing through round windows, whimsical fairytale healing atmosphere

Table 14. Detailed prompts used for generated images in Fig. 10 - Part 2

Image	Prompt
Fig. 11 , Row 1	transylvania castle on hiltop and dusk bats and scifi
Fig. 11 , Row 2	fantasy character portrait digital painting, anime style, detailed with beautiful emotive lighting suggesting personality, and background that suggests character backstory
Fig. 11 , Row 3	Women in Saree playing Holi
Fig. 11 , Row 4	a house burning at night

Table 15. Detailed prompts used for generated images in Fig. 11

Image	Prompt
Fig. 7 , Row 1	a slice of pizza floating through space with stars in the background
Fig. 7 , Row 2	A Great Dane dog in the style of Vincent Van Gogh

Table 16. Detailed prompts used for generated images in Fig. 7

Diffusion Model Alignment

Diffusion model alignment steers generated content and internal representations toward prompts, human preferences, safety goals, and task constraints.

See all 185 papers

SWIFT: Self-guided Weak-to-strong dIFfusion models via Target extrapolation
Oct 19, 2026Yanjin He, Cuifeng Shen, Wanzhou Lei +5Diffusion Models Diffusion Model Guidance
University of Michigan, Ann Arbor · Alibaba Group · University of California, Berkeley +1
Accelerated Likelihood Maximization for Diffusion-based Versatile Content Generation
Jun 30, 2026Hyunsoo Lee, Inwoo Hwang, Young Min KimDiffusion Sampling Diffusion Model Acceleration
ECE, Seoul National University · INMC & IPAI, Seoul National University
Free Lunch for Pass@ $k$ ? Low Cost Diverse Sampling for Diffusion Language Models
Mar 5, 2026Sean Lamont, Christian Walder, Paul Montague +2Diversity-Aware Sampling Output Diversity in Language Models
Australian National University · Defence Science and Technology Group · Google DeepMind +1

Oct 19, 2026J/K move · Enter open · S save

SWIFT: Self-guided Weak-to-strong dIFfusion models via Target extrapolation

Yanjin He, Cuifeng Shen, Wanzhou Lei, Maohua Li+4

University of Michigan, Ann Arbor · Alibaba Group · University of California, Berkeley+1

Diffusion Models Diffusion Model Guidance

Open paper AskPDF arXiv

Training high-quality diffusion models is computationally expensive, and inference-time guidance often adds further sampling cost. In this work, we introduce Swift, a simple yet effective self-guided training method that uses a frozen weak denoiser to improve a stronger model without adding inference overhead. For each noisy input, Swift extrapolates the clean regression target away from the weak denoiser's prediction, amplifying components that remain underfit. The weak guide can be an early training checkpoint, requires no backpropagation during main-model training, and is discarded afterward. The resulting model retains the original sampler and supports both conditional and unconditional generation without label dropout or an unconditional branch. We explain this mechanism through a spectral analysis of finite-time population gradient flow in an RKHS. With an early-stopped weak guide, we prove that a nonempty range of extrapolation strengths yields strictly lower posterior-mean error than matched standard training, despite shifting the regression target away from the posterior mean. Experiments on CIFAR-10 and ImageNet-256 demonstrate improvements across model scales and representations. In one ImageNet-256 configuration, Swift surpasses the baseline's best FID using roughly half the total training compute, including guide preparation and training overhead. On RAEv2, it achieves FID 1.15 without inference-time guidance and 0.99 with guidance. These results establish target extrapolation as an effective approach to weak-to-strong diffusion training with efficient single-model inference.

Abstract

Diffusion Model Alignment

Explore similar work

SWIFT: Self-guided Weak-to-strong dIFfusion models via Target extrapolation

Accelerated Likelihood Maximization for Diffusion-based Versatile Content Generation

Free Lunch for Pass@kkk? Low Cost Diverse Sampling for Diffusion Language Models

Free Lunch for Pass@ $k$ ? Low Cost Diverse Sampling for Diffusion Language Models