SCALE: Improving Guidance-Free Visual Generation via Self-Contrastive Alignment for Likelihood Estimation

Jiwan Hur; Dong-Jae Lee; Dongyeun Lee; Gyojin Han; Junmo Kim

cs.CVOct 16, 2026

SCALE: Improving Guidance-Free Visual Generation via Self-Contrastive Alignment for Likelihood Estimation

Authors: Jiwan Hur, Dong-Jae Lee, Dongyeun Lee, Gyojin Han, Junmo Kim

Organizations: KAIST, South Korea

Abstract

Classifier-free guidance (CFG) is essential for improving sample quality in visual generation, but it incurs substantial sampling overhead. Existing guidance-free approaches halve this cost by approximating the CFG with a single model inference, but their performance is bounded by the CFG and inherits its limitations. In this paper, we propose self-contrastive guidance (SCG), which contrasts the model's prediction on real and self-generated samples to obtain a direct, model-based signal that pushes generation toward the true data manifold. Integrating SCG into the standard likelihood objective via reparameterization yields Self-Contrastive Alignment for Likelihood Estimation (SCALE), a guidance-free training framework that produces SCG-enhanced predictions in a single forward pass. Across class-conditional and text-to-image generation on diffusion and autoregressive models, SCALE consistently outperforms CFG-mimicking guidance-free baselines and improves further when combined with DPO, demonstrating that the self-contrastive signal yields gains complementary to both CFG and preference-based fine-tuning.

Figures & tables

Panel of Figure 1 — Figure 1: Comparison between CFG and our self-contrastive guidance (SCG). (a) CFG amplifies the conditional signal to improve sample quality, often reducing sample diversity by overemphasizing the condition. (b) SCG derives a model-based signal that measures the discrepancy between real and self-generated samples, pushing generation toward the true data manifold. (c) SCALE integrates CFG and SCG into a single model, achieving superior guidance-free generation performance.

Figure 2: Toy experiment on guidance-free generation with diffusion models. (a) Ground-truth samples from a three-class 2D Swiss-roll distribution. (b) Unguided conditional sampling scatters outliers across low-density regions. (c, d) Guidance-free methods that mimic CFG [ 41 , 6 ] generate samples that drift away from class boundaries (red boxes) and off the data manifold (green boxes). (e) SCALE sharpens the distribution while preserving both the class boundary and the data manifold. The effective guidance scale is fixed at 5.0 across all guidance-free methods.

Figure 3: Illustration of SCALE algorithm without generation step $t$ for brevity. Left: SCALE training derives a model-based signal that measures the discrepancy between real ( ${\bm{x}}$ ) and self-generated ( $\tilde{\bm{x}}$ ) samples, learning to approximate the self-contrastive guidance with condition $\alpha$ . Right: At sampling time, lowering $\alpha$ yields high-quality samples by amplifying the implicit guidance signal.

Table 4

Model Types	Methods	FID-50k $\downarrow$	IS $\uparrow$	Precision $\uparrow$	Recall $\uparrow$
Diffusion Models	ADM [ 11 ]	10.94	101.1	0.69	0.63
	LDM-4 [ 49 ]	10.56	103.4	0.72	0.62
	CDM [ 19 ]	4.88	158.7	–	–
	DiT-XL/2 [ 44 ]	9.62	121.5	0.67	0.67
	$\hskip 9.24994pt+$ Guid. Distill [ 41 ]	2.11	–	–	–
	$\hskip 9.24994pt+$ GFT [ 6 ]	1.99	266.6	0.81	0.60

Table 2: Comparisons of guidance-free class-conditional generation on ImageNet

256\times 256

. Bold for the best and underline for the second-best metrics for each model type.

Methods	PickScore $\uparrow$	CLIP $\uparrow$	HPSv2.1 $\uparrow$	FID-30k $\downarrow$	Aesthetic $\uparrow$	Img. Reward $\uparrow$
SD 1.5 [ 49 ]	18.90	0.252	0.187	22.47	4.97	-1.30
$\hskip 9.24994pt+$ Guid. Distill [ 41 ]	20.56	0.314	0.252	15.12	5.33	0.06
$\hskip 9.24994pt+$ GFT [ 6 ]	20.50	0.321	0.248	12.50	5.27	0.02
$\hskip 9.24994pt+$ SCALE ( $\alpha=0.8$ )	20.58	0.324	0.253	12.83	5.30	0.18
$\hskip 9.24994pt+$ SCALE ( $\alpha=0.25$ )	20.73	0.324	0.261	16.52	5.33	0.30

Table 3: Guidance-free generation results on text-to-image generation using SD1.5.

Panel of Figure 4 — Figure 4: Performance comparison of SCALE and other guidance-free baselines using SD 1.5 [ 49 ] .

Figure 5: Guidance-free generation results of base SD 1.5 [ 49 ] , GFT, and SCALE, with and without DPO post-training [ 60 ] . Prompts are sampled from the Pick-a-Pic test set [ 29 ] and PartiPrompts [ 67 ] .

Models	Methods	Guidance	NFEs $\downarrow$	FID $\downarrow$	IS $\uparrow$	Precision $\uparrow$	Recall $\uparrow$
DiT-XL/2 [ 44 ]	CFG † [ 20 ]	Required	50 $\times$ 2	2.17	243.7	0.80	0.60
	SAG † [ 21 ]	Required	50 $\times$ 3	2.17	241.2	0.80	0.59
	G++ † [ 27 ]	Required	50 $\times$ 3	2.14	240.0	0.76	0.63
	SCALE (Ours)	Free	50	1.87	268.0	0.81	0.60
LlamaGen-L [ 54 ]	CFG [ 20 ]	Required	256 $\times$ 2	3.07	256.0	0.83	0.52
LlamaGen-L [ 54 ]	SCALE (Ours)	Free	256	2.58	253.0	0.82	0.55

Table 4: Comparison with guided sampling methods on the ImageNet

256\times 256

. † Re-evaluated using 50-step DPM-Solver++ [ 40 ] for a fair comparison. See Appendix G.3 for more details.

Panel of Figure 6 — Figure 6: Effect of varying $\alpha$ and $\beta$ of SCALE using DiT-XL/2 [ 44 ] at sampling time.

Table 11

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Example trial from the blind A/B human preference study. Left (A): GFT [ 6 ] . Right (B): SCALE. Participants selected the image without these labels.

Figure 8: Effect of interpolation choice. (Left) Linear vs. patch-wise interpolation in the VQ token space (LlamaGen tokenizer [ 54 ] ). Linear interpolation collapses to noise at intermediate $\lambda$ , while PatchMix transitions smoothly between ${\bm{x}}_{1}$ and ${\bm{x}}_{2}$ . (Right) LPIPS distance between real samples and PatchMix-interpolated samples on DiT-XL/2 [ 44 ] : lower $\lambda$ keeps the interpolated sample closer to the real-data manifold, providing a stable anchor for self-contrastive guidance.

Methods	Guid.	NFEs $\downarrow$	FID $\downarrow$	IS $\uparrow$	Precision $\uparrow$	Recall $\uparrow$
DiT-XL/2 [ 44 ]	✗	250	9.62	121.5	0.67	0.67
$\hskip 9.24994pt+$ CFG [ 20 ]	✓	250 $\times$ 2	2.27	278.24	0.83	0.57
$\hskip 9.24994pt+$ SAG [ 21 ]	✓	250 $\times$ 3	2.11	–	–	–
$\hskip 9.24994pt+$ G++ [ 27 ]	✓	250 $\times$ 3	1.83	281.53	0.78	0.64
$\hskip 9.24994pt+$ SCALE (Ours)	✗	250	2.10	288.08	0.81	0.59

Appendix

Table 8: Comparison with guided sampling methods on the ImageNet generation at

256\times 256

Figure 9: Visualization of SCALE by varying $\alpha$ and $\beta$ using DiT-XL/2 [ 44 ] .

Figure 10: Visualization of SCALE by varying $\alpha$ and $\beta$ using LlamaGen-L [ 54 ] .

Figure 11: Guidance-free generation results of base SD 1.5 [ 49 ] , GFT, and SCALE (ours), with and without DPO post-training [ 60 ] . Prompts are sampled from the Pick-a-Pic test set [ 29 ] and PartiPrompts [ 67 ] .

Figure 12: Visualization of SCALE by varying $\alpha$ using SD 1.5 [ 49 ] with DPO post-training [ 60 ] .

Image Generation

Image generation studies models that create or edit images from text, examples, layouts, and other structured controls.

See all 463 papers

VAGS: Velocity Adaptive Guidance Scale for Image Editing and Generation
May 15, 2026Yan Luo, Ahmadou Aidara, Jingyi Lu +3Classifier-Free Guidance Diffusion-Based Image Generation
Harvard AI and Robotics Lab, Harvard University · School of Computing and Data Science, The University of Hong Kong · Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University
Guiding a Diffusion Model by Swapping Its Tokens
Apr 9, 2026Weijia Zhang, Yuehao Liu, Shanyan Guan +4Diffusion Model Guidance Diffusion Guidance
MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University · vivo Mobile Communication Co., Ltd.
Guidance Matters: Rethinking the Evaluation Pitfall for Text-to-Image Generation
Feb 26, 2026Dian Xie, Shitong Shao, Lichen Bai +5Diffusion Model Guidance T2I Generation Evaluation
The Hong Kong University of Science and Technology (Guangzhou) · Harbin Institute of Technology (Shenzhen) · Cogniser Information Technology

May 15, 2026J/K move · Enter open · S save

VAGS: Velocity Adaptive Guidance Scale for Image Editing and Generation

Yan Luo, Ahmadou Aidara, Jingyi Lu, Jeremy Moebel+2

Harvard AI and Robotics Lab, Harvard University · School of Computing and Data Science, The University of Hong Kong · Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University

Classifier-Free Guidance Diffusion-Based Image Generation Diffusion-Based Image Editing

Open paper AskPDF arXiv

Classifier-free guidance (CFG) is the primary control over how strongly text semantics move a flow-based sampler, yet standard practice holds its scale fixed across the entire ODE trajectory. This is a fundamental mismatch: early steps are noise-dominated and carry weak semantic signal, while late steps commit image structure and demand stronger directional commitment; more critically, the value of any guidance strength depends on whether the guided velocity is consistent with the model's current dynamics or working against them. We propose Velocity-Adaptive Guidance Scale (VAGS), a training-free replacement that multiplies the nominal scale by a bounded factor combining a temporal signal-level term with the cosine similarity between task-relevant velocity fields. For inversion-free editing, VAGS measures the alignment between source- and target-guided velocities, so edit strength at each step reflects local compatibility between preservation and transformation. For generation, VAGS-Gen uses the alignment between unconditional and conditional velocities as the analogous signal. Neither variant requires fine-tuning, auxiliary networks, or extra forward passes, and fixed CFG is recovered as a special case. On PIE-Bench and DIV2K for editing, and COCO17, CUB-200, and Flickr30K for generation, VAGS consistently improves structural fidelity and generation quality over fixed CFG and recent training-free guidance variants. The code is publicly available at https://github.com/Harvard-AI-and-Robotics-Lab/Velocity_Adaptive_Guidance_Scale.

Abstract

Image Generation

Explore similar work

VAGS: Velocity Adaptive Guidance Scale for Image Editing and Generation

Guiding a Diffusion Model by Swapping Its Tokens

Guidance Matters: Rethinking the Evaluation Pitfall for Text-to-Image Generation