VISTA: Test-Time Compositional Alignment for Visual Autoregressive Generation

Hossein Shahabadi; Niki Sepasian; Mahdieh Soleymani Baghshah

cs.CVAug 23, 2026

VISTA: Test-Time Compositional Alignment for Visual Autoregressive Generation

Authors: Hossein Shahabadi, Niki Sepasian, Mahdieh Soleymani Baghshah

Organizations: Department of Computer Engineering Sharif University of Technology

Abstract

Visual autoregressive (VAR) models have emerged as a fast, high-quality alternative to diffusion for text-to-image generation, but like diffusion models they exhibit persistent compositional failures, producing images that violate the attribute bindings and spatial relations specified in the prompt. While a rich line of test-time alignment methods has developed for diffusion, no comparable approach exists for next-scale VAR generation, whose stateful, discrete, multi-resolution sampling process makes existing techniques inapplicable. We close this gap with VISTA (Visual Autoregressive Semantic Test-time Alignment), the first gradient-based test-time alignment framework for next-scale autoregressive image generation. Built on Infinity, VISTA intervenes directly in the generation process, optimizing intermediate representations through the frozen transformer to steer visual predictions toward compositional constraints, without modifying model parameters or requiring additional training. VISTA introduces the mechanisms needed to make such optimization stable across scales, together with an extensible objective space that any differentiable constraint on cross-attention can plug into. Across two benchmarks and two model scales, VISTA improves every targeted compositional category, raising the mean targeted score by nearly 20% on a 2B backbone and almost 6% on an 8B backbone, with the largest gains on spatial relations. Image quality is preserved: an independent preference model VISTA never optimizes scores its outputs nearly 20% higher. Notably, the 2B model with VISTA surpasses a backbone four times its size, indicating that a substantial part of the compositional gap between model scales is recoverable at test time.

Figures & tables

Figure 1: VISTA on “a white crow and a black flower”. Generation proceeds coarse to fine, each scale conditioned on all coarser ones; VISTA steers only the outlined coarse scales, scoring the model’s own cross-attention with a differentiable compositional loss and taking normalized gradient steps on that scale’s input $\mathbf{z}_{s}$ until the objectives converge or the step budget is spent. Right: flower and crow attention at Res 3 and Res 4 (the $6{\times}6$ and $8{\times}8$ grids), with and without steering. Without it the maps stay collapsed onto one another and a second bird appears in place of the flower; with it they separate and remain separated through the unsteered fine scales. $\mathcal{L}_{\mathrm{entity}}$ is shown but not instantiated (Sec. 3.4 ).

Model	Color	Texture	Shape	2D	3D	Cmplx	Num.	Non-S.	Avg. 6	Avg. 8
SDXL	0.593	0.519	0.466	0.215	0.341	0.319	0.504	0.311	0.409	0.409
PixArt- $\alpha$	0.407	0.444	0.367	0.202	0.350	0.324	0.506	0.308	0.349	0.363
Flux-Dev	0.746	0.644	0.482	0.273	0.393	0.363	0.613	0.309	0.483	0.478
Flux-Schnell	0.725	0.683	0.559	0.271	0.373	0.364	0.604	0.312	0.496	0.486
Infinity-2B	0.749	0.632	0.475	0.234	0.405	0.384	0.572	0.310	0.480	0.470
+ TTS-VAR ( $N{=}1$ )	0.773	0.690	0.543	0.269	0.424	0.391	0.596	0.311	0.515	0.500

Table 1: T2I-CompBench, four seeds. Prior-system rows are from 19 and use no cache compression; every Infinity row does (Sec. 4.1 ). VISTA applies

\mathcal{L}_{\mathrm{bind}}+\mathcal{L}_{\mathrm{2D}}+\mathcal{L}_{\mathrm{depth}}

over three scales; TTS-VAR is the authors’ implementation at

N{=}1

under our setup. Numeracy and Non-Spatial (shaded) are untargeted and unchanged by construction; Avg. 6 covers the six targeted categories, Avg. 8 all eight. Bold : best per group; underline : best overall.

Model	Color	Attr.	Pos.	Single	Two	Count	Overall
Infinity-2B	0.827	0.560	0.250	0.994	0.808	0.591	0.672
+ VISTA	0.809	0.645	0.745	0.994	0.808	0.591	0.765
Infinity-8B	0.886	0.765	0.578	1.000	0.937	0.778	0.824
+ VISTA	0.907	0.788	0.825	1.000	0.937	0.778	0.873

Table 2: GenEval, four seeds. All rows run with ScaleKV and VISTA rows use the configuration of Table 1 . Shaded categories are untargeted and unchanged by construction; Overall averages all six. Bold : best per group; underline : best overall.

Steered scales	Targ. 2B $\uparrow$	Targ. 8B $\uparrow$	ImageReward $\uparrow$	Aesthetic $\uparrow$	Time (2B) $\downarrow$
None	0.480	0.560	1.025	5.734	3.09 s
1	0.546	0.579	1.234	5.683	4.58 s
2	0.567	0.587	1.234	5.615	5.28 s
3	0.575	0.593	1.230	5.609	5.96 s
4	0.579	0.586	1.204	5.608	13.13 s

Table 3: The steered-scale knob, measured three ways: targeted-category average on T2I-CompBench, fidelity on

600

paired prompts by two judges VISTA never optimizes (one conditioned on the prompt, one not), and per-image time on Infinity-2B. Scales are added coarse-to-fine from

4{\times}4

; the shaded row is our default.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 2 — Figure 2: Base and VISTA generations across spatial relations, attribute binding, and their combination. Purple marks spatial relation phrases; teal marks attribute–noun binding pairs.

Model	Seed	Col.	Tex.	Shp.	2D	3D	Cpx.	Avg.
2B	0	0.826	0.750	0.568	0.454	0.455	0.395	0.575
	1	0.823	0.755	0.579	0.432	0.447	0.400	0.573
	2	0.848	0.746	0.576	0.439	0.453	0.399	0.577
	std	0.014	0.005	0.006	0.011	0.004	0.003	0.002
8B	0	0.853	0.802	0.652	0.425	0.426	0.405	0.594
	1	0.863	0.792	0.664	0.417	0.424	0.402	0.594

Appendix

Table 4: Per-seed T2I-CompBench scores for the default configuration (three steered scales,

N_{\mathrm{max}}{=}5

), over four seeds. Avg. is the mean over the six targeted categories.

$N_{\mathrm{max}}=1$
Scales	B	S	D	B+S	B+D	S+D	B+S+D
1	3.60	3.36	3.38	3.47	3.47	3.45	3.50
2	3.47	3.48	3.48	3.61	3.60	3.56	3.63
3	3.57	3.56	3.56	3.74	3.74	3.68	3.77
4	4.95	4.94	4.95	5.82	5.81	5.51	5.93
$N_{\mathrm{max}}=5$

Appendix

Table 5: Per-image generation time (seconds) on Infinity-2B. B, S, and D denote the binding, spatial, and depth objectives. Baseline is

3.09

Term	Eq.	Taken from prior work	Ours
$\mathcal{L}_{\mathrm{bind}}$	13	Multi-positive contrastive form of CONFORM 14	Application to VAR scales
$\mathcal{L}_{\mathrm{2D}}$	14	PoS objective of PSG 16 , unchanged	Sharpening; inversion detail
$\mathcal{L}_{\mathrm{sep}}$	8	Segregation principle of A-STAR 1 only	Extent-based functional form
$\mathcal{L}_{\mathrm{bc}}$	9	—	Entire term
$\mathcal{L}_{\mathrm{entity}}$	—	—	Not instantiated
$\mathcal{L}_{\mathrm{ns}}$	—	—	Not instantiated

Appendix

Table 6: Provenance of each objective term. “Principle only” means we adopt the motivating observation but not the loss function, which we state explicitly in the relevant paragraph of Sec. 3.4 .

Metric	Base	1	2	3 (default)	4	$\Delta$ at default	$p$
		VISTA, steered scales
ImageReward	1.025	1.234	1.234	1.230	1.204	$+20.1\%$	$3.6\times 10^{-7}$
CLIPScore	0.321	0.325	0.326	0.325	0.324	$+1.1\%$	$6.1\times 10^{-4}$
Aesthetic (v1)	6.056	6.022	6.023	5.976	5.957	$-1.3\%$	$1.6\times 10^{-3}$
Aesthetic (v2)	5.734	5.683	5.615	5.609	5.608	$-2.2\%$	$2.0\times 10^{-10}$

Appendix

Table 7: Image fidelity under three judges VISTA does not optimize, as a function of the number of steered scales. All columns are paired with the same base generations.

p

values are for the default configuration.

$\mathcal{L}_{\mathrm{bind}}$	Color	Texture	Shape	Mean	$\Delta$
CONFORM (ours)	0.830	0.764	0.570	0.721	$+17.1\%$
SynGen divergence	0.823	0.745	0.565	0.711	$+15.5\%$
KL	0.830	0.740	0.556	0.709	$+15.1\%$
Focal	0.769	0.735	0.579	0.694	$+12.8\%$
Divide-and-Bind	0.778	0.633	0.459	0.623	$+1.2\%$
JSD	0.730	0.642	0.475	0.616	$+0.0\%$

Appendix

Table 8: Binding objectives on the three categories where only

\mathcal{L}_{\mathrm{bind}}

is active. All rows share the same optimization machinery and the same planar and depth terms, which are inactive here.

\Delta

is on the mean.

Table 11

Explore similar work

DiverseVAR: Balancing Diversity and Quality of Next-Scale Visual Autoregressive Models
Nov 26, 2025Mingue Park, Prin Phunyaphibarn, Phillip Y. Lee +1Image Generation Autoregressive Image Generation
KAIST
Visual Implicit Autoregressive Modeling
May 2, 2026Pengfei Jiang, Jixiang Luo, Luxi Lin +2Visual Autoregressive Models Autoregressive Image Generation
Institute of Artificial Intelligence (TeleAI), China Telecom · Xiamen University
VPG: Visual Prefix Guidance for Autoregressive Image and Video Generation
May 28, 2026Xinyao Liao, Qiyuan He, Yicong Li +4Visual Autoregressive Models Autoregressive Image Generation
National University of Singapore · Huazhong University of Science & Technology

Nov 26, 2025J/K move · Enter open · S save

DiverseVAR: Balancing Diversity and Quality of Next-Scale Visual Autoregressive Models

Mingue Park, Prin Phunyaphibarn, Phillip Y. Lee, Minhyuk Sung

KAIST

Image Generation Autoregressive Image Generation

Open paper AskPDF arXiv

We introduce DiverseVAR, a framework that enhances the diversity of text-conditioned visual autoregressive models (VAR) at test time without requiring retraining, fine-tuning, or substantial computational overhead. While VAR models have recently emerged as strong competitors to diffusion and flow models for image generation, they suffer from a critical limitation in diversity, often producing nearly identical images even for simple prompts. This issue has largely gone unnoticed amid the predominant focus on image quality. We address this limitation at test time in two stages. First, inspired by diversity enhancement techniques in diffusion models, we propose injecting noise into the text embedding. This introduces a trade-off between diversity and image quality: as diversity increases, the image quality sharply declines. To preserve quality, we propose scale-travel: a novel latent refinement technique inspired by time-travel strategies in diffusion models. Specifically, we use a multi-scale autoencoder to extract coarse-scale tokens that enable us to resume generation at intermediate stages. Extensive experiments show that combining text-embedding noise injection with our scale-travel refinement significantly enhances diversity while minimizing image-quality degradation, achieving a new Pareto frontier in the diversity-quality trade-off. Project page: https://diverse-var.github.io

Abstract

Explore similar work

DiverseVAR: Balancing Diversity and Quality of Next-Scale Visual Autoregressive Models

Visual Implicit Autoregressive Modeling

VPG: Visual Prefix Guidance for Autoregressive Image and Video Generation