Visual autoregressive (VAR) models have emerged as a fast, high-quality alternative to diffusion for text-to-image generation, but like diffusion models they exhibit persistent compositional failures, producing images that violate the attribute bindings and spatial relations specified in the prompt. While a rich line of test-time alignment methods has developed for diffusion, no comparable approach exists for next-scale VAR generation, whose stateful, discrete, multi-resolution sampling process makes existing techniques inapplicable. We close this gap with VISTA (Visual Autoregressive Semantic Test-time Alignment), the first gradient-based test-time alignment framework for next-scale autoregressive image generation. Built on Infinity, VISTA intervenes directly in the generation process, optimizing intermediate representations through the frozen transformer to steer visual predictions toward compositional constraints, without modifying model parameters or requiring additional training. VISTA introduces the mechanisms needed to make such optimization stable across scales, together with an extensible objective space that any differentiable constraint on cross-attention can plug into. Across two benchmarks and two model scales, VISTA improves every targeted compositional category, raising the mean targeted score by nearly 20% on a 2B backbone and almost 6% on an 8B backbone, with the largest gains on spatial relations. Image quality is preserved: an independent preference model VISTA never optimizes scores its outputs nearly 20% higher. Notably, the 2B model with VISTA surpasses a backbone four times its size, indicating that a substantial part of the compositional gap between model scales is recoverable at test time.
Figures & tables
Figure 1: VISTA on “a white crow and a black flower”. Generation proceeds coarse to fine, each scale conditioned on all coarser ones; VISTA steers only the outlined coarse scales, scoring the model’s own cross-attention with a differentiable compositional loss and taking normalized gradient steps on that scale’s input zs until the objectives converge or the step budget is spent. Right: flower and crow attention at Res 3 and Res 4 (the 6×6 and 8×8 grids), with and without steering. Without it the maps stay collapsed onto one another and a second bird appears in place of the flower; with it they separate and remain separated through the unsteered fine scales. Lentity is shown but not instantiated (Sec. 3.4 ).
Model
Color
Texture
Shape
2D
3D
Cmplx
Num.
Non-S.
Avg. 6
Avg. 8
SDXL
0.593
0.519
0.466
0.215
0.341
0.319
0.504
0.311
0.409
0.409
PixArt- α
0.407
0.444
0.367
0.202
0.350
0.324
0.506
0.308
0.349
0.363
Flux-Dev
0.746
0.644
0.482
0.273
0.393
0.363
0.613
0.309
0.483
0.478
Flux-Schnell
0.725
0.683
0.559
0.271
0.373
0.364
0.604
0.312
0.496
0.486
Infinity-2B
0.749
0.632
0.475
0.234
0.405
0.384
0.572
0.310
0.480
0.470
+ TTS-VAR ( N=1 )
0.773
0.690
0.543
0.269
0.424
0.391
0.596
0.311
0.515
0.500
Table 1: T2I-CompBench, four seeds. Prior-system rows are from 19 and use no cache compression; every Infinity row does (Sec. 4.1 ). VISTA applies Lbind+L2D+Ldepth over three scales; TTS-VAR is the authors’ implementation at N=1 under our setup. Numeracy and Non-Spatial (shaded) are untargeted and unchanged by construction; Avg. 6 covers the six targeted categories, Avg. 8 all eight. Bold : best per group; underline : best overall.
Model
Color
Attr.
Pos.
Single
Two
Count
Overall
Infinity-2B
0.827
0.560
0.250
0.994
0.808
0.591
0.672
+ VISTA
0.809
0.645
0.745
0.994
0.808
0.591
0.765
Infinity-8B
0.886
0.765
0.578
1.000
0.937
0.778
0.824
+ VISTA
0.907
0.788
0.825
1.000
0.937
0.778
0.873
Table 2: GenEval, four seeds. All rows run with ScaleKV and VISTA rows use the configuration of Table 1 . Shaded categories are untargeted and unchanged by construction; Overall averages all six. Bold : best per group; underline : best overall.
Steered scales
Targ. 2B ↑
Targ. 8B ↑
ImageReward ↑
Aesthetic ↑
Time (2B) ↓
None
0.480
0.560
1.025
5.734
3.09 s
1
0.546
0.579
1.234
5.683
4.58 s
2
0.567
0.587
1.234
5.615
5.28 s
3
0.575
0.593
1.230
5.609
5.96 s
4
0.579
0.586
1.204
5.608
13.13 s
Table 3: The steered-scale knob, measured three ways: targeted-category average on T2I-CompBench, fidelity on 600 paired prompts by two judges VISTA never optimizes (one conditioned on the prompt, one not), and per-image time on Infinity-2B. Scales are added coarse-to-fine from 4×4 ; the shaded row is our default.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 2: Base and VISTA generations across spatial relations, attribute binding, and their combination. Purple marks spatial relation phrases; teal marks attribute–noun binding pairs.
Model
Seed
Col.
Tex.
Shp.
2D
3D
Cpx.
Avg.
2B
0
0.826
0.750
0.568
0.454
0.455
0.395
0.575
1
0.823
0.755
0.579
0.432
0.447
0.400
0.573
2
0.848
0.746
0.576
0.439
0.453
0.399
0.577
std
0.014
0.005
0.006
0.011
0.004
0.003
0.002
8B
0
0.853
0.802
0.652
0.425
0.426
0.405
0.594
1
0.863
0.792
0.664
0.417
0.424
0.402
0.594
Appendix
Table 4: Per-seed T2I-CompBench scores for the default configuration (three steered scales, Nmax=5 ), over four seeds. Avg. is the mean over the six targeted categories.
Scales
B
S
D
B+S
B+D
S+D
B+S+D
Nmax=1
1
3.60
3.36
3.38
3.47
3.47
3.45
3.50
2
3.47
3.48
3.48
3.61
3.60
3.56
3.63
3
3.57
3.56
3.56
3.74
3.74
3.68
3.77
4
4.95
4.94
4.95
5.82
5.81
5.51
5.93
Nmax=5
Appendix
Table 5: Per-image generation time (seconds) on Infinity-2B. B, S, and D denote the binding, spatial, and depth objectives. Baseline is 3.09 s.
Term
Eq.
Taken from prior work
Ours
Lbind
13
Multi-positive contrastive form of CONFORM 14
Application to VAR scales
L2D
14
PoS objective of PSG 16 , unchanged
Sharpening; inversion detail
Lsep
8
Segregation principle of A-STAR 1 only
Extent-based functional form
Lbc
9
—
Entire term
Lentity
—
—
Not instantiated
Lns
—
—
Not instantiated
Appendix
Table 6: Provenance of each objective term. “Principle only” means we adopt the motivating observation but not the loss function, which we state explicitly in the relevant paragraph of Sec. 3.4 .
VISTA, steered scales
Metric
Base
1
2
3 (default)
4
Δ at default
p
ImageReward
1.025
1.234
1.234
1.230
1.204
+20.1%
3.6×10−7
CLIPScore
0.321
0.325
0.326
0.325
0.324
+1.1%
6.1×10−4
Aesthetic (v1)
6.056
6.022
6.023
5.976
5.957
−1.3%
1.6×10−3
Aesthetic (v2)
5.734
5.683
5.615
5.609
5.608
−2.2%
2.0×10−10
Appendix
Table 7: Image fidelity under three judges VISTA does not optimize, as a function of the number of steered scales. All columns are paired with the same base generations. p values are for the default configuration.
Lbind
Color
Texture
Shape
Mean
Δ
CONFORM (ours)
0.830
0.764
0.570
0.721
+17.1%
SynGen divergence
0.823
0.745
0.565
0.711
+15.5%
KL
0.830
0.740
0.556
0.709
+15.1%
Focal
0.769
0.735
0.579
0.694
+12.8%
Divide-and-Bind
0.778
0.633
0.459
0.623
+1.2%
JSD
0.730
0.642
0.475
0.616
+0.0%
Appendix
Table 8: Binding objectives on the three categories where only Lbind is active. All rows share the same optimization machinery and the same planar and depth terms, which are inactive here. Δ is on the mean.
We introduce DiverseVAR, a framework that enhances the diversity of text-conditioned visual autoregressive models (VAR) at test time without requiring retraining, fine-tuning, or substantial computational overhead. While VAR models have recently emerged as strong competitors to diffusion and flow models for image generation, they suffer from a critical limitation in diversity, often producing nearly identical images even for simple prompts. This issue has largely gone unnoticed amid the predominant focus on image quality. We address this limitation at test time in two stages. First, inspired by diversity enhancement techniques in diffusion models, we propose injecting noise into the text embedding. This introduces a trade-off between diversity and image quality: as diversity increases, the image quality sharply declines. To preserve quality, we propose scale-travel: a novel latent refinement technique inspired by time-travel strategies in diffusion models. Specifically, we use a multi-scale autoencoder to extract coarse-scale tokens that enable us to resume generation at intermediate stages. Extensive experiments show that combining text-embedding noise injection with our scale-travel refinement significantly enhances diversity while minimizing image-quality degradation, achieving a new Pareto frontier in the diversity-quality trade-off. Project page: https://diverse-var.github.io
Mingue Park, Prin Phunyaphibarn, Phillip Y. Lee +1
Visual Autoregressive Modeling (VAR) based on next-scale prediction achieves strong generation quality, but their explicit deep stacks fix the amount of computation per scale and inflate memory at high resolutions. We introduce Visual Implicit Autoregressive Modeling (VIAR), a next-scale autoregressive generator that embeds an implicit equilibrium layer between shallow pre/post blocks. The implicit layer is trained with Jacobian-Free Backpropagation, yielding constant training memory, while inference exposes a per-scale iteration knob that enables compute control. On ImageNet 256x256 benchmark, VIAR attains FID 2.16, and sFID 8.07 with only 38.4% parameters of VAR, matching or surpassing strong AR baselines and remaining competitive with large diffusion models. By controlling the per-scale knob, VIAR can reduce peak memory from 19.24 GB to 8.53 GB and doubles throughput from 15.16 to 32.08 images/s on a single RTX 4090, without retraining. Ablations show that fewer steps are sufficient for fixed-point iterations to converge and that VIAR consistently dominates VAR across quality efficiency operating points. In zero shot in-painting and class-conditional editing, VIAR produces sharper details and smoother boundaries while preserving global structure, validating the benefits of implicit equilibria and per-scale compute control for practical, deployable visual generation.
Pengfei Jiang, Jixiang Luo, Luxi Lin +2
Institute of Artificial Intelligence (TeleAI), China Telecom · Xiamen University
Autoregressive image and video generators are trained with teacher-forced histories but must sample from their own generated prefixes at inference time, making them vulnerable to exposure bias and prefix drift. Existing remedies either modify training or apply sampling-time guidance aimed primarily at external semantic conditions, such as class labels or text prompts, rather than testing whether a next-step prediction provides strong posterior support for the generated prefix itself. We propose Visual Prefix Guidance (VPG), a training-free inference-time guidance method for autoregressive image and video generation. VPG improves next-step prediction by contrasting the model's output under the generated prefix with its output under a corrupted prefix, then extrapolating logits toward candidates that strengthen the posterior support of the generated prefix. Across class-conditional image generation with VAR, text-to-image generation with Infinity, and text-to-video generation with InfinityStar, VPG improves generation quality without retraining the base model, reducing FID on VAR by 0.36 on average and improving benchmark performance on both image and video generation.
Xinyao Liao, Qiyuan He, Yicong Li +4
National University of Singapore · Huazhong University of Science & Technology