ToProVAR: Efficient Visual Autoregressive Modeling via Tri-Dimensional Entropy-Aware Semantic Analysis and Sparsity Optimization

Jiayu Chen; Ruoyu Lin; Zihao Zheng; Jingxin Li; Maoliang Li; Guojie Luo; Xiang Chen

cs.CVFeb 26, 2026

ToProVAR: Efficient Visual Autoregressive Modeling via Tri-Dimensional Entropy-Aware Semantic Analysis and Sparsity Optimization

Authors: Jiayu Chen, Ruoyu Lin, Zihao Zheng, Jingxin Li, Maoliang Li, Guojie Luo, Xiang Chen

Organizations: School of Computer Science, Peking University · School of Electronics Engineering and Computer Science, Peking University · National Key Laboratory for Multimedia Information Processing, Peking University

Abstract

Visual Autoregressive(VAR) models enhance generation quality but face a critical efficiency bottleneck in later stages. In this paper, we present a novel optimization framework for VAR models that fundamentally differs from prior approaches such as FastVAR and SkipVAR. Instead of relying on heuristic skipping strategies, our method leverages attention entropy to characterize the semantic projections across different dimensions of the model architecture. This enables precise identification of parameter dynamics under varying token granularity levels, semantic scopes, and generation scales. Building on this analysis, we further uncover sparsity patterns along three critical dimensions-token, layer, and scale-and propose a set of fine-grained optimization strategies tailored to these patterns. Extensive evaluation demonstrates that our approach achieves aggressive acceleration of the generation process while significantly preserving semantic fidelity and fine details, outperforming traditional methods in both efficiency and quality. Experiments on Infinity-2B and Infinity-8B models demonstrate that ToProVAR achieves up to 3.4x acceleration with minimal quality loss, effectively mitigating the issues found in prior work. Our code will be made publicly available.

Figures & tables

Figure 1: A comparison between our method and state-of-the-art compression methods. The SOTA methods often suffer from issues such as semantic loss, structure distortion, and detail collapse.

Figure 2: Different Optimization Dimensions – FastVAR vs. SkipVAR vs. ToProVAR

Figure 3: Tri-dimensional attention entropy analysis in VAR models: (a) Token-Level Semantic Salience : Pruning low-saliency tokens preserves quality, while pruning high-saliency ones causes severe degradation. (b) Layer-level Semantic Representation : Global Layers capture structure and are pruning-sensitive, whereas detail layers refine local semantics and can be pruned. (c) Scale-level Semantic Depth : complex objects require deeper scales for fine details, while simple ones stabilize earlier, enabling adaptive depth pruning.

Figure 4: Tri-Dimensional Entropy-Aware VAR Sparsity Optimization: (a) Scale-level : compute the low-entropy ratio $\rho_{s}$ across scales and select the pruning start depth via threshold $\tau$ . (b) Layer-level : for each scale, perform SVD on the entropy map to separate Global layers from Detail layers. (c) Token-level : within prunable layers/scales, increase pruning rate with scale and use entropy-based gating $p_{\text{prune}}$ to remove low-salience tokens, preserving salient regions.

Methods	GenEval				DPG				Latency(s) $\downarrow$	Speedup
Methods	Two Obj.	Position	Color Attri.	Overall $\uparrow$	Entity	Relation	Attribute	Overall $\uparrow$	Latency(s) $\downarrow$	Speedup
Infinity-2B	79.01	24.00	58.00	0.69	90.81	88.19	87.89	83.41	2.10	1.0 $\times$
+FastVAR	78.79	27.75	59.50	0.68	88.86	91.57	87.46	83.39	0.80	2.6 $\times$
+SkipVAR	76.77	26.50	57.50	0.67	89.30	87.07	87.01	82.94	1.10	2.0 $\times$
+ToProVAR	78.80	29.50	62.00	0.69	87.39	88.92	90.01	83.07	0.61	3.4 $\times$
Infinity-8B	96.97	61.00	75.00	0.83	90.92	93.57	88.83	86.68	4.86	1.0 $\times$

Table 1: Quantitative comparison on GenEval and DPG. Note, GenEval follows the official protocol without rewritten prompts. Latency is measured on a single GPU with batch size 1.

Methods	HPSv2.1					ImageReward $\uparrow$	Latency(s) $\downarrow$	Speedup
Methods	Photo	Concept-Art	Anime	Paintings	Overall $\uparrow$	ImageReward $\uparrow$	Latency(s) $\downarrow$	Speedup
Infinity-2B	29.40	30.38	31.72	30.39	30.47	0.94	1.57	1.0 $\times$
+FastVAR	28.86	29.90	31.12	29.92	29.95	0.92	0.62	2.5 $\times$
+SkipVAR	29.25	30.25	31.50	30.45	30.39	0.93	0.87	1.8 $\times$
+ToProVAR	29.26	30.15	31.44	30.23	30.27	0.93	0.58	2.7 $\times$
Infinity-8B	29.42	31.27	32.45	30.83	30.99	1.04	5.31	1.0 $\times$

Table 2: Quantitative comparison on HPSv2.1 and ImageReward, two human preference benchmarks. Latency is measured on a single GPU with batch size

1

Method	Landscape			People			Food
Method	Latency	FID $\downarrow$	CLIP $\uparrow$	Latency	FID $\downarrow$	CLIP $\uparrow$	Latency	FID $\downarrow$	CLIP $\uparrow$
Infinity-2B	1.67	44.1	0.267	1.71	58.91	0.281	1.69	84.2	0.270
+FastVAR	0.60	45.1	0.264	0.61	71.8	0.274	0.61	84.7	0.273
+SkipVAR	1.01	58.1	0.260	1.06	73.7	0.253	0.88	102.3	0.256
+ToProVAR	0.50	44.5	0.264	0.48	58.84	0.283	0.46	84.3	0.274

Table 3: Quantitative comparisons of FID and CLIP score with different generation categories on the MJHQ30K dataset. Latency is measured on a single GPU with batch size

1

Figure 5: Qualitative comparison of various methods on complex prompts. Our method effectively prevents semantic loss, structure distortion, and detail collapse while maintaining high visual fidelity.

Table 9

Figure 6: Qualitative Visualizations of the three components ablation(Scale, Layer, Token) on Infinity-2B.The full tri-stage framework better preserves global layout and local details than partial variants that prune along only a subset of dimensions.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

model	scales	layers
Infinity-2B	11	32
Infinity-8B	13	40

Appendix

Table 7: The basic information of models.

Method	Model	Method Parameter
FastVAR	Infinity-2B	{10:0.4;11:0.6}
	Infinity-8B	{10:0.4;11:0.6;12:1.0;13:1.0}
SkipVAR	Infinity-2B	0.84
	Infinity-8B	0.84@2B

Appendix

Table 8: Acceleration Configurations for FastVAR and SkipVAR. NOTE: Notation ’s:p’ means scale index s is pruned with pruning ratio p,All ratios are applied per-layer at inference time.

Method	Two Obj.	Position	Color	Attri.	Overall $\uparrow$	Latency (s) $\downarrow$	Speedup $\uparrow$
HART	0.62	0.13	0.18	0.51	0.51	0.95	1.0 $\times$
+FastVAR	0.59	0.13	0.19	0.50	0.50	0.64	1.5 $\times$
+ToProVAR (ours)	0.61	0.13	0.18	0.51	0.51	0.56	1.7 $\times$

Appendix

Table 9: Comparison of ToProVAR and FastVAR on the GenEval benchmark using HART.

Method	Latency (s) $\downarrow$	Speed $\uparrow$	GenEval $\uparrow$
Infinity-2B	2.10	1.0 $\times$	0.690
Fix Scale Exit	0.41	5.1 $\times$	0.403
Scale Depth Loc.	0.47	4.5 $\times$	0.477
Scale Depth Loc. + Fine-grained Token Prun.	0.78	2.7 $\times$	0.603
Scale Depth Loc. + Layer Repr. Ident.	0.57	3.7 $\times$	0.679
ToProVAR (Scale + Layer + Token)	0.61	3.4 $\times$	0.690

Appendix

Table 10: Extended ablation study of the tri-dimensional progressive framework on Infinity-2B.

Figure 8: Robustness analysis of the scale-depth threshold $\tau$ . Left: mean attention entropy vs. scale. Right: low-entropy ratio $\rho_{s}$ vs. scale, together with the distribution of $\rho_{s}$ for images whose SSIM to the full-scale baseline exceeds $0.8$ .

Figure 9: Visualization of token-level pruning in ToProVAR. Gray tokens indicate those pruned by ToProVAR, while colored tokens correspond to preserved, semantically salient regions.

Figure 10: Additional visualizations of layer representations on HART, Infinity-2B, and Infinity-8B.We show example attention maps for the representative Global and Detail layers.

Figure 11: Additional visualizations of failure layer representation case.

Figure 12: Additional visualization results of FastVAR, SkipVAR, and ToProVAR on Infinity-2B and Infinity-8B. Compared to the baselines, ToProVAR more faithfully preserves global structure and fine details under similar or higher acceleration, yielding visually sharper and more semantically consistent generations.

Explore similar work

StepVAR: Structure-Texture Guided Pruning for Visual Autoregressive Models
Mar 2, 2026Keli Liu, Zhendong Wang, Wengang Zhou +1Inference Acceleration Visual Autoregressive Models
University of Science and Technology of China
Depth Adaptive Efficient Visual Autoregressive Modeling
Apr 19, 2026Chunliang Li, Tianze Cao, Sanyuan ZhaoEfficient Transformer Inference Visual Autoregressive Models
Beijing Institute of Technology · Beijing Institute of Technology, Zhuhai
Where to Refine, When to Stop: Rethinking Redundancy via Latent Discrepancy for Efficient Visual Autoregressive Generation
May 29, 2026Changwang Mei, Peisong Wang, Zekun Li +7Efficient Neural Network Inference Visual Autoregressive Models
Nanjing University of Science and Technology · C2DL, Institute of Automation, Chinese Academy of Sciences · Beijing Academy of Artificial Intelligence +2

Mar 2, 2026J/K move · Enter open · S save

StepVAR: Structure-Texture Guided Pruning for Visual Autoregressive Models

Keli Liu, Zhendong Wang, Wengang Zhou, Houqiang Li

University of Science and Technology of China

Inference Acceleration Visual Autoregressive Models Visual Token Pruning Autoregressive Image Generation Autoregressive Video Generation

Open paper AskPDF arXiv

Visual AutoRegressive (VAR) models based on next-scale prediction enable efficient hierarchical generation, yet the inference cost grows quadratically at high resolutions. We observe that the computationally intensive later scales predominantly refine high-frequency textures and exhibit substantial spatial redundancy, in contrast to earlier scales that determine the global structural layout. Existing pruning methods primarily focus on high-frequency detection for token selection, often overlooking structural coherence and consequently degrading global semantics. To address this limitation, we propose StepVAR, a training-free token pruning framework that accelerates VAR inference by jointly considering structural and textural importance. Specifically, we employ a lightweight high-pass filter to capture local texture details, while leveraging Principal Component Analysis (PCA) to preserve global structural information. This dual-criterion design enables the model to retain tokens critical for both fine-grained fidelity and overall composition. To maintain valid next-scale prediction under sparse tokens, we further introduce a nearest neighbor feature propagation strategy to reconstruct dense feature maps from pruned representations. Extensive experiments on state-of-the-art text-to-image and text-to-video VAR models demonstrate that StepVAR achieves substantial inference speedups while maintaining generation quality. Quantitative and qualitative evaluations consistently show that our method outperforms existing acceleration approaches, validating its effectiveness and general applicability across diverse VAR architectures.

Abstract

Explore similar work

StepVAR: Structure-Texture Guided Pruning for Visual Autoregressive Models

Depth Adaptive Efficient Visual Autoregressive Modeling

Where to Refine, When to Stop: Rethinking Redundancy via Latent Discrepancy for Efficient Visual Autoregressive Generation