CIAR: Interval-based Collaborative Decoding for Image Generation Acceleration

Keming Ye; Zhou Zhao; Fan Wu; Shengyu Zhang

cs.CVMar 26, 2026

CIAR: Interval-based Collaborative Decoding for Image Generation Acceleration

Authors: Keming Ye, Zhou Zhao, Fan Wu, Shengyu Zhang

Organizations: Zhejiang University · Shanghai Jiao Tong University

Abstract

Auto-regressive (AR) models have recently made notable progress in image generation, achieving performance comparable to diffusion-based approaches. However, their computational intensity and sequential nature impede on-device deployment, causing disruptive latency. We address this via a cloud-device collaboration framework CIAR, which utilizes on-device self-verification to handle two key properties of visual synthesis: the vast token vocabulary required for high-fidelity images and inherent spatial redundancy which leads to extreme predictability in homogeneous regions, while object boundaries exhibit high uncertainty. Uniform verification wastes resources on such redundant tokens. Our solution centers on an on-device token uncertainty quantifier, which adopts continuous probability intervals to accelerate processing and make it feasible for large visual vocabularies instead of conventional discrete solution sets. Additionally, we incorporate a Interval-enhanced decoding module to further speed up decoding while maintaining visual fidelity and semantic consistency via a distribution alignment training strategy. Extensive experiments demonstrate that CIAR achieves a 2.18x speed-up and reduces cloud requests by 70%, while preserving image quality compared to existing methods.

Figures & tables

Figure 1: (a) Acceptance analysis of Lantern. The pie chart shows the ratio of max-prob vs. other tokens, and the bar chart compares Lantern without verification to the baseline. (b) Comparison of decoding frameworks. From left to right: baseline, Lantern, and our CIAR with Inter-Head and cloud-device collaboration, which reduces latency while preserving output quality.

Figure 2: Overview of CIAR. (a) The cloud-side AR model generates image token prefixes from the input prompt. These prefixes are then sent to (b) a lightweight device model with Inter-Head accepts confident tokens locally and sends uncertain ones with interval features to the cloud for verification and distribution alignment. (c) Interval-Based Alignment Strategy. (d) Computation of uncertainty intervals in the Inter-Head.

		Metric
Models	Methods	CLIP (↑)	FID (↓)	F1(↑)	HPSv2(↑)	Latency(s)	steps	Cloud Call
	Base	0.3161	23.6900	0.6097	22.74	x1.00	x1.00	100.00%
	Eagle2	0.3115	25.0700	0.6017	22.74	x1.02	x1.16	87.02%
	Lantern	0.3159	25.5494	0.5834	21.29	x1.66	x2.01	50.11%
	Entropy-Lens	0.3132	24.5828	0.5796	22.03	x1.70	x2.05	52.34%
	CoDe (N = 0.3)	0.2827	35.6670	0.4625	18.08	x2.04	x2.734	30.00%

Table 1: Main results. Our method significantly reduces latency and cloud requests while maintaining comparable CLIP score, FID, F1, and HPSv2 performance.

		Metric
Models	Methods	CLIP ( $\uparrow$ )	FID ( $\downarrow$ )	F1( $\uparrow$ )	HPSv2( $\uparrow$ )	Speedup	Steps	Cloud Call
	Base	0.3161	23.6900	0.6097	22.74	x1.00	x1.00	100.00%
	Random	0.3142	30.1872	0.5369	18.16	x2.28	x2.36	36.46%
	Entropy-Lens	0.3132	24.5828	0.5796	22.03	x1.70	x2.05	52.34%
	SoftmaxCorr	0.3149	31.1009	0.5130	19.11	x2.27	x2.31	36.49%
LlamaGen(Stage I)	Ours	0.3159	24.2459	0.5997	22.48	x2.53	x3.00	30.44%

Table 2: Performance comparison of different uncertainty method.

Figure 3: Comparison of different prefix rate

Figure 4: Visual analysis of different methods. “CIAR w/o Inter-Enhance” denotes our method without interval-enhanced decoding using interval features.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Comparison of different threshold

Figure 7: Visual analysis of Discrete metric and Continuous metric

			Metric
Models	Method	Param. $N$	CLIP ( $\uparrow$ )	FID ( $\downarrow$ )	F1( $\uparrow$ )	HPSv2( $\uparrow$ )	Latency(s)	Cloud Call
	Base	-	0.3161	23.6900	0.6097	22.74	x1.00	100.00%
		0.10	0.2566	44.7305	0.4028	15.84	x2.71	10.00%
		0.15	0.2620	42.6082	0.4088	16.36	x2.41	15.00%
		0.20	0.2627	40.3943	0.4508	16.83	x2.14	20.00%
		0.25	0.2783	38.3103	0.4639	17.43	x2.13	25.00%

Appendix

Table 4: Impact of different

N

values on CoDe and comparison with CIAR.

		Metric
Models	#Layers	CLIP ( $\uparrow$ )	FID ( $\downarrow$ )	F1( $\uparrow$ )	HPSv2( $\uparrow$ )	Latency(s)	Steps	Cloud Call
	1 Layer	0.3159	24.2459	0.5997	22.48	x2.53	x3.00	30.44%
	2 Layers	0.3113	23.8820	0.6270	22.44	x2.54	x3.44	27.06%
	3 Layers	0.3171	23.8404	0.5899	22.44	x2.19	x3.58	26.10%
LlamaGen(Stage I)	5 Layers	0.3171	23.8232	0.6180	22.45	x2.14	x3.66	25.68%

Appendix

Table 5: Performance comparison of device models with varying depths.

		Metric
Method	#Head	CLIP ( $\uparrow$ )	FID ( $\downarrow$ )	F1( $\uparrow$ )	HPSv2( $\uparrow$ )	Latency(s)	Steps	Cloud Call
	1 Head	0.3159	24.2459	0.5997	22.48	x2.53	x3.00	30.44%
	2 Heads	0.3157	24.1517	0.5878	22.38	x2.46	x2.98	29.45%
	3 Heads	0.3151	24.1301	0.6099	22.30	x2.45	x2.97	28.86%
	4 Heads	0.3137	24.6195	0.5999	22.26	x2.49	x2.97	28.73%
Ensemble	5 Heads	0.3147	24.6723	0.5988	22.26	x2.50	x3.00	28.38%

Appendix

Table 6: Impact of Inter-Head capacity with Ensemble and Voting aggregation mechanisms.

		Metric
Models	Loss	CLIP ( $\uparrow$ )	FID ( $\downarrow$ )	F1( $\uparrow$ )	HPSv2( $\uparrow$ )	Latency(s)	Steps	Cloud Call
	w/o $L_{c}$	0.3149	25.2721	0.5889	21.92	x2.37	x2.98	28.05%
	w/o $L_{u}$	0.2891	47.8981	0.4110	16.65	x2.39	x2.99	32.17%
	w/o $L_{1}$	0.2945	25.2849	0.5889	22.45	x2.37	x2.98	28.05%
	w/o $L_{\text{anchor}}$	0.2168	68.6400	0.4114	15.78	x2.30	x2.85	33.79%
LlamaGen(Stage I)	Ours	0.3159	24.2459	0.5997	22.48	x2.53	x3.00	30.44%

Appendix

Table 7: Ablation study on the Inter-DRO loss.

Network	Method	Device (ms)	Cloud (ms)	Communication (ms)	Total (ms)	Ratio (Comm/Tot)
5G	Lantern	11.63	11586.29	4424.63	16022.54	0.2762
5G	Ours	13.54	5102.6	1678.13	6794.27	0.247
4G	Lantern	11.9	11577.91	22291.31	33881.12	0.6579
4G	Ours	14.1	4847.2	8173.95	13035.26	0.6271
WiFi	Lantern	11.69	11436.11	8768.01	20215.81	0.4337
WiFi	Ours	14.11	5052.52	3343.94	8410.58	0.3976

Appendix

Table 8: Comparison of communication costs between Lantern and CIAR under different network conditions. Device and Cloud denote the computational latency on the respective endpoints, while Communication represents the data transmission overhead.

Network	Bandwidth (Mbps)	Latency (ms)
5G	300.00	10.00
4G	20.00	50.00
WiFi	100.00	20.00

Appendix

Table 9: Detailed configurations of different network conditions.

Model	Method	IS $\uparrow$	FID $\downarrow$	Precision $\uparrow$	Recall $\uparrow$	Latency
	DDIM-50 steps	241	2.27	-	-	x1.00
	DDIM-25 steps	232	3.18	-	-	x2.00
	DDIM-20 steps	221	3.81	-	-	x2.50
	DDIM-12 steps	184	7.80	-	-	x4.17
DiT	Ours	240	2.32	-	-	x4.90
	VAR-d24	311	2.11	0.82	0.59	x1.00

Appendix

Table 10: Performance of CIAR applied to VAR and DiT architectures on ImageNet-1K.

Explore similar work

Continuous Speculative Decoding for Autoregressive Image Generation
Nov 18, 2024Zili Wang, Zheng Zhang, Kun Ding +3Visual Autoregressive Models Speculative Decoding
School of Artificial Intelligence, University of Chinese Academy of Sciences, China · MAIS, Institute of Automation, Chinese Academy of Sciences, China · JD.COM Inc, China +1
SJD-PV: Speculative Jacobi Decoding with Phrase Verification for Autoregressive Image Generation
Mar 2, 2026Zhehao Yu, Baoquan Zhang, Bingqi Shan +5Speculative Decoding Autoregressive Image Generation
Harbin Institute of Technology, Shenzhen · SIFAR
Parallel Jacobi Decoding for Fast Autoregressive Image Generation
Jun 4, 2026Boya Liao, Ying Li, Siyong Jian +1Visual Autoregressive Models Autoregressive Image Generation
Westlake University

Nov 18, 2024J/K move · Enter open · S save

Continuous Speculative Decoding for Autoregressive Image Generation

Zili Wang, Zheng Zhang, Kun Ding, Qi Yang+2

School of Artificial Intelligence, University of Chinese Academy of Sciences, China · MAIS, Institute of Automation, Chinese Academy of Sciences, China · JD.COM Inc, China+1

Visual Autoregressive Models Speculative Decoding Autoregressive Image Generation Autoregressive Decoding

Open paper AskPDF arXiv

Continuous visual autoregressive (AR) models have demonstrated promising performance in image generation, but their inherently sequential nature results in slow inference speed. Speculative decoding, a successful acceleration technique for large language models (LLMs), has effectively accelerated discrete visual AR models. However, the absence of an analogous theory for continuous distributions precludes its use in accelerating continuous AR models. To fill this gap, this work presents continuous speculative decoding, and addresses challenges from: 1) low acceptance rate, caused by inconsistent output distribution modeled by target and draft models, and 2) modified distribution without analytic expression, caused by a complex integral. For challenge 1), we address low acceptance rates through an approximated criterion, a novel denoising trajectory alignment strategy based on reparameterization proximity, and token pre-filling. For challenge 2), we introduce acceptance-rejection sampling algorithm with an appropriate upper bound, thereby avoiding explicitly calculating the integral. Furthermore, our denoising trajectory alignment is also reused in acceptance-rejection sampling, effectively avoiding repetitive diffusion model inference. Extensive experiments on various models at 256x256 and 512x512 resolutions demonstrate that our approach achieves over 2x wall-time speedup while preserving the image generation quality. Codes is available at: https://github.com/MarkXCloud/CSpD

Abstract

Explore similar work

Continuous Speculative Decoding for Autoregressive Image Generation

SJD-PV: Speculative Jacobi Decoding with Phrase Verification for Autoregressive Image Generation

Parallel Jacobi Decoding for Fast Autoregressive Image Generation