SJD-PAC: Accelerating Speculative Jacobi Decoding via Proactive Drafting and Adaptive Continuation

Jialiang Kang; Han Shu; Wenshuo Li; Yingjie Zhai; Xinghao Chen

cs.CVMar 19, 2026

SJD-PAC: Accelerating Speculative Jacobi Decoding via Proactive Drafting and Adaptive Continuation

Authors: Jialiang Kang, Han Shu, Wenshuo Li, Yingjie Zhai, Xinghao Chen

Organizations: Peking University · Huawei Technologies

Abstract

Speculative Jacobi Decoding (SJD) offers a draft-model-free approach to accelerate autoregressive text-to-image synthesis. However, the high-entropy nature of visual generation yields low draft-token acceptance rates in complex regions, creating a bottleneck that severely limits overall throughput. To overcome this, we introduce SJD-PAC, an enhanced SJD framework. First, SJD-PAC employs a proactive drafting strategy to improve local acceptance rates in these challenging high-entropy regions. Second, we introduce an adaptive continuation mechanism that sustains sequence validation after an initial rejection, bypassing the need for full resampling. Working in tandem, these optimizations significantly increase the average acceptance length per step, boosting inference speed while strictly preserving the target distribution. Experiments on standard text-to-image benchmarks demonstrate that SJD-PAC achieves a $3.8\times$ speedup with lossless image quality. Code is available at https://github.com/KangJialiang/SJD-PAC.

Figures & tables

Panel of Figure 1 — Figure 1: Analysis of acceptance lengths in SJD [ 22 ] . The speedup shown in (b) is derived entirely from additionally accepted tokens (acceptance length $>1$ ).

Figure 2: High sensitivity of AR T2I models to token perturbations. Left : Original generation. Right : Replacing a single token (0.04% of total tokens) during the autoregressive process causes a severe visual artifact (red box).

Figure 3: A conceptual overview of our proposed components compared to Vanilla SJD, illustrating how each method handles a token rejection ( red ✗ ) at iteration $t$ . Vanilla SJD (Top) : A rejection at position 2 forces a break , and the entire subsequent sequence is resampled ( violet $\mathbf{\circlearrowright}$ ). w/ PD (Middle) : After a rejection, Proactive Drafting builds a diverse $K$ -ary tree at the rejection point to increase the probability of acceptance in the subsequent iteration. w/ AC (Bottom) : After a rejection, Adaptive Continuation continues the verification loop, preserving subsequent valid tokens ( violet ✓ ) instead of terminating.

Figure 4: Total Variation distance

d_{\text{TV}}

for text and image generation with respect to the perturbation offset

j

Configuration	Training- Free	Lossless	Acceleration (↑)		Image Quality
Configuration	Training- Free	Lossless	Step	Latency	FID (↓)	CLIP-Score (↑)
MS-COCO 2017
Lumina-mGPT [ 14 ]	✓	✓	$1.00\times$	$1.00\times$	30.79	31.31
w/ EAGLE [ 12 ]	✗	✓	$2.94\times$	$2.10\times$	30.68	31.73
w/ SJD [ 22 ]	✓	✓	$2.22\times$	$2.05\times$	31.13	31.33
w/ $\text{LANTERN++}_{k=10,\lambda=2}$ [ 17 ]	✗	✗	$3.19\times$	$2.28\times$	29.96	30.11

Table 1: Main results comparing our SJD-PAC method against baselines on acceleration and image quality. We report Step Compression (↑), wall-clock Latency speedup (↑), FID (↓), and CLIP-Score (↑) on the Lumina-mGPT and Emu3 models. Evaluations are conducted on the MS-COCO 2017 and PartiPrompts benchmarks. Bold indicates the highest speedups in each category.

Method	Window Len	Step Compression (↑)
Baseline	32	$2.31\times$
+ PD	32	$2.71\times$
+ PD + AC	32	$3.52\times$
+ PD + AC	64	$\mathbf{4.51\times}$

Table 2: Ablation study on the contribution of each component (PD, AC) and the effect of window length (

L

$K$	$D$	Step Compression (↑)
2	3	$4.23\times$
3	3	$4.42\times$
5	3	$4.47\times$
4	2	$4.34\times$
4	4	$4.49\times$
4	3	$\mathbf{4.51\times}$

Table 3: Hyperparameter sensitivity analysis for Proactive Drafting’s

K

-ary factor (

K

) and tree depth (

D

). All experiments use the full method with

L=64

Figure 5: Qualitative Comparison. Images generated using various prompts from the MS-COCO dataset [ 13 ] . The outputs from our accelerated method ( SJD-PAC ) are visually indistinguishable from those of the baseline models, corroborating our lossless guarantee.

Figure 6: Qualitative comparison with lossy acceleration.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Operation	Vanilla AR	SJD-PAC
Model Forward	$40.8$ ms	$44.3$ ms
Tree Verification & Generation	-	$3.2$ ms
KV Cache Update & Backtracking	-	$2.2$ ms
Other Overheads	$1.0$ ms	$1.0$ ms

Appendix

Table 4: Runtime breakdown of a single decoding step in milliseconds. Comparison between Vanilla AR and SJD-PAC with window length

L=64

Explore similar work

SJD-VP: Speculative Jacobi Decoding with Verification Prediction for Autoregressive Image Generation
Mar 28, 2026Bingqi Shan, Baoquan Zhang, Xiaochen Qi +3Speculative Decoding Autoregressive Image Generation
Harbin Institute of Technology, Shenzhen · Institute of Perceptual Intelligence, Pengcheng Laboratory, Shenzhen · SIFAR
PathRelax: Parallel-Path Relaxed Speculative Jacobi Decoding for Accelerating Auto-Regressive Text-to-Image Generation
Jun 9, 2026Haodong Lei, Hongsong Wang, Bingxuan Dai +1T2I Generation Speculative Decoding
College of Software Engineering Southeast University · School of Computer Science and Engineering Southeast University · School of Cyber Science and Engineering Southeast University +1
SJD-PV: Speculative Jacobi Decoding with Phrase Verification for Autoregressive Image Generation
Mar 2, 2026Zhehao Yu, Baoquan Zhang, Bingqi Shan +5Speculative Decoding Autoregressive Image Generation
Harbin Institute of Technology, Shenzhen · SIFAR

Mar 28, 2026J/K move · Enter open · S save

SJD-VP: Speculative Jacobi Decoding with Verification Prediction for Autoregressive Image Generation

Bingqi Shan, Baoquan Zhang, Xiaochen Qi, Xutao Li+2

Harbin Institute of Technology, Shenzhen · Institute of Perceptual Intelligence, Pengcheng Laboratory, Shenzhen · SIFAR

Speculative Decoding Autoregressive Image Generation Autoregressive Decoding

Open paper AskPDF arXiv

Speculative Jacobi Decoding (SJD) has emerged as a promising method for accelerating autoregressive image generation. Despite its potential, existing SJD approaches often suffer from the low acceptance rate issue of speculative tokens due to token selection ambiguity. Recent works attempt to mitigate this issue primarily from the relaxed token verification perspective but fail to fully exploit the iterative dynamics of decoding. In this paper, we conduct an in-depth analysis and make a novel observation that tokens whose probabilities increase are more likely to match the verification-accepted and correct token. Based on this, we propose a novel Speculative Jacobi Decoding with Verification Prediction (SJD-VP). The key idea is to leverage the change in token probabilities across iterations to guide sampling, favoring tokens whose probabilities increase. This effectively predicts which tokens are likely to pass subsequent verification, boosting the acceptance rate. In particular, our SJD-VP is plug-and-play and can be seamlessly integrated into existing SJD methods. Extensive experiments on standard benchmarks demonstrate that our SJD-VP method consistently accelerates autoregressive decoding while improving image generation quality.

Abstract

Explore similar work

SJD-VP: Speculative Jacobi Decoding with Verification Prediction for Autoregressive Image Generation

PathRelax: Parallel-Path Relaxed Speculative Jacobi Decoding for Accelerating Auto-Regressive Text-to-Image Generation

SJD-PV: Speculative Jacobi Decoding with Phrase Verification for Autoregressive Image Generation