Organizations: State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China. · Q. Zhao and X.-Y. Zhang are the School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS), Beijing 100049, China · Shanghai Jiao Tong University, Shanghai 200240, China. · Faculty of Business and Economics, The University of Hong Kong, Hong Kong, China.
Diffusion-based multimodal large language models (dMLLMs) decode by iterative denoising with bidirectional attention, which precludes the standard autoregressive key--value cache. Practical deployments therefore rely on approximate caching that reuses intermediate states across denoising steps. Such caching, however, often makes a response repeat one token at adjacent positions in a run that breaks the sentence, a failure we term the Repeat Curse. Tracing the context-token information flow reveals three disruptions. Within a step, caching disperses the attention that anchors a position on nearby context tokens. Across steps, similarity-based refresh is not staleness-aware, so some positions are decoded against stale key--value states. Across layers, the deep-layer entropy of the context fails to converge where repetition arises. We accordingly propose CoTA++, a training-free, plug-and-play framework that counteracts each disruption where it originates: Context-Token Attention Re-anchoring re-forms the attention routing against the current context, Decode-Aware Refresh reserves part of the cache's refresh budget for the positions about to be committed, and Context-Token Entropy-Guided Voting discounts the confidence of a position by the deep-layer entropy of its context, which reorders the unmasking. The three components act at non-overlapping stages of a decoding step and compose freely with existing dMLLMs and caches. Across three dMLLMs and two caching backends, CoTA++ reduces the sample repetition rate of long responses from up to 99.6% to at most 17.0%, recovers most of the accuracy lost to caching on nine benchmarks, and keeps the speedup of caching at under 15% overhead, making approximate caching reliable for dMLLMs.
Figures & tables
Fig. 1 : Motivation. LLaDA-V [ 7 ] responses to the same query: vanilla decoding is normal (top); dLLM-Cache [ 18 ] induces the Repeat Curse (repetitions in red, middle); CoTA++ removes them while retaining the acceleration (bottom).
Fig. 2 : The Repeat Curse and its three information-flow signatures. (a) Sample repetition rate on 500 MS-COCO captions, L=512 . (b)–(d) LLaDA-V, L=128 , vanilla against dLLM-Cache. (b) Within-step attention of suffix queries: caching disperses local anchoring (F1). (c) KV staleness at the decode moment along one response: most repeats (red) decode stale (F2). (d) Cross-layer entropy after generation at that response’s repeat positions and at normal positions: repeats fail to consolidate (F3).
Fig. 3 : Overview. Caching disrupts the information flow of context tokens on the three axes of dMLLM inference, and CoTA++ counteracts each disruption where it originates.
Method
Cache granularity
Refresh criterion
Decode- aware
Intervention level
Fast-dLLM [ 20 ]
block
block boundary
✗
schedule
Prefix-KV [ 9 ]
prefix only
none (exact)
n/a
schedule
dLLM-Cache [ 18 ]
per position
periodic + similarity
✗
schedule
SlowFast [ 19 ]
per position
periodic + similarity
✗
schedule + decoding
dkv-cache [ 47 ]
per position
delayed commit
✗
schedule
CoTA [ 1 ]
plug-in
inherited
✗
attention + decoding
TABLE I : Positioning of CoTA++ among dMLLM caching methods. Only CoTA++ makes its refresh decode-aware.
Fig. 4 : Radar charts of the Repeat Curse. Near zero without caching, every metric surges with it (labels: vanilla → cached).
Fig. 5 : Context tokens are anchors, and caching disperses them (Finding F1). (a) Attention of vanilla LLaDA-V (brighter is higher). (b)–(d) Change in local anchoring under dLLM-Cache on 94 MS-COCO images, Δw5 (cached − vanilla), where w5 is the share of a position’s suffix attention within ±5 positions. Repeat regions: repeat positions ±5 . Distant regions: all other positions. Numbers in (b) : ratio of the two losses. Error bars: 95% intervals over images.
Component sweep
SRR (%) ↓
Prefix interval Ep
1 99.0
5 97.2
15 97.6
25 96.8
Suffix interval Es
1 0.6
3 68.4
5 90.8
7 96.8
Similarity fraction α
0 99.8
0.25 96.8
0.75 25.2
1 0.4
Reuse policy
prefix-only 0.6
dLLM-Cache 96.8
—
—
TABLE II: Ablation of dLLM-Cache components on LLaDA-V ( L=128 ; SRR, %). Each row sweeps one component with the others at their defaults (shaded).
Fig. 6 : Cache staleness concentrates on repeat positions (Finding F2). (a) Staleness at the decode moment on 94 images. (b) – (d) Reference sample, on shared axes: (b) staleness of each position up to its decode (faded afterwards), (c) and (d) how each KV state is read at layer 22 (attention received, averaged over heads and queries). Triangles: repeat positions.
Positions
Δ>τ
(a) Quarter (steps)
Repeat
Normal
Repeat
Normal
Images
p
1st (0–31)
15
2,899
0.20
0.02
3/13
0.54
2nd (32–63)
110
2,898
0.63
0.19
43/51
7.8×10−9
3rd (64–95)
337
2,671
0.47
0.40
51/70
2.7×10−5
4th (96–127)
484
2,524
0.50
0.42
64/86
5.1×10−6
All
946
10,992
0.50
0.25
85/94
3.4×10−15
TABLE III : Staleness, refresh and repetition on 94 images. Δ>τ : share decoded beyond τ=3 . Images: where repeat exceeds normal. p : Wilcoxon signed-rank test over images.
Fig. 7 : Consolidation fails at repeat positions (Finding F3). (a) Entropy per layer (shaded: deep window). (b) Repetition rate by decile of the deep-window entropy of the committed neighbors within ±5 . Intervals: 95% , over 94 images.
(a) End of generation
Repeat
Normal
Images
p
Deep-window entropy (bits)
13.7
10.1
92/94
4.6×10−17
Halved not before layer 31 (%)
79
36
89/94
2.0×10−15
(b) Decode moment
Top fifth
Rest
Images
p
Repetition rate (%)
21.2
4.6
66/94
1.5×10−7
Share of all repetitions (%)
54
46
TABLE IV : Consolidation and repetition on 94 images. Top fifth: highest context entropy. Images: where the first value exceeds the second. p : Wilcoxon signed-rank test over images.
Fig. 8 : The CoTA++ framework. Top : a cached dMLLM reuses the feed-forward, attention and KV states of Step K at Step K−1 and unmasks the response in order of confidence. Badges mark where each component acts. Bottom : each component acts on its own object. CTAR re-anchors the attention of the positions its monitor selects, DAR refreshes the KV states of the positions about to commit, and CTEV discounts the confidence by the deep-layer entropy of the context.
Component
Intervention level
Extra cost per step
CTAR
Attention routing, in-pass
O(∣D∣∣Tt∣(S+d)d)
DAR
Refresh set, at the step boundary
O(M) index ops
CTEV
Voting score, after the pass
O(∣W∣Md∣V∣)
TABLE V : Per-step cost of the three components. D : re-anchored layer band; Tt : positions selected by the monitor; W : deep window of the entropy readout; M : suffix length; S : sequence length; d : hidden size; ∣V∣ : vocabulary size.
L=512
L=128
L=64
Avg.
Model
Method
ARR ↓
SRR ↓
MRL ↓
ARL ↓
95pRL ↓
ARR ↓
SRR ↓
MRL ↓
ARL ↓
95pRL ↓
ARR ↓
SRR ↓
MRL ↓
ARL ↓
95pRL ↓
ARR ↓
SRR ↓
Vanilla
0.01
2.4
3
2.07
2.6
0.00
0.2
2
2.00
2.0
0.01
0.4
2
2.00
2.0
0.01
1.0
dLLM-Cache [ 18 ]
14.70
99.6
455
5.90
3.0
7.01
96.8
73
2.88
3.0
1.61
70.8
38
2.12
2.0
7.77
89.1
+ CoTA [ 1 ]
0.17
47.4
3
2.00
2.0
0.10
11.4
2
2.00
2.0
0.04
2.4
2
2.00
2.0
0.10
20.4
+ CoTA++
0.03
8.2
3
2.12
3.0
0.02
2.4
2
2.00
2.0
0.01
0.4
2
2.00
2.0
0.02
3.7
SlowFast [ 19 ]
0.28
17.6
232
2.89
2.0
0.22
17.4
3
2.03
2.0
0.55
31.2
2
2.00
2.0
0.35
22.1
TABLE VI : Repeat Curse across dMLLMs and caching backends on the 500 COCO images at three response lengths L , under the instruction “Please describe the image in detail.”, on content tokens (§ VI-A ). ARR and SRR are in percent. MRL, ARL and 95pRL are the maximum, mean and 95th percentile of the lengths of all repeated runs in the 500 responses. Avg.: mean over the three lengths. ↓ : lower is better. Bold: best within a backend group. Shaded: CoTA++.
L=512
L=128
L=64
Avg.
Model
Method
Judge ↑
Len
Judge ↑
Len
Judge ↑
Len
Judge ↑
Vanilla
3.93
475
4.77
127
4.34
63
4.35
dLLM-Cache
3.03
509
3.59
127
4.17
63
3.60
+ CoTA
3.89
510
4.73
127
4.38
63
4.33
+ CoTA++
4.00
510
4.77
127
4.40
63
4.39
SlowFast
4.12
254
4.34
99
3.94
56
4.13
TABLE VII : Response quality and length in the setting of Table VI . Judge is the mean score of the 500 descriptions on a 1 – 10 scale (§ VI-A ), Len the mean response length in tokens. Bold: best within a backend group. Shaded: CoTA++.
LLaDA-V
MMaDA
LaViDa
Method
Rep-4 ↓
Dist-2 ↑
Rep-4 ↓
Dist-2 ↑
Rep-4 ↓
Dist-2 ↑
Vanilla
24.6
58.2
2.2
89.3
2.0
86.8
dLLM-Cache
37.9
45.7
7.7
78.5
14.5
73.6
+ CoTA
21.9
57.7
2.4
88.2
2.0
86.6
+ CoTA++
20.5
59.1
2.2
89.2
1.9
86.9
TABLE VIII : Phrase-level repetition at L=512 under dLLM-Cache: repeated 4 -grams (Rep-4) and distinct 2 -grams (Dist-2) in percent (§ VI-A ). Bold: best among the cached rows. Shaded: CoTA++.
Fig. 9 : Case study , LLaDA-V under dLLM-Cache: the uncached model, the cached model, and the cached model with CoTA++. (a) The typical case, L=128 . (b) A residual failure, L=512 : a doubled period at a sentence boundary.
(a) dLLM-Cache
Model
Doc
Chart
MMS
MME P
MME C
SEED
MMB
LLaVA W
MVista
MVerse
LLaDA-V [ 7 ]
83.9
78.3
60.2
1508
492
74.8
83.0
51.7
60.4
29.2
+ dLLM-Cache
70.5
78.2
60.0
1503
497
74.8
83.0
30.2
55.9
26.1
+ CoTA [ 1 ]
81.6
78.6
60.0
1505
499
74.8
83.2
50.0
57.0
27.5
+ CoTA++
82.1
78.6
60.0
1505
499
74.8
83.2
50.4
57.2
28.3
MMaDA [ 8 ]
–
10.1
37.3
1087
285
56.6
42.6
29.7
34.2
18.8
TABLE IX : General multimodal benchmarks under (a) dLLM-Cache and (b) SlowFast: DocVQA, ChartQA, MMStar, MME (perception, cognition), SEED, MMBench, LLaVA-Bench (in-the-wild), MathVista and MathVerse. MMaDA uses the MMaDA-8B-MixCoT checkpoint, the only one with a released evaluation protocol.
L=64
L=128
L=512
DAR
CTAR
CTEV
ARR
SRR
MRL
ARR
SRR
MRL
ARR
SRR
MRL
1.61
70.8
38
7.01
96.8
73
14.70
99.6
455
✓
0.02
1.4
2
0.04
4.4
2
0.10
11.2
165
✓
0.13
8.0
2
0.21
20.6
3
0.31
62.0
160
✓
0.77
35.6
34
0.86
59.6
56
1.72
94.0
420
✓
✓
0.02
0.6
2
0.02
3.0
2
0.05
9.6
3
TABLE X : Component grid (LLaDA-V with dLLM-Cache, 500 images): every subset of the three components at three response lengths. Shaded: CoTA++.
Configuration
Latency (s/img) ↓
Throughput (tok/s) ↑
Relative latency
Memory (GiB)
Vanilla
21.4
2.9
–
30.6
dLLM-Cache
8.8
7.1
1.00 ×
32.5
+ DAR
9.0
7.0
1.01 ×
32.5
+ CTAR
9.4
6.7
1.06 ×
32.5
+ CTEV
9.4
6.7
1.07 ×
33.4
+ CoTA++
10.2
6.2
1.15 ×
33.4
TABLE XI : Efficiency on LLaDA-V at L=64 , 20 images, each configuration alone on one A100: latency in seconds per image, throughput in decoded tokens per second, latency relative to dLLM-Cache, and peak GPU memory.
Fig. 10 : Attention at the decode moment, with and without CTAR , layers 25 – 32 ; triangles: repeat positions. (a)–(b) LLaDA-V and (c)–(d) LaViDa under dLLM-Cache, L=128 .
Fig. 11 : Staleness of the KV state at the decode moment, with and without DAR. Mean over 100 images at each decoding step, dLLM-Cache, L=128 .
Fig. 12 : Context entropy at the decode moment, with and without CTEV , dLLM-Cache, L=128 . Top: repeat rate of the committed position. Bottom: the decisions CTEV changes.
Configuration
ARR ↓
SRR ↓
MRL ↓
Rep-4 ↓
Vanilla
0.00
0.0
0
0.037
dLLM-Cache
7.52
96.0
73
0.083
CTAR: re-weight the stored routing, or re-form it
Additive distance bias
6.41
98.0
73
0.073
Gated gain
9.07
99.0
82
0.086
Re-formed, layers 25 – 32 (CTAR)
0.32
21.0
3
0.035
TABLE XII : Design decisions of the three components , LLaDA-V under dLLM-Cache, L=128 , 100 images, each component alone.
Fig. 13 : Tokens that open a repeated run under the cache , sized by run count; a leading hyphen marks a word piece.
Diffusion language model decoding generates text through iterative parallel refinement, balancing fewer denoising steps with accuracy, coherence, and control.
Diffusion multimodal large language models (dMLLMs) frequently produce long-form outputs marred by semantic drift and repetition, with quality generally degrading as output length increases. We identify two structural deficiencies in existing decoding methods as primary drivers of these failures: confidence-based scoring ignores decoded-neighbor support, and block partitioning prevents access to high-readiness semantic anchors, together causing tokens to be committed before their local context is sufficiently established. We propose \ours{} (Context-Aware Cluster Decoding), a training-free decoding method that scores each masked position by a multiplicative composite of softmax confidence and neighbor proximity, promoting contextually ready tokens above isolated candidates while suppressing low-confidence positional noise, operating block-free to keep high-readiness anchors globally accessible. \ours{} further applies architecture-aware calibration to handle confidence heterogeneity induced by diverse visual integration strategies. Experiments on three dMLLMs across four benchmarks demonstrate consistent quality gains and hallucination reduction over Original, with larger gains in several longer generation settings, highlighting the importance of neighbor support and visual integration strategy for future dMLLM decoding method design. Our code is openly available at https://github.com/zhaoyk-sysu/CACD-dMLLM.
Yikai Zhao, Qiyan Zhao, Jiaquan Zhang +3
Sun Yat-sen University · Shanghai Jiao Tong University · University of Electronic Science and Technology of China +2
Diffusion-based large language models (dLLMs) rely on bidirectional attention, which prevents lossless KV caching and requires a full forward pass at every denoising step. Existing approximate KV caching methods reduce this cost by selectively updating cached states, but their decision overhead scales with context length or model depth. We propose EntropyCache, a training-free KV caching method that uses the maximum entropy of newly decoded token distributions as a constant-cost signal for deciding when to recompute. Our design is grounded in two empirical observations: (1) decoded token entropy correlates with KV cache drift, providing a cheap proxy for cache staleness, and (2) feature volatility of decoded tokens persists for multiple steps after unmasking, motivating recomputation of the k most recently decoded tokens. The skip-or-recompute decision requires only O(V) computation per step, independent of context length and model scale. Experiments on LLaDA-8B-Instruct and Dream-7B-Instruct show that EntropyCache achieves 15.2×-26.4× speedup on standard benchmarks and 22.4×-24.1× on chain-of-thought benchmarks, with competitive accuracy and decision overhead accounting for only 0.5% of inference time. Code is available at https://github.com/mscheong01/EntropyCache.
Diffusion Large Language Models (DLLMs) enable non-autoregressive decoding, but efficient inference support remains immature: unlike autoregressive models, whose requests reuse a shared prefix key-value (KV) cache, DLLMs use bidirectional attention, so a shared context's KV states depend on the tokens still being decoded, leaving directly reused caches stale and full recomputation necessary. We present ACache, a cross-request cache reuse mechanism for shared spans, or affixes, at any position: prefix, infix, or suffix. ACache measures the influence of affix tokens on the masked generation region to identify a small request-specific subset as Anchor Tokens, and recomputes only their KV states while reusing the remaining affix cache. Built on state-of-the-art intra-request caching mechanisms, ACache recovers most of the accuracy lost to direct affix-cache reuse on average when recomputing around 20% of affix tokens, and at that budget preserves more accuracy than selection criteria adapted from prior cross-request cache-reuse systems. We co-design ACache with a modern inference engine, whose attention reads each request's recomputed Anchor KV states alongside one affix cache shared across concurrent requests. Against the same system with only intra-request caching, ACache cuts recompute latency by up to 56.7%, translating to as much as 1.71× end-to-end throughput, while reducing peak KV cache memory by up to 45.8%.