CONDUIT: A Unified Residual-Stream Restoration Framework for KV Cache Reuse in Vision-Language Models

Pengan Chen; Kaisheng Zheng; Liang Hong; Lixia Yi; Jiyue Jiang; Jiayang Chen; Yixuan Wang; Yimin Fan

cs.CLSep 5, 2026

CONDUIT: A Unified Residual-Stream Restoration Framework for KV Cache Reuse in Vision-Language Models

Authors: Pengan Chen, Kaisheng Zheng, Liang Hong, Lixia Yi, Jiyue Jiang, Jiayang Chen, Yixuan Wang, Yimin Fan,

Organizations: The Chinese University of Hong Kong · Fudan University · LIGHTSPEED · Independent Researcher

Abstract

Vision-language models (VLMs) often answer new questions about recurring visual content, where reusing the key-value (KV) cache can avoid re-encoding expensive visual prefixes. Exact-prefix reuse, however, fails when the same visual content appears under a changed prefix. Selective recomputation can recover quality under a small visual-token budget, but only when the right stale tokens are refreshed. Raw-attention selection can waste budget on high-attention tokens with small value-norm proxy scores and on query-irrelevant images. To address these failure modes, we propose CONDUIT, a training-free refresh policy that unifies single- and multi-image reuse as residual-stream restoration. Building on norm-weighted attention, CONDUIT ranks cached visual tokens using cached-key query attention and an accessible pre-output cached-value-norm proxy, then applies empirical image-level relevance amplification before one global selection. With one image, the coefficient is one and the rule reduces to intra-image token selection. The method preserves model architecture and weights, adding only a single query-conditioned scoring pass at inference. At a 10% refresh budget, CONDUIT achieves 97.0-99.5% of the corresponding full-prefill five-dataset average across three VLM backbones and leads budgeted methods on average; on the MMLongBench-Doc latency subset, it uses 13.5% of full-prefill FLOPs and achieves a 2.99x time-to-first-token speedup.

Figures & tables

Figure 1: Overview of Conduit . Visual-prefix changes make exact-prefix KV reuse invalid even when image content recurs. Conduit builds a norm-weighted token score from cached-key query attention and an accessible pre-output cached-value-norm proxy, applies empirical image-level relevance amplification, and recomputes only the highest-scoring visual tokens while reusing the rest.

Figure 2: Conduit refresh-mask construction. Cached values provide a pre-output value-norm proxy, while cached keys and the new query estimate query attention. Their product gives each token’s throughput score; a per-image coefficient reweights scores before a global top- $k$ selector chooses tokens to recompute.

	Method	LongDocURL	MMLongBench-Doc	SlideVQA	InfoSeek	ViQuAE	Avg.
Qwen2.5-VL-3B	Full prefill	56.39 ±0.42	34.19 ±0.44	70.46 ±0.42	44.36 ±0.42	42.58 ±0.44	49.60
	Cache reuse	47.93 ±0.35	31.80 ±0.45	64.59 ±0.44	32.34 ±0.40	34.88 ±0.35	42.31
	MPIC	48.16 ±0.38	31.35 ±0.40	64.31 ±0.36	42.16 ±0.40	38.00 ±0.40	44.80
	CacheBlend	48.02 ±0.42	31.18 ±0.39	64.26 ±0.39	32.60 ±0.36	37.38 ±0.40	42.69
	KVShare	48.10 ±0.41	31.51 ±0.37	64.43 ±0.40	32.36 ±0.43	36.33 ±0.37	42.55
	ProphetKV	54.35 ±0.40	32.25 ±0.37	68.52 ±0.44	42.30 ±0.40	40.36 ±0.38	47.56

Table 1: Main shifted-prefix results at

r=0.10

. Entries are computed with each benchmark’s official metric on three VLM backbones; higher is better. Full prefill and cache reuse are reference anchors, and bold marks the best budgeted method per column (highest mean). Cells report mean ± standard deviation across

5

random seeds (pp); the Avg. column reports the unweighted mean across the five datasets. Complete budget sweeps are in Appendix B .

Figure 3: Budget–quality curves by backbone. Each panel reports the unweighted average over LongDocURL, MMLongBench-Doc, SlideVQA, InfoSeek, and ViQuAE; horizontal lines mark full prefill and zero-refresh cache reuse.

Qwen2.5-VL-3B
Method	$r$	MMBench	OCRBench	POPE	MMStar	Avg.
Full prefill	–	75.1	82.8	87.5	54.1	74.9
Cache reuse	0	74.7	82.4	87.9	53.4	74.6
Conduit	0.05	74.9	82.7	87.8	54.4	75.0
Qwen2.5-VL-7B
Full prefill	–	78.3	88.4	87.7	60.9	78.8

Table 2: Single-image reuse in the

K{=}1

limit. Entries are computed with each benchmark’s official metric; higher is better. Since

c_{1}=1

, Conduit reduces to intra-image throughput selection. At

r=0

no visual token is recomputed; at

r=0.05

the top-throughput

5\%

are refreshed.

Method	Score	TFLOPs	TTFT (s)	Speedup
Full prefill	43.59	90.36	0.519	1.00 $\times$
Cache reuse	41.41	2.47	0.049	10.69 $\times$
CacheBlend	41.82	11.36	0.180	2.89 $\times$
MPIC	41.78	9.97	0.090	5.76 $\times$
KVShare	41.19	11.02	0.284	1.83 $\times$
ProphetKV	42.06	12.13	0.169	3.07 $\times$

Table 3: Serving trade-off on a context-stratified MMLongBench-Doc latency subset. Quality, prefill TFLOPs, and TTFT are averaged over three backbones and

4

8

16

k contexts. Budgeted methods use

r=0.10

; higher Score and lower TFLOPs / TTFT are better.

$\boldsymbol{\lambda}$	LongDocURL	MMLB-Doc	SlideVQA	InfoSeek	ViQuAE	Avg.
0	55.28	33.73	69.79	42.85	41.02	48.53
0.25	55.29	34.06	70.29	43.30	41.27	48.84
0.50	55.92	34.38	70.97	43.30	41.91	49.30
0.75	55.83	34.37	70.59	43.44	41.91	49.23
1.00	56.00	34.40	70.88	43.39	42.00	49.33

Table 4: Sensitivity to image reweighting on Qwen2.5-VL-3B at

r=0.10

. Entries use official benchmark metrics; MMLB-Doc abbreviates MMLongBench-Doc. Higher is better.

\lambda=0

is exactly the “drop

c_{j}

” ablation.

Variant	Qwen2.5-VL-3B	Qwen2.5-VL-7B	InternVL3-9B
Conduit (full)	49.33	58.70	55.97
$\;-$ drop $c_{j}$	48.53	58.23	55.43
$\;-$ drop $\\|V\\|$	48.44	57.35	55.44

Table 5: Factor ablation of the composite score (

r=0.10

, averaged over the five shifted-prefix datasets). Dropping the image coefficient sets

c_{j}\equiv 1

(intra-image throughput only); dropping the value norm ranks by query attention alone.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

$r=0.05$
Method	LongDocURL	MMLongBench-Doc	SlideVQA	InfoSeek	ViQuAE	Avg.
Full prefill	56.39 ±0.42	34.19 ±0.44	70.46 ±0.42	44.36 ±0.42	42.58 ±0.44	49.60
Cache reuse	47.93 ±0.35	31.80 ±0.45	64.59 ±0.44	32.34 ±0.40	34.88 ±0.35	42.31
MPIC	47.87 ±0.38	31.66 ±0.43	64.49 ±0.41	40.29 ±0.44	36.78 ±0.39	44.22
CacheBlend	47.61 ±0.45	31.83 ±0.44	64.74 ±0.38	32.42 ±0.36	36.30 ±0.39	42.58
KVShare	47.44 ±0.43	31.89 ±0.35	64.64 ±0.36	32.35 ±0.37	36.05 ±0.40	42.47

Appendix

Table 6: Complete Qwen2.5-VL-3B shifted-prefix results at

r\in\{0.05,0.10\}

. Entries are computed with the official metrics for LongDocURL, MMLongBench-Doc, SlideVQA, InfoSeek, and ViQuAE; higher is better. Full prefill and cache reuse are reference anchors, bold marks the best budgeted method per cell (highest mean), and Avg. is the unweighted mean across the five datasets. Cells report mean ± standard deviation across

5

random seeds (pp).

$r=0.05$
Method	LongDocURL	MMLongBench-Doc	SlideVQA	InfoSeek	ViQuAE	Avg.
Full prefill	65.40 ±0.39	53.43 ±0.44	73.70 ±0.39	54.54 ±0.42	55.58 ±0.41	60.53
Cache reuse	56.59 ±0.42	51.48 ±0.40	68.83 ±0.37	32.46 ±0.39	45.76 ±0.41	51.02
MPIC	55.81 ±0.37	51.81 ±0.36	68.46 ±0.38	46.39 ±0.43	49.06 ±0.35	54.31
CacheBlend	55.87 ±0.41	51.86 ±0.37	68.85 ±0.35	32.68 ±0.36	45.87 ±0.42	51.03
KVShare	56.08 ±0.44	51.42 ±0.37	68.23 ±0.39	32.81 ±0.43	45.76 ±0.44	50.86

Appendix

Table 7: Complete Qwen2.5-VL-7B shifted-prefix results at

r\in\{0.05,0.10\}

5

random seeds (pp).

$r=0.05$
Method	LongDocURL	MMLongBench-Doc	SlideVQA	InfoSeek	ViQuAE	Avg.
Full prefill	57.37 ±0.42	43.28 ±0.40	72.89 ±0.39	51.11 ±0.37	60.17 ±0.42	56.96
Cache reuse	48.01 ±0.43	41.02 ±0.41	65.01 ±0.39	38.09 ±0.44	55.06 ±0.45	49.44
MPIC	47.62 ±0.43	41.77 ±0.37	67.03 ±0.40	42.70 ±0.41	56.28 ±0.37	51.08
CacheBlend	48.27 ±0.37	42.35 ±0.40	66.34 ±0.35	39.88 ±0.37	52.12 ±0.41	49.79
KVShare	48.67 ±0.43	40.50 ±0.42	64.22 ±0.42	38.41 ±0.43	54.97 ±0.42	49.35

Appendix

Table 8: Complete InternVL3-9B shifted-prefix results at

r\in\{0.05,0.10\}

5

random seeds (pp).

	Qwen2.5-VL-3B			Qwen2.5-VL-7B			InternVL3-9B
Method	4k	8k	16k	4k	8k	16k	4k	8k	16k
Full prefill	0.145	0.333	0.780	0.260	0.642	1.380	0.182	0.327	0.624
Cache reuse	0.040	0.042	0.052	0.031	0.035	0.054	0.052	0.057	0.074
CacheBlend	0.092	0.137	0.273	0.095	0.161	0.349	0.113	0.149	0.250
MPIC	0.037	0.054	0.096	0.049	0.084	0.184	0.071	0.096	0.140
KVShare	0.116	0.184	0.421	0.124	0.317	0.653	0.128	0.189	0.427

Appendix

Table 9: Time to first token (s) by backbone and context length on the context-stratified MMLongBench-Doc latency subset; lower is better. Budgeted methods (CacheBlend, MPIC, KVShare, ProphetKV, Conduit ) use a

10\%

refresh budget; full prefill and cache reuse are reference anchors. Averaging across the nine backbone–context settings yields the TTFT column of Table 3 .

	Reduction vs. random					Reduction vs. $\\|V\\|$ -only
Rule	$r=.05$	$.10$	$.20$	$.30$	$.50$	$r=.05$	$.10$	$.20$	$.30$	$.50$
$\mathrm{TopK}(\bar{\alpha})$	5.44	11.55	35.68	94.35	530.13	5.40	10.96	33.75	83.97	471.39
$\mathrm{TopK}(\\|V\\|)$	1.01	1.05	1.06	1.12	1.12	–	–	–	–	–
$\mathrm{TopK}(\bar{\alpha}\cdot\\|V\\|)$	5.38	11.46	35.46	94.09	535.96	5.34	10.87	33.53	83.74	476.57

Appendix

Table 11: Projected residual-error reduction for selection rules. Each entry is the ratio

\mathcal{E}(\text{baseline})/\mathcal{E}(\text{rule})

16

LongDocURL prompts (Qwen2.5-VL-3B, layer 24); the baseline is random selection (left block) or

\mathrm{TopK}(\|V\|)

(right block), and higher is better. Dashes mark self-comparison cells, not missing runs.

Table 14

Gap quartile	Accuracy	Evidence-page budget
	Ours / full	Ours / raw-attn
Q1 (near tie)	46.2 / 46.7	26.1% / 28.5%
Q2	36.5 / 39.9	36.1% / 34.4%
Q3	42.5 / 43.3	49.7% / 43.7%
Q4	30.3 / 32.9	76.9% / 69.1%

Appendix

Table 14: Accuracy and evidence-page budget by attention-gap quartile on the MMLongBench-Doc evidence-annotated subset (Qwen2.5-VL-3B,

r=0.10

). Metrics are computed within each quartile.

Panel of Figure 4 — Figure 4: LongDocURL qualitative example. The prompt contains four document pages and asks for the ratio associated with the period 2001–2003 and the row for marriages. Correct answering requires binding the question to the target table tuple within the document context, rather than selecting a salient nearby number.

Efficient VLM Inference

Efficient vision-language model inference reduces the compute, memory, latency, and energy needed to process images, video, and text.

See all 487 papers

TGV-KV: Text-Grounded KV Eviction for Vision-Language Models
Jun 2, 2026Jizhihui Liu, Ruizi Han, Miao Zhang +4Vision-Language Models Efficient VLM Inference
Harbin Institute of Technology, Shenzhen · Peng Cheng Laboratory
Reroute, Don't Remove: Recoverable Visual Token Routing for Vision-Language Models
Jun 10, 2026Cheng-Yu Yang, Shao-Yuan Lo, Yu-Lun LiuEfficient VLM Inference Visual Token Pruning
National Yang Ming Chiao Tung University · National Taiwan University
ReToken: Improving Long-Context VLMs with Visual Retrieval Token
Jul 30, 2026Yao Xiao, Reuben Tan, Zhen Zhu +3Vision-Language Models Vision-Language Retrieval
University of Illinois at Urbana-Champaign · Microsoft Research · Google DeepMind

Jun 2, 2026J/K move · Enter open · S save

TGV-KV: Text-Grounded KV Eviction for Vision-Language Models

Jizhihui Liu, Ruizi Han, Miao Zhang, Rui Shao+3

Harbin Institute of Technology, Shenzhen · Peng Cheng Laboratory

Vision-Language Models Efficient VLM Inference KV-Cache Eviction

Open paper AskPDF arXiv

Vision-Language Models (VLMs) inherit the auto-regressive generation paradigm and cache the keys and values (KV) of all previous tokens to accelerate inference, resulting in memory consumption that scales linearly with context length. This issue is particularly pronounced in VLMs due to substantial redundancy in the visual modality. Although KV cache eviction approaches can effectively reduce inference memory, they often incur significant performance degradation in VLMs, as most are designed for language models and overlook the inherent gap between text and vision. By systematically analyzing the modality gap in VLMs in this work, we argue that the importance of visual information should be grounded in textual guidance and accordingly propose a Text-Grounded KV Eviction method for VLMs (TGV-KV). TGV-KV comprises three submodules: (1) Text-Vision Budgeting (TVB) assigns budget to each layer based on the mutual information interaction. (2) Text-Weighted Ranking (TWR) assesses the priority of text and ranks vision importance based on weighted text-image attention. (3) Text-Prioritised Retention (TPR) policy strategically preserves text KV to avoid acute information loss. We evaluate TGV-KV across five models with different sizes and architectures, showing that TGV-KV preserves 99.2% full-KV accuracy on the VizWiz-VQA task with LLaVA-NeXT and boosts end-to-end throughput by 52.6% with an extreme retention budget of 5%. Code is available at https://github.com/Danielement321/TGV-KV.

Abstract

Efficient VLM Inference

Explore similar work

TGV-KV: Text-Grounded KV Eviction for Vision-Language Models

Reroute, Don't Remove: Recoverable Visual Token Routing for Vision-Language Models

ReToken: Improving Long-Context VLMs with Visual Retrieval Token