Beyond Query Similarity: Safety-Conditioned Cache Routing for LLM Response Reuse
Organizations: PDL Lab, College of Computer Science and Technology National University of Defense Technology
Abstract
Semantic response caching reuses a cached response for a semantically similar query to improve LLM inference efficiency. However, query similarity alone can lead to unsafe reuse when safety conditions change. Existing approaches improve cache safety through stricter similarity checks, context-aware validation, or response verification, yet they may still miss safety conflicts between the cached response and the new request or introduce unnecessary verification overhead. We argue that safe and efficient semantic caching requires a reuse policy that makes the cached response's safety conditions explicit and uses them to guide the reuse decision with a balance between safety and utility. In this paper, we propose SafeCache, a safety-aware framework that separates candidate retrieval from safety approval and formulates semantic response reuse as a safety-conditioned routing problem. SafeCache matches each cached-response condition against the new request and aggregates the per-condition mismatches into an overall risk score. A separate contradiction guard prevents explicit conflicts from being diluted by strong matches. It further estimates matching uncertainty and calibrates the routing policy under a required unsafe-reuse target and a verifier-call budget to choose among direct reuse, selective verification, or regeneration. Experiments show that SafeCache achieves the better safety-utility trade-off among the evaluated baselines, enabling safer response reuse at comparable cache utility.
Figures & tables
| Metric | Ours (SafeCache) | GPTCache | vCache | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Setting | |||||||||
| VCHR (%) | 36.371 | 40.159 | 40.854 | 43.447 | 41.926 | 39.651 | 38.557 | 38.456 | 38.456 |
| FH (%) | 0.177 | 0.368 | 0.417 | 21.152 | 15.816 | 10.382 | 6.041 | 1.887 | 0.815 |
| Rel. cost ( ) | 0.648 | 0.612 | 0.605 | 0.449 | 0.502 | 0.558 | 0.589 | 0.608 | 0.612 |
| Variant | VCHR (%) | FH (%) |
|---|---|---|
| full SafeCache | 40.159 | 0.368 |
| w/o Explicit Contradiction Check | ||
| w/o Matching Uncertainty | ||
| w/o Uncertainty-Aware Routing | ||
| w/o Conflict-Conditioned Threshold | ||
| w/o Risk-Constrained Selection |
| Component | Cost ( ) | Component | Cost ( ) |
|---|---|---|---|
| Safety modeling | Verifier | ||
| Safety matching | Backend regen. | 0.597 | |
| Cache routing | Total | 0.612 |
| Method | Setting | HR (%) |
| Ours (SafeCache) | 0.000 | |
| 0.000 | ||
| 0.000 | ||
| GPTCache | ||
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| Source | Pairs |
|---|---|
| Schema-Guided Dialogue | 1,542 |
| -bench | 624 |
| SituatedQA | 300 |
| XSTest | 399 |
| Balanced seed set | 360 |
| Total | 3,225 |
| Rule category | Extracted evidence | Purpose in reuse assessment |
|---|---|---|
| State cues | Authorization, temporal, dialogue, tool, and workflow states | Represent conditions whose changed value can invalidate a cached response. |
| Subject and resource cues | Referenced users, records, accounts, items, and other protected targets | Align each cached-response requirement with the corresponding target in the new request. |
| Response-action cues | Answer, request confirmation, or refuse | Detect when the response action required by the new context differs from the cached action. |
| Privacy cues | Public or private scope | Preserve access-scope changes that may be missed by query similarity. |
| Context facts | Remaining normalized state statements | Retain supporting facts used by condition matching and contradiction checks. |
| Setting | Invalid / accepted | Rate (%) | 95% upper (%) | Splits |
|---|---|---|---|---|
| 0.181 | 0.311 | |||
| 0.383 | 0.547 | |||
| 0.430 | 0.600 |
| Verifier source | VCHR (%) | FH (%) | Invalid / accepted |
|---|---|---|---|
| Primary DeepSeek | 36.371 | 0.177 | |
| Fallback DeepSeek | 36.447 | 0.214 | |
| Local Qwen2.5-14B | 35.833 | 0.179 |
| Method | Evaluation class | Visible inputs | Operating rule and scope |
|---|---|---|---|
| GPTCache | Similarity-style adapter | Cached and new query embeddings | Reuse when query cosine similarity exceeds ; no context or verifier is used. |
| vCache | Static pair-level reproduction | Cached and new query embeddings | Apply the published static cosine-threshold rule at ; online adaptation is outside this comparison. |
| Cache-Me | Released-prompt reproduction | Cached and new queries, query similarity | Above , reuse only after a positive query-pair filter decision; no request context is shown. |
| ContextCache | Public-branch reproduction | Cached/new queries and contexts | Mean-pool sentence embeddings and apply the released query/context thresholds of . |
| Conditional-PPL | Adapted likelihood filter | Query similarity, new query, cached response | Fix the query threshold at and select the likelihood cutoff on the calibration subset using Max-F1 or Youden-J. |
| GroundedCache | Adapted evidence validation | Query similarity, cached/new contexts, cached response | Require compatible version markers, evidence overlap, and response support; select thresholds on the calibration subset. |
| Comparator | Setting | Claim | VCHR | FH | |
|---|---|---|---|---|---|
| GPTCache | Sup. | 1.203 | .0092 | ||
| vCache | Sup. | 2.297 | .0044 | ||
| vCache | Sup. | 2.397 | .0148 | ||
| vCache | Sup. | 2.397 | .0449 | ||
| Cache-Me | NI+Sup. | .0441 | |||
| Cache-Me | Sup. | 2.297 | .0253 |
| Method | Setting | Accepted | Valid | Invalid |
|---|---|---|---|---|
| SafeCache | 4,968 | 4,959 | 9 | |
| SafeCache | 5,478 | 5,457 | 21 | |
| SafeCache | 5,579 | 5,555 | 24 | |
| GPTCache | 7,473 / 6,755 / 6,016 | 5,897 / 5,690 / 5,391 | 1,576 / 1,065 / 625 | |
| vCache | 5,588 / 5,335 / 5,279 | 5,248 / 5,235 / 5,235 | 340 / 100 / 44 | |
| Cache-Me | 6,626 / 6,101 / 5,471 | 5,789 / 5,553 / 5,248 | 837 / 548 / 223 |
| Change family | Exposures | False hits | FH (%) |
|---|---|---|---|
| Dialogue state | 6,913 | 9 | 0.278 |
| Freshness | 1,487 | 0 | 0.000 |
| Tool state | 3,010 | 0 | 0.000 |
| Authorization | 240 | 0 | 0.000 |
| Semantic collision | 240 | 0 | 0.000 |
| Refusal validity | 2,021 | 0 | 0.000 |
| Agree (%) | TA | TR | FA | FR | |
|---|---|---|---|---|---|
| 1,911 | 97.28 | 1,818 | 41 | 9 | 43 |
| Signal | AUROC | AUPRC |
|---|---|---|
| 0.8430 | 0.8066 | |
| 0.8127 | 0.8714 | |
| Boolean conflict | 0.7445 | 0.6645 |
Semantic Caching
Semantic caching reuses results for meaningfully similar queries to reduce the latency and cost of AI, retrieval, and database systems.
See all 25 papers