Overview

Semantic caching studies when a system can reuse a prior result for a new input that is similar in meaning, reducing repeated model, retrieval, or database work. Methods include embedding-based matching, learned retrieval and eviction policies, structured query normalization, and caches that store intermediate evidence or semantic views. In retrieval-augmented generation and other LLM services, freshness checks, evidence grounding, and certified reuse can help ensure that a fast cache hit is still correct. The main tension is that similarity and high hit rates do not guarantee that answers are interchangeable, while stricter safety checks can reduce the speed and cost benefits.

Latest papers

CardsList

25 papers3 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

  1. Beyond Query Similarity: Safety-Conditioned Cache Routing for LLM Response Reuse

    Nov 7, 2026Shilong Pan, Zhiliang Tian, Weiguo Chen +2Semantic CachingSafety-Utility Trade-Off in Language Models

  2. Semantic Caches for LLM Web Services Leave Their Error Budget Unspent

    Nov 2, 2026Hanshuai Cui, Qianli Ma, Zhiqing TangSemantic CachingLLM Serving

  3. Multi-Turn Semantic Caching for Low-Cost LLM Serving

    Oct 31, 2026Baran Atalar, Selly Beyene, Xutong Liu +1Semantic CachingLLM Serving

  4. Semantics Delivery Network: Rethinking Web Retrieval Infrastructure for LLM Agents

    Sep 18, 2026Peichun Hua, Yunming XiaoSemantic CachingAgentic RAG

  5. Towards Anticipatory Databases Through Shared Data and Workload Semantics

    Sep 13, 2026Farzaneh Zirak, Kasper Overgaard Mortensen, Farhana Choudhury +1Cache ReplacementSemantic Caching

  6. PACE: Perceived-Latency-Aware Cascading Service Routing and Filler Control for QoE-Efficient Retrieval-Augmented Dialogue Serving

    Sep 9, 2026Lin Huang, Yujuan Tan, Weisheng Li +4Retrieval-Augmented GenerationInference-Time Optimization

  7. Which Eviction Policy Should an LLM Cache Use? A Systematic Study Across Workloads, Capacities, and Encoders

    Aug 20, 2026Yash Kulkarni, Shubham Harkare, Arvind Yogesh Suresh BabuSemantic CachingLLM Inference Efficiency

  8. LaCache: Robust Semantic Caching for LLM Serving

    Aug 3, 2026Jiacheng Liang, Yuhui Wang, Tanqiu Jiang +1LLM ServingLLM Security

  9. From ambiguous utterances to governed reuse classes: canonicalization, quotient invariance, and conditional decidability

    Jul 11, 2026Cosimo Spera, Ray GarciaSemantic Caching

  10. Risk-Constrained Freshness-Aware Semantic Caching for Open-Web Retrieval-Augmented LLMs

    Jul 5, 2026Muhammad Mansoor, Tahir Ahmad, Yeo-Chan YoonLLM Inference EfficiencyRetrieval-Augmented Generation

  11. When Classic Cache Policies Fail: Learning-Augmented Replacement for Semantic Retrieval Buffers

    Jul 1, 2026Yushi Sun, Bowen Cao, Wai LamLearning-Augmented AlgorithmsSemantic Caching

  12. SeKV: Resolution-Adaptive KV Cache with Hierarchical Semantic Memory for Long-Context LLM Inference

    Jun 30, 2026Amirhossein Abaskohi, Giuseppe Carenini, Peter West +1KV-Cache OffloadingKV Caching

  13. Closing the Operational Gap in Semantic Caching

    Jun 18, 2026Aditeya Baral, Radoslav Ralev, Iliya Sotirov Zhechev +2LLM EvaluationCost-Aware Inference

  14. Hallucination Mitigation with Agentic AI, Nested Learning, and AI Sustainability via Semantic Caching

    May 27, 2026Diego Gosmar, Deborah A. DahlAI Agent ReliabilityMulti-Agent LLM Systems

  15. MVR-cache: Optimizing Semantic Caching via Multi-Vector Retrieval and Learned Prompt Segmentation

    May 24, 2026Ali Noshad, Zishan Zheng, Yinjun WuLLM Inference EfficiencyRL for Combinatorial Optimization

  16. Continuous Semantic Caching for Low-Cost LLM Serving

    Apr 21, 2026Baran Atalar, Xutong Liu, Jinhang Zuo +3LLM ServingCost-Aware Inference

  17. Local-Splitter: A Measurement Study of Seven Tactics for Reducing Cloud LLM Token Usage on Coding-Agent Workloads

    Apr 14, 2026Justice Owusu Agyemang, Jerry John Kponyo, Elliot Amponsah +2LLM Agent Workflow OptimizationLLM Inference Efficiency

  18. RedParrot: Accelerating NL-to-DSL for Business Analytics via Query Semantic Caching

    Mar 7, 2026Tong Wang, Yongqin Xu, Jianfeng Zhang +6Enterprise AnalyticsLLM Inference Acceleration

  19. Semantics-Aware Caching for Concept Learning

    Mar 6, 2026Louis Mozart Kamdem Teyou, Caglar Demir, Axel-Cyrille Ngonga NgomoConcept LearningSemantic Caching

  20. From Similarity to Vulnerability: Key Collision Attack on LLM Semantic Caching

    Jan 30, 2026Zhixiang Zhang, Zesen Liu, Yuchong Xie +2Adversarial AttacksLLM Agent Security