Semantic Caching

Follow this topic Ask about this topic

Momentum

0 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 14

All topics

LipCache: A Local Inference Proxy with Certified Caching for Edge Image Classification Service
Aug 13, 2026Zhengzhe Xiang, Yinlin Chen, Fuli Ying +3Efficient Neural Network Inference Edge Inference
LaCache: Robust Semantic Caching for LLM Serving
Aug 3, 2026Jiacheng Liang, Yuhui Wang, Tanqiu Jiang +1LLM Serving LLM Security
Stony Brook University
From ambiguous utterances to governed reuse classes: canonicalization, quotient invariance, and conditional decidability
Jul 11, 2026Cosimo Spera, Ray GarciaSemantic Caching
Minerva CQ (Bourbaki Intelligent Systems, Inc.), Los Gatos, CA, USA
Risk-Constrained Freshness-Aware Semantic Caching for Open-Web Retrieval-Augmented LLMs
Jul 5, 2026Muhammad Mansoor, Tahir Ahmad, Yeo-Chan YoonLLM Inference Efficiency Retrieval-Augmented Generation
Jeju National University, Jeju, Republic of Korea
When Classic Cache Policies Fail: Learning-Augmented Replacement for Semantic Retrieval Buffers
Jul 1, 2026Yushi Sun, Bowen Cao, Wai LamLearning-Augmented Algorithms Semantic Caching
LIGHTSPEED, Tencent · Shenzhen, China · The Chinese University of Hong Kong +1
SeKV: Resolution-Adaptive KV Cache with Hierarchical Semantic Memory for Long-Context LLM Inference
Jun 30, 2026Amirhossein Abaskohi, Giuseppe Carenini, Peter West +1KV-Cache Offloading KV Caching
University of British Columbia · Microsoft Research
Closing the Operational Gap in Semantic Caching
Jun 18, 2026Aditeya Baral, Radoslav Ralev, Iliya Sotirov Zhechev +2LLM Evaluation Cost-Aware Inference
New York University · Redis
Hallucination Mitigation with Agentic AI, Nested Learning, and AI Sustainability via Semantic Caching
May 27, 2026Diego Gosmar, Deborah A. DahlAI Agent Reliability Multi-Agent LLM Systems
Head of AI, Tesisquare · Member, Open Voice Interoperability Initiative · Linux Foundation AI & Data +4
Grounded Cache Routing for Retrieval-Augmented Generation: When Is It Safe to Reuse an Answer?
May 26, 2026Syed Huma ShahRetrieval-Augmented Generation RAG Security
Duke University · Syed Huma Shah$^*$
MVR-cache: Optimizing Semantic Caching via Multi-Vector Retrieval and Learned Prompt Segmentation
May 24, 2026Ali Noshad, Zishan Zheng, Yinjun WuLLM Inference Efficiency RL for Combinatorial Optimization
School of Computer Science, Peking University, Beijing, China · School of Information, Renmin University of China, Beijing, China
CacheRAG: A Semantic Caching System for Retrieval-Augmented Generation in Knowledge Graph Question Answering
Apr 28, 2026Yushi Sun, Lei ChenRetrieval-Augmented Generation Graph Retrieval-Augmented Generation
HKUST Hong Kong, China · HKUST(GZ) / HKUST Guangzhou / Hong Kong, China
Continuous Semantic Caching for Low-Cost LLM Serving
Apr 21, 2026Baran Atalar, Xutong Liu, Jinhang Zuo +3LLM Serving Cost-Aware Inference
Carnegie Mellon University Pittsburgh, Pennsylvania, USA · University of Washington Tacoma Tacoma, Washington, USA · City University of Hong Kong China +1
From Similarity to Vulnerability: Key Collision Attack on LLM Semantic Caching
Jan 30, 2026Zhixiang Zhang, Zesen Liu, Yuchong Xie +2Adversarial Attacks LLM Agent Security
Department of Computer Science and Engineering, The Hong Kong University of Science and Technology · Fudan University
PACE: Perceived-Latency-Aware Cascading Service Routing and Filler Control for QoE-Efficient Retrieval-Augmented Dialogue Serving
Date pendingLin Huang, Yujuan Tan, Weisheng Li +4Retrieval-Augmented Generation Inference-Time Optimization
Chongqing University. No. 55, University Town South Road, Gaoxin District, Chongqing, 401331, China · Inspur Yunzhou Industrial Internet Co., Ltd. No.1036 Langchao Road, Lixia District, Jinan, Shandong, 250101, China. · Guoqi Zhimo (Chongqing) Technology Co., Ltd. 5th Floor, Building B15, Xiantao Data Valley, Yubei District, Chongqing, 401122, China. +4

Aug 13, 2026, cs.DCJ/K move · Enter open · S save

LipCache: A Local Inference Proxy with Certified Caching for Edge Image Classification Service

Zhengzhe Xiang, Yinlin Chen, Fuli Ying, Binbin Zhou+2

Efficient Neural Network Inference Edge Inference Semantic Caching Lipschitz Neural Networks Image Classification

Open paper AskPDF arXiv

As edge-side vision services continue to expand toward low-latency, high-throughput scenarios, reducing the inference cost of vision models without sacrificing reliability has become a central concern. Existing semantic caching methods largely rely on empirical similarity thresholds; while such thresholds improve hit rates, they tend to introduce silent misclassifications near decision boundaries. To address this, we propose \texttt{LipCache}, a certified semantic caching framework for image classification. Without modifying the existing deployed main model, \texttt{MainNet}, the framework introduces a lightweight network, \texttt{GuardNet}, that maps inputs into a low-dimensional feature space subject to a Lipschitz constraint. It then computes a per-sample certified reuse radius from the local classification margin and the spectral norm of the classification head. At runtime, a cached result is reused only when the query feature falls inside the certified reuse ball; otherwise, the query falls back to \texttt{MainNet}. Thus, cache hits are transformed from empirical threshold tests into geometric certification decisions with explicit theoretical boundaries. Across standard image classification tasks like CIFAR, Tiny-ImageNet, and SVHN, \texttt{LipCache} achieves a measured speedup of up to

1.65\times

with limited end-to-end accuracy degradation, while all accepted cache hits satisfy the \texttt{GuardNet}-side certified-consistency condition. Furthermore, an enhanced \texttt{GuardNet} training recipe substantially improves cache hit rates in the Tiny-ImageNet multi-class extension while maintaining a certified-consistency rate of

100\%

. These results demonstrate that per-sample certified reuse can reduce main-model fallback while preserving theoretical consistency, providing a feasible approach to reliable cache-assisted inference at the edge.

Momentum

Related topics

Latest papers 14