cs.PFNov 18, 2026

Measuring and Charging for Retained Context in Long-Context LLM Services

Authors: Yi Lu, Qianshi Lin, Yumeng Yang, Hongxia Wang

Organizations: School of Statistics and Data Science, Nanjing Audit University, Nanjing 211815, Jiangsu, China

Abstract

Long-context large language model (LLM) services charge for repeated context, yet billing weights need not reflect its contribution to answer quality. We present a three-layer framework separating content relevance, observable proxies and auditable positional weights. Context-ablation experiments show that relevance dominates position and that distance-decay families do not outperform a relevance-only baseline. A measured serving stack shows that cache hits reduce repeated-prefix compute cost by over two orders of magnitude at 65,536 tokens, while cold resends constrain sustainable discounts. Cross-provider measurements separate cache value from tokenisation and list-price differences. Calibrated simulations reduce the 99th percentile of session charges by 31-42%, although revenue-matched nonlinear tariffs achieve similar reductions and median charges increase. The framework provides a bounded per-token accounting rule and measured deployment constraints, rather than a validated utility estimator.

Figures & tables

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. The Efficiency Frontier: A Unified Framework for Cost-Performance Optimization in LLM Context Management

    May 21, 2026Binqi Shen, Lier Jin, Hanyu Cai +2LLM EvaluationCost-Aware Inference