LLM Inference Efficiency

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

47 papers in the last four weeks, up 262% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 234

All topics

From "Who Is This User?" to "What Does This Purchase Mean?": A Deployed Pipeline for Semantic User Profiling at Bank Scale
Sep 17, 2026Ryota Mitsuhashi, Tetsuro Morimura, Hirotake ItoLLM Inference Efficiency Financial Services
CyberAgent, Tokyo, Japan
PrefixBench-H100: Characterizing Prefix Reuse and Time-to-First-Token in H100 LLM Serving
Sep 17, 2026Omkar Shewale, Deepak Kumar, Divakar Kumar YadavLLM Inference Efficiency LLM Inference
Illinois Institute of Technology · University of Wisconsin–Milwaukee
Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling
Sep 16, 2026Mobina Kashaniyan, Ali JannesariLLM Inference Efficiency LLM Inference
Department of Computer Science Iowa State University, USA
Ask the Tool, Don't Guess: Agent Tool Calls Hold Their Progress, and the Serving System Should Read It
Sep 16, 2026Yipeng Liu, Yingqiang Zhang, Feifei Li +1LLM Inference Efficiency KV Caching
Tsinghua University, China · Alibaba Cloud Computing, China · Zhejiang University, China
Dependency-Aware Trajectory Refinement for Efficient Multi-Turn Agent Fine-Tuning
Sep 16, 2026Zhuo Chen, Zhen Zhang, Xinyu Wang +1LLM Inference Efficiency LLM Agent Workflow Optimization
Shanghai Engineering Research Center of Intelligent Vision and Imaging
Where Should Agents Live? Energy-Memory Characterization of Agentic AI for the Edge-Cloud Continuum
Sep 16, 2026Carolina Fortuna, Vid Hanžel, Tim Strnad +1LLM Inference Efficiency Energy-Efficient ML
Department of Communication Systems, Jožef Stefan Institute, Jamova cesta 39, 1000 Ljubljana, Slovenia
When to Call an LLM: A Confidence-Gated Hybrid for Cost-Effective Emotion Recognition in Conversational AI
Sep 16, 2026Sai Babu Udayagiri, Arjun Chouhan, Ravisekhar Kanagala +1LLM Inference Efficiency Selective Prediction
Five9
The Inference Engineering Pareto Atlas: Which Optimizations Dominate the Cost, Quality, and Latency Frontier?
Sep 15, 2026Srikanta Datta Tumkur, Jay Iyer, Mehar Simhadri +3LLM Inference Efficiency LLM Quantization
Vizuara
Shared-Prefix KV Reuse Across Standard LoRA Adapters: Quality and Serving Tradeoffs
Sep 15, 2026Dushyant RajputLLM Inference Efficiency LLM Serving
AltSlate Labs LLP
Validating Hybrid-State Cache Recovery for GLM-5.3-Flash with vLLM and LMCache
Sep 14, 2026Frank LiLLM Inference Efficiency KV-Cache Management
UNSW Sydney
Dissecting GPU Utilization for LLM Inference on Nvidia Hopper
Sep 14, 2026Mohammad Siavashi, Gerald Q. Maguire Jr., Dejan Kostic +1LLM Inference Efficiency GPU Acceleration
KTH Royal Institute of Technology, Stockholm, Sweden
GGUF-Metadata Prediction of Single-Sequence llama.cpp Throughput Across Three Systems
Sep 14, 2026Xinyu Qiu, Chuhong Xu, Bo Su +3LLM Inference Efficiency
Northeastern University · Sofia University · Indiana University +2
Pull: Lazy Materialization of Working Memory for Stateful LLM Conversations
Sep 13, 2026Jiangang ChenLLM Inference Efficiency LLM Memory
Chengdu Beiluoshimen Technology Co., Ltd.
TriCalRAG: A Three-Strategy, Retrieval-Augmented Benchmark for On-Premise LLM-Based Root Cause Analysis in AIOps
Sep 13, 2026Rohit Patel, Susil Kumar Mohanty, Jeenal ChaudharyLLM Inference Efficiency Retrieval-Augmented Generation
Department of Computer Science and Engineering Indian Institute of Technology Jodhpur, Jodhpur, India
Phase-Decoupled, Model-Calibrated Power Control for Disaggregated LLM Serving
Sep 11, 2026Jae Gon Kim, Donghoon Yoo, Hanyul Ryu +3LLM Inference Efficiency Disaggregated LLM Serving
Xenoscube, Inc.
REVA: Reusable Evidence View Aggregation for Context-Efficient RAG Serving
Sep 11, 2026Tuan Nguyen, Qiran Hu, Banruo Liu +3LLM Inference Efficiency Retrieval-Augmented Generation
VinUni-Illinois Smart Health Center, VinUniversity, Vietnam · University of Illinois Urbana-Champaign, USA
From Fixed Keys to Readable Schemas: Small Language Models for Vehicle Agent Function Calls
Sep 10, 2026Hamed Jafarzadeh Asl, Yuanhao Yu, Vahid Partovi NiaLLM Inference Efficiency On-Device Language Model Inference
Huawei Noah’s Ark Lab, Canada · Huawei Noah’s Ark Lab · Polytechnique Montreal, Canada
Composable CXL Memory as a Kubernetes-Native Shared Memory for LLM Serving
Sep 9, 2026Hongjian Fan, Kevin Zhang, David Habinsky +1LLM Inference Efficiency KV Caching
Research Group, Seagate Technology, LLC
PELM: Power Efficient On-Device LLM Inference with Speculative Decoding and Dynamic Voltage Frequency Scaling
Sep 9, 2026Weisi Yang, Stephen XiaLLM Inference Efficiency On-Device Language Model Inference
Northwestern University Evanston, Illinois, USA
A Measurement Study of LLM Inference Trade-offs Across Edge Continuum Hardware
Sep 8, 2026Maysam Khatib, Moysis Symeonides, Demetris Trihinas +2LLM Inference Efficiency LLM Evaluation
University of Cyprus, Nicosia, Cyprus · University of Nicosia, Nicosia, Cyprus
MetaKV: Adaptive KV Cache Compression for Constrained LLM Inference
Sep 7, 2026Michael Wang, Keith Li, Roozbeh BostandoostLLM Inference Efficiency LLM Inference
Lake Washington School District · XSchool · UMass Amherst
Signed Rescue Routing: Harm-Aware Cascades for Efficient LLM Inference
Sep 7, 2026Zheyuan Wang, Siyu Li, Peiqiao Song +3LLM Inference Efficiency LLM Routing
School of Artificial Intelligence, Beijing Normal University · School of Information Science and Engineering, Chongqing Jiaotong University · School of Computer Science and Technology, Dalian University of Technology +1
What Else Needs Fixing? Exploring Cost-Effective Test-Time Compute for Revision Propagation in Artifacts Generated Through Conversation
Sep 3, 2026Daisuke KikutaLLM Inference Efficiency LLM Evaluation
NTT, Inc.
LeanStream: A Speculate-and-Refine Streaming Framework for Efficient on-Device LLM Inference
Sep 2, 2026Renyuan Liu, Yuyang Leng, Kaiyan Liu +7LLM Inference Efficiency On-Device Language Model Inference
Richard · George Mason University · Global Technology Applied Research, JPMorganChase +1
Codebook Agent: Amortized Topology Design for LLM Multi-Agent Systems
Sep 2, 2026Jinxi Yu, Yubei Li, Eric Hanchen Jiang +6Multi-Agent LLM Systems LLM Inference Efficiency
University of California, Los Angeles
How Do Prompt Variations Affect Energy Consumption in On-Device LLMs?
Sep 1, 2026Wei Hu, Xiaolong Tu, Dawei Chen +3LLM Inference Efficiency On-Device Language Model Inference
Georgia State University · Toyota Motor North America
TRIAGE: Three-level Routing and Intelligent Agent Guidance for Efficient Execution
Sep 1, 2026Ruocan WeiLLM Inference Efficiency LLM Agent Skill Learning
China Telecom Cloud, Beijing, China
Cheap Verifiers, Large Blind Spots: Measuring the Reliability Cost of Cost-Saving Cascades
Sep 1, 2026Dushyant RajputLLM Inference Efficiency LLM Answer Verification
AltSlate Labs LLP
ContextPipe: Database-Inspired Context Assembly for Long-Horizon Agents
Sep 1, 2026Peng Xu, Zuyu Zhang, Yuze Sun +3LLM Inference Efficiency Long-Horizon LLM Agents
MatrixOrigin · Tsinghua University
What It Costs to Compose, Rebuild, and Correct Precomputed Memory
Aug 31, 2026Asa ShepardLLM Inference Efficiency Memory-Augmented Language Models
Department of Computer Science, Williams College

Sep 17, 2026, cs.AIJ/K move · Enter open · S save

From "Who Is This User?" to "What Does This Purchase Mean?": A Deployed Pipeline for Semantic User Profiling at Bank Scale

Ryota Mitsuhashi, Tetsuro Morimura, Hirotake Ito

CyberAgent, Tokyo, Japan

LLM Inference Efficiency Financial Services User Modeling

Open paper AskPDF arXiv

Per-user LLM inference on transaction histories binds the inference budget linearly to user count, which becomes prohibitive at applied scale. We re-cast attribute inference from per-user to per-transaction-pattern. The pipeline runs in three phases: Resolve abstracts item names with optional web grounding, Profile infers attributes for each frequent pattern, and Tag clusters free-text attributes into a queryable database. In Profile, a single LLM call per pattern emits predefined categorical labels, free-text attributes, and per-attribute prevalence estimates. Because inference runs over patterns rather than users, the budget grows with the pattern count rather than the user count. On the public Open e-commerce corpus, the database is statistically indistinguishable from an LLM that reads each user's raw history directly in AUC across the evaluated attributes, and the prevalence estimates carry discriminative signal between positive and negative users. The pipeline is deployed at a major Japanese bank profiling on the order of tens of millions of users, with close to a three-order-of-magnitude reduction in LLM inference targets versus a per-user pipeline. The code is publicly available on https://github.com/CyberAgentAILab/profiling-agent-open-ecommerce.

Momentum

Related topics

Latest papers 234