LLM Inference Efficiency

LLM: Large Language Model

Ask about this topic

Momentum

47 papers in the last four weeks, up 262% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 234

All topics
CardsList

  1. ProgRouter: Online Progress-Guided Orchestration for Multi-Agent LLM Workflows under Quality-Cost Tradeoffs

    Aug 26, 2026Songyuan Li, Ahmed M. Abdelmoniem, Shiqiang WangLLM Inference EfficiencyMulti-Agent Orchestration

    Aston University, UK · Queen Mary University of London, UK · University of Exeter, UK

  2. The Embedder's Dilemma: LLMs Are Better, but at What Cost?

    Aug 13, 2026Adnan El Assadi, Niklas Muennighoff, Jinhyuk LeeLLM Inference EfficiencyLLM Evaluation

    Harvard University · Stanford University · Independent Researcher

  3. Lifecycle-Optimal Tokenization: Vocabulary Size as a Deployment-Regime-Dependent Infrastructure Parameter

    Aug 11, 2026Rima Mittal, Ankit Gubrani, Satyanarayana KakolluLLM Inference EfficiencyLLM Serving

  4. TSPORec: Token Selection via Preference Optimization for LLM-Based Sequential Recommendation

    Aug 10, 2026Wenqiao Zhu, Chao Xu, Haipang Wu +1LLM Inference EfficiencySequential Recommendation

  5. Test-Time Augmentation for LLMs: When Input Diversity Beats Output Diversity at Matched Compute

    Aug 10, 2026Nikita Kozodoi, Zainab Afolabi, Jack ButlerLLM Inference EfficiencySelf-Consistency Decoding

    Amazon Web Services

  6. Measuring and Reducing WebGPU Dispatch Overhead for LLM Inference

    Aug 9, 2026Jędrzej MaczanLLM Inference EfficiencyLLM Inference

    1Independent Researcher

  7. LLM within MCP Matters: Measuring Inefficient Resource Utilization Driven by LLMs

    Aug 9, 2026Minhan Cho, Soyoung Park, Kihyeon Jeong +3LLM Inference EfficiencyInstruction Following

    Remember & Company AI Lab · Sungkyunkwan University · National Assembly Research Service +2

  8. Blast Radius

    Aug 7, 2026MY Pitsane, Hope MogaleLLM Inference EfficiencyAgentic Code Generation

    Algorithm Reconnaissance Division Mankind Research Labs North-West University, RSA · Algorithm Reconnaissance Division Mankind Research Labs University of Pretoria, RSA

  9. A Two-Tier Perspective on Inference-Time Parallelism in Multi-Agent LLM Systems

    Aug 6, 2026Zihan Xu, Haolin Tian, Hai JiangLLM Inference EfficiencyMulti-Agent LLM Systems

    Beijing University of Posts and Telecommunications.

  10. Interpretable Adaptive Sampling for LLM Test-Time Scaling

    Aug 4, 2026Mobina Kashaniyan, Ali JannesariLLM Inference EfficiencyTest-Time Scaling

    Iowa State University Ames, IA, USA

  11. Pin Once, Swap Light: Subspace-Aligned Centroid-Residual Training for Efficient Ultra-LoRA Serving

    Aug 4, 2026Xiang Li, Pengcheng Wang, Huazheng Wang +1LLM Inference EfficiencyLLM Serving

    1Purdue University · 2Oregon State University

  12. Same Task, Different Work: Prompt-Induced Waste in Coding Agents

    Aug 2, 2026Sarel Weinberger, Amir HozezLLM Inference EfficiencyOverthinking in Language Models

    PointFive

  13. Stop When Memory Suffices: Evidence-Conditioned Progressive Execution for LLM Agents

    Aug 2, 2026Yidan Lin, Kaixiang Wang, Jiong Lou +1LLM Inference EfficiencyEfficient Inference

  14. IR2Solve: Structured Intermediate Representations for Cost-Efficient Optimization Autoformulation

    Jul 31, 2026Penglin Zhu, Linhai Zhang, Jungang Xu +2LLM Inference EfficiencyNatural Language to Optimization Modeling

    School of Computer Science and Technology, University of the Chinese Academy of Sciences

  15. Think Short, Defer Smart, Act, and Repeat: Calibrated Reasoning and Uncertainty-Aware Deferral for Edge LLM Agents

    Jul 29, 2026Amirmohammad Farzaneh, Osvaldo SimeoneLLM Inference EfficiencyAI Agent Reliability

    Institute for Intelligent Networked Systems (INSI) Northeastern University London London, UK

  16. From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs

    Jul 29, 2026Tina Vartziotis, Rodopi Kosteli, Elli Vartziotis +5LLM Inference EfficiencyEnergy-Efficient ML

    1National Technical University of Athens, Patission Complex 42, 10682 Athens, Greece · 2Harvard University, 1350 Massachusetts Avenue, 02138 Cambridge, MA, USA · 3TWT GmbH Science & Innovation, Industriestraße 6, 70565 Stuttgart, DE +4

  17. LLMET: Enabling Cross-Layer Evaluation of Emerging M3D Memories for Energy-Efficient LLM Serving

    Jul 29, 2026Ming-Yen Lee, Hanchen Yang, Faaiq Waqar +4LLM Inference EfficiencyLLM Serving

    Georgia Institute of Technology, Atlanta, GA, USA · Samsung Semiconductor Inc., San Jose, CA, USA

  18. Conformal Cascade: Distribution-Free Accuracy Guarantees for Multi-Tier LLM Inference

    Jul 27, 2026Yifan Dou, Shikan Lian, Shibo LiLLM Inference EfficiencyLLM Inference Scheduling

    Department of Computer Science, Florida State University

  19. When LLM Defenses Backfire: Characterizing Safety, Performance, and Cost Trade-offs

    Jul 27, 2026Tong Zhang, Zexin Li, Simin Chen +1LLM Inference EfficiencyLanguage Model Safety Evaluation

    Fudan University · University of California, Riverside · Columbia University

  20. SpecBox: Speculative Sandbox Scheduling for Efficient LLM Agent Serving

    Jul 27, 2026Yihui Zhang, Tianyu Wo, Jinghao Wang +7LLM Inference EfficiencyLLM Agent Serving

    Beihang University · University of Leeds · The University of Sydney

  21. Understanding Tone-Dependent Inference Cost in Large Language Models

    Jul 27, 2026Akhil Kumar, Om DobariyaLLM Inference EfficiencyLLM Evaluation

    Pennsylvania State University University Park, PA, USA

  22. Profiling Lightweight Large Language Models

    Jul 23, 2026Tomohiro Harada, Enrique Alba, Gabriel LuqueLLM Inference EfficiencyLLM Evaluation

    Graduate School of Science and Engineering, Saitama University, Saitama, Japan

  23. CORVUS: Context Optimization and Reduction Via Underlying Synchronization for LLM Coding Agents

    Jul 20, 2026Mingwei Zheng, David OBrien, Siwei Cui +4LLM Inference EfficiencyAI Coding Agents

    Department of Computer Science, Purdue University · 2AWS AI Labs

  24. VDAR-Router: Adaptive LLMs Routing via Verbalized Query Difficulty Analysis Retrieval

    Jul 20, 2026Yu-Chien Tang, Jun-Chen Hung, Wen-Chih Peng +1LLM Inference EfficiencyAdaptive Model Routing

    National Yang Ming Chiao Tung University · Department of Computer Science Hsinchu, Taiwan

  25. Human Grounded Evaluation of Large Language Models for Optical Network Automation

    Jul 20, 2026Kiarash Rezaei, Omran Ayoub, Paolo Monti +1LLM Inference EfficiencyLLM Evaluation

    Department of Electrical Engineering, Chalmers University of Technology, 412 96 Gothenburg, Sweden · University of Applied Sciences and Arts of Southern Switzerland, 6928 Lugano, Switzerland

  26. Constrained Path Reasoning: Measuring When Committed Stages Earn Their Cost

    Jul 19, 2026Honglin LiLLM Inference EfficiencyInference-Time Search

    ShanghaiTech University

  27. WaterMoE: Expert-Routing-based Watermarking for High Fidelity and Efficiency

    Jul 14, 2026Z Sun, Q Jiang, S Sheng +1LLM Inference EfficiencyLLM Watermarking

    Shanghai Jiao Tong University · Shanghai Innovation Institute · Tianjin University

  28. Securing LLMs in the Wild: Privacy and Security Challenges at the Edge

    Jul 13, 2026Ren-Yi Huang, Mingchen Li, Dumindu Samaraweera +1LLM Inference EfficiencyEdge Computing

    Department of Electrical and Computer Engineering, University of South Florida · Department of Mathematics, Embry-Riddle Aeronautical University

  29. Valid ≠\ne Necessary: Diagnosing Latent Inefficiency in Chain-of-Thought

    Jul 13, 2026Daeyeop Lee, Hwanjo YuLLM Inference EfficiencyOverthinking in Language Models

    1KT Corporation · 2Pohang University of Science and Technology

  30. Attention to Detail: Evaluating Energy, Performance, and Accuracy Trade-offs Across vLLM Configurations

    Jul 10, 2026Nada Zine, Tristan Coignion, Vincenzo Stoico +3LLM Inference EfficiencyLLM Evaluation

    Univ. Lille, CNRS, Inria · Univ. Bordeaux, CNRS, LaBRI · Vrije Universiteit Amsterdam +1