Throughput

Follow this topic Ask about this topic

Recent momentum

-80%

1 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-14

1 new paper

A weekly snapshot of new work published in Throughput.

47 papers

Latest in Throughput

Open your feed →

GGUF-Metadata Prediction of Single-Sequence llama.cpp Throughput Across Three Systems
Sep 14, 2026Xinyu Qiu, Chuhong Xu, Bo Su +3Throughput
Phase-Decoupled, Model-Calibrated Power Control for Disaggregated LLM Serving
Sep 11, 2026Jae Gon Kim, Donghoon Yoo, Hanyul Ryu +3Throughput
RoutePack: Expert Placement and Attention-Aware Data Packing for MoE Reinforcement Learning
Aug 12, 2026Yibo Shen, Xudong Han, Xiaowei Zhu +2Multiple Model Experts Throughput
TideRL: Boosting Agentic RL Goodput with Readiness-Aware Scheduling
Aug 11, 2026Yanyu Ren, Xizheng Wang, Xiao Liu +8Safe Reinforcement Learning Throughput
AFD-Ledger: Deployment Provisioning for Attention--FFN Disaggregation
Aug 5, 2026Chengyu Qiu, Xiao Fu, Fengcun Li +6Throughput Per-Token Latency
Schedule-Informed Temporal Fusion Forecasting of Hourly Airport Security-Checkpoint Throughput
Aug 3, 2026Yinxiao Zhang, Sen Wang, Yi GaoAir Traffic Control Queueing
Tevatron Meets Megatron: Expert-Parallel LLM Reranker Training on an Academic Budget
Aug 2, 2026Zhichao Xu, Xueguang Ma, Shengyao Zhuang +5Cross-Encoder Reranker Cold Mixture-Of-Expert Models
TAPAS: Throughput-adaptive Perception for Autonomous Systems
Jul 19, 2026Aman Vyas, Vasista Kodumagulla, Zain Taufique +2Perception Trajectory-Based Representations
A Multi-Agent System for 5G Throughput Prediction in Multi-Operator Urban Environments
Jul 18, 2026Muhammad Kabeer, Rosdiadee Nordin, Nadiva Nuriftitah +1Mobile Networks Decentralized Multi-Agent System
Long-Context Fine-Tuning with Limited VRAM
Jul 16, 2026Vladimir Fedosov, Aleksandr Sazhin, Artemiy Grinenko +1Efficient Long-Context Inference Dynamic Sparse Attention
ACID: Adaptive Caching for vIDeo generation
Jul 14, 2026Om Agrawal, Saurabh Agarwal, Aditya AkellaVideo Diffusion Models Cache
BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving
Jul 9, 2026Yuanjie Zhu, Liangwei Yang, Ke Xu +4Throughput
Breaking Database Lock-in: Agentic Regeneration of High Performance Storage Readers for Database Bypass
Jul 8, 2026Victor Giannakouris, Immanuel TrummerText-To-Sql Data Lakes
Adaptive Inference Batching using Policy Gradients
Jul 6, 2026Ruslan SharifullinThroughput
A Workflow-Aware Serving Layer for Agentic Applications
Jul 3, 2026Jiayi Qian, Zishen Wan, Hanchen Yang +3Agentic Workflows Workflow Automation and Optimization
Fine-Grained Computation Offload for Off-the-Shelf Servers in Tens of Lines
Jul 2, 2026Bojie LiOffloading Throughput
BaseRT: Best-in-Class LLM Inference on Apple Silicon via Native Metal
Jul 1, 2026Prabod Rathnayaka, Fabian Waschkowski, Lukas WesemannLLM Inference Efficiency Throughput
DiLaServe: High SLO Attainment Serving for Diffusion Language Models
Jun 27, 2026Tzu-Tao Chang, Benjamin Yuanyang Hong, Kiet Pham +1Large Language Model Serving Throughput
Service-Induced Congestion in Memory-Constrained LLM Serving
Jun 14, 2026Ruicheng Ao, Jing Dong, Gan Luo +1Large Language Model Serving Throughput
Resource-aware Computation-Communication Overlap for multi-GPU ML Workloads
Jun 8, 2026Minyu Cui, Miquel PericasMulti-Gpu Systems Throughput
Vortex: Efficient and Programmable Sparse Attention Serving for AI Agents
Jun 4, 2026Zhuoming Chen, Xinrui Zhong, Qilong Feng +5Dynamic Sparse Attention Throughput
SparDA: Sparse Decoupled Attention for Efficient Long-Context LLM Inference
Jun 3, 2026Yaosheng Fu, Guangxuan Xiao, Xin Dong +2Dynamic Sparse Attention LLM Inference Efficiency
UltraEP: Unleash MoE Training and Inference on Rack-Scale Nodes with Near-Optimal Load Balancing
Jun 2, 2026Xinming Wei, Chao Jin, Tuo Dai +10Modern Mixture-Of-Experts Throughput
Exploiting In-Sensor Computing for Energy-Efficient Earth Observation
May 31, 2026Luigi Capogrosso, Pietro Bonazzi, Loris Hoxhaj +1Cloud-Aware Earth Observation Sequences Earth Observation
Threshold-Based Exclusive Batching for LLM Inference
May 30, 2026Weifang Zhang, Yuzhou Nie, Bowen Pang +2Throughput LLM Inference Efficiency
MLIPilot: LLM-Driven Auto-Research for Machine-Learned Interatomic Potentials
May 29, 2026Etinosa Osaro, Santosh Adhikari, Stamatia Zavitsanou +2Interatomic Potentials Autoresearch
How Far Can Disaggregation Go? A Design-Space Exploration of Attention-FFN Disaggregation for Efficient MoE LLM Serving
May 27, 2026Hanjiang Wu, Abhimanyu Rajeshkumar Bambhaniya, Sarbartha Banerjee +9Mixture-Of-Expert Inference Throughput
Demystifying the Mythos or Disrupting Bugonomics? From Zero-Day Asymmetry to Defender Remediation Throughput
May 23, 2026Alfredo Pesoli, Herman Errico, Lorenzo CavallaroVulnerability Detection & Repair Remediation
MAxLM: Multi-Agent Language Model-Based Scheduling and Resource Allocation in MU-MIMO-OFDMA-Enabled Wireless Networks
May 15, 2026Adnan Quadri, Hongxiang LiOrthogonal Frequency-Division Multiplexing Wireless Networks
The Illusion of Power Capping in LLM Decode: A Phase-Aware Energy Characterisation Across Attention Architectures
May 12, 2026Bole Ma, Ayesha Afzal, Jan Eitzinger +1Large Language Model Decoding Throughput
Single-Thread JPEG Decoder Benchmarks Mis-Evaluate ML Data Loaders
May 9, 2026Vladimir Iglovikov, Dmitry KosarevskyThroughput Pytorch
ReLibra: Routing-Replay-Guided Load Balancing for MoE Training in Reinforcement Learning
May 9, 2026Chao Jin, Xinming Wei, Yinmin Zhong +6Balanced Learning Large Language Model Training
Requests of a Feather Must Flock Together: Batch Size vs. Prefix Homogeneity in LLM Inference
May 7, 2026Saksham Rathi, Preeti, Mythili VutukuruLLM Inference Efficiency Throughput
One Pool, Two Caches: Adaptive HBM Partitioning for Accelerating Generative Recommender Serving
May 6, 2026Wenjun Yu, Shuguang Han, Amelie Chi ZhouGraphics Processing Unit Memory Adaptive Scheduling
Caliper-in-the-Loop: Black-Box Optimization for Hyperledger Fabric Performance Tuning
May 4, 2026Yash Madhwal, Arseny Bolotnikov, Mark Prikhno +5Black-Box Optimization Bayesian Optimization
SPECTRE: Hybrid Ordinary-Parallel Speculative Serving for Resource-Efficient LLM Inference
May 4, 2026Jincheng Xie, Yawen Ling, Qi Xiao +4Large Language Model Serving Speculative Decoding
Benchmarking KV-Cache Optimizations across Task Quality and System Performance for Long-Context Serving
May 3, 2026Nikita Agrawal, Ruben MayerKey-Value Cache Compression Large Language Model Serving
Human-Less LLM Serving: Quantifying the Human Tax on Throughput
May 3, 2026Jianhui Lian, Li Chen, Dan Li +1Throughput Large Language Model Serving
SCALE-LoRA: Auditing Post-Retrieval LoRA Composition with Residual Merging and View Reliability
May 2, 2026Shuaipeng Zhou, Yu ZhangLow-Rank Adapters Adaptive-Rank Low-Rank Adaptation Baselines
SURGE: SuperBatch Unified Resource-efficient GPU Encoding for Heterogeneous Partitioned Data
May 1, 2026Shashank Kapadia, Deep Narayan Mishra, Sujal Reddy Alugubelli +3Throughput Granularity
Unifying Sparse Attention with Hierarchical Memory for Scalable Long-Context LLM Serving
Apr 29, 2026Zihan Zhao, Baotong Lu, Shengjie Lin +8Dynamic Sparse Attention Graphics Processing Unit Memory
Leverage Laws: A Per-Task Framework for Human-Agent Collaboration
Apr 27, 2026Stan LoosmoreHuman-Ai Collaboration Task-Level
GLM-5 Serving Parameter Tuning for OpenClaw: Single-Deployment MaaS Inference Optimization for Long-Context Agent Workloads
Apr 27, 2026Minjie Hua, Ning Wang, Peijun Yang +2Throughput Openclaw
Latency and Cost of Multi-Agent Intelligent Tutoring at Scale
Apr 27, 2026Iizalaarab Elhaimeur, Nikos ChrisochoidesMulti-Agent Large Language Model Systems Low-Latency Systems
MTServe: Efficient Serving for Generative Recommendation Models with Hierarchical Caches
Apr 24, 2026Xin Wang, Chi Ma, Shaobin Chen +14Recommendation Architecture Graphics Processing Unit Memory
Data Driven Optimization of GPU efficiency for Distributed LLM-Adapter Serving
Feb 27, 2026Ferran Agullo, Joan Oliveras, Chen Wang +5Large Language Model Serving Throughput
Walrus: A Cross-Domain Foundation Model for Continuum Dynamics
Nov 19, 2025Michael McCabe, Payel Mukhopadhyay, Tanya Marwah +22Wireless Foundation Models Fluid Dynamics

Sep 14, 2026 · cs.AIJ/K move · Enter open · S save

GGUF-Metadata Prediction of Single-Sequence llama.cpp Throughput Across Three Systems

Xinyu Qiu, Chuhong Xu, Bo Su, Ziyao Chen+2

We predict single-sequence model throughput from GGUF metadata using roofline-shaped predictors with quantization-specific scale factors fitted on reference models. The scored cohort comprises 318 phase-depth measurements from 53 host-file configurations on two Apple M4 Max systems and an NVIDIA RTX 5080. On host-specific held-out sets of four, five, and two configurations, an active-parameter decode model obtains 13.1%, 14.4%, and 36.1% mean absolute percentage error (MAPE), versus 49.4%, 55.3%, and 51.9% when charging total parameters. Leave-one-host-out coefficients fitted on the other two systems yield 11.6%, 16.8%, and 36.0% test MAPE. A low-bit model ladder changes ordering across runtime stacks. The P2 prefill baseline gives 18.7%, 22.2%, and 108.2% test MAPE. GGUF structure helps on all three systems, but fitted efficiencies are not universal.

Throughput

Open paper Ask about it PDF