Graphics Processing Unit Resources

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, up 75% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 30

All topics

Decode-Latency Feedback Prefill: A Model-Free Controller and Its Generalization Limits
Sep 29, 2026Gaurav Agarwal, Ashish Garg, Isha SinghalTime-To-First-Token Graphics Processing Unit Resources
Towards Certificate-Driven Software Porting: A Self-Improving Agentic Harness for Scientific Program Optimization
Sep 28, 2026Piyush Jha, Aishik Ghosh, Vijay GaneshEvolutionary Search Code Optimization
School of Computer Science, Georgia Institute of Technology, USA · School of Physics, Georgia Institute of Technology, USA · Lawrence Berkeley National Laboratory, USA
Validating Memory-Optimal Transformer Kernels on Real Hardware: From Formal Derivation to Measured Performance Across Two HPC Clusters
Sep 27, 2026Lenore M. Mullin, Gaetan HainsGraphics Processing Unit Resources High-Performance Computing
Professor Emerita, College of Nanotechnology, Science, and Engineering, University at Albany (SUNY), Albany, NY, USA.
ASCEND: Personal AI Agents for Autonomous Scientific Computing Across HPC Clusters and GPU Workstations
Sep 26, 2026J. Paul Liu, Uthpala Herath, Andrew PetersenHigh-Performance Computing Artificial Intelligence Scientists
AI Hub for Science, College of Sciences, NC State University, Raleigh, NC · Research Computing and Support Services, Office of Information Technology, Duke University, Durham, NC · Office of Information Technology, NC State University, Raleigh, NC
EMA: Elastic and Performance Transparent Memory Across GPUs
Sep 22, 2026Yi Xu, Tian Xia, Ion StoicaGraphics Processing Unit Resources Peak Memory
UC Berkeley
A principled approach for energy-efficient training via phase-aware GPU frequency tuning
Sep 21, 2026Miguel Braga, Júlio Pinto, Rahma Nouaji +5Energy-Efficient Graphics Processing Unit Resources
INESC TEC & U. Minho · McGill University
Dissecting GPU Utilization for LLM Inference on Nvidia Hopper
Sep 14, 2026Mohammad Siavashi, Gerald Q. Maguire Jr., Dejan Kostic +1Graphics Processing Unit Resources LLM Inference Optimization
KTH Royal Institute of Technology, Stockholm, Sweden
LeanStream: A Speculate-and-Refine Streaming Framework for Efficient on-Device LLM Inference
Sep 2, 2026Renyuan Liu, Yuyang Leng, Kaiyan Liu +7LLM Inference Optimization Streaming
Richard · George Mason University · Global Technology Applied Research, JPMorganChase +1
Verify Before You Distill: Prompt-Level Teacher Gating for On-Policy Distillation
Sep 2, 2026Zhiwei Zhang, Zechen Sun, Fei Zhao +6Efficient On-Policy Distillation Teacher
AllSpark Team
MeshSplatBench: A Unified Benchmark for Triangle- and Mesh-Based Neural Rendering
Sep 1, 2026Kaixuan Zhang, Minxian Li, Mingwu Ren +1Neural Rendering Graphics Processing Unit Resources
Nanjing University of Science and Technology · State Key Laboratory of Intelligent Manufacturing of Advanced Construction Machinery · University of Surrey
Workload Identification with Physical Side Channels for AI Governance
Aug 31, 2026Simone Gargiulo, Gabriel KulpGraphics Processing Unit Resources Workload
Pivotal Research · Intelligence Security Laboratories
Gradient Under Microscope: Benchmarking Resource Utilization of Memory-Efficient Gradient Computation Methods
Aug 9, 2026Sarthak Mahapatra, Zihan Zhou, Khatoon Khedri +2Gradient Adam
What Irregularity Costs: CUDA C++, Rust, and Triton on a Hash-Blocked GPU Workload
Aug 8, 2026Petr KorolevCuda Graphics Processing Unit Resources
Spacial Intelligence Labs
Energy-Efficient LLM Serving via Disaggregated Attention--FFN and Flexible Frequency Scaling
Aug 3, 2026Cunchen Hu, Liangliang Xu, Tian Liu +9Graphics Processing Unit Resources Rate Scaling
China Telecom Cloud Computing Research Institute · Xidian University · University of Science and Technology of China +1
Device-First Feedback: Toward Mobile-Native LLM-Driven Neural Architecture Search
Jul 29, 2026Saif U Din, Muhammad Ahsan Hussain, Radu Timofte +1Large Language Model Fine-Tuning Neural Architecture Search
Computer Vision Lab, CAIDAS & IFI, University of Würzburg, Germany
Rendering on Real Silicon: GPU Render-Timing as a Passive, AI-Resistant CAPTCHA Signal
Jul 25, 2026David Noever, Forrest McKeeCaptcha Graphics Processing Unit Resources
PeopleTec, Inc.
How Fast Can Reward Models Score? A Systems Study of C++ and PyTorch Inference Runtimes for RLHF
Jul 22, 2026Venkata Naga Sai Vishnu Rohit Pulipaka, Anish Katta, Deva Rohit Reddy PeddireddyDestination-Only Scoring Fast Inference
The FIL Hypothesis: Inductive Biases Help with Kernel Engineering
Jun 29, 2026Nikolai Rozanov, Subhabrata Dutta, Preslav Nakov +1Inductive Bias Feedback Loop
1NLP Department, MBZUAI, Abu Dhabi, UAE · Department of Computing, Imperial College London, London, UK · 2Ubiquitous Knowledge Processing, TU Darmstadt, Darmstadt, Germany
Accelerating Disaggregated RL for Visual Generative LLMs with Diffusion-Based Parallelism and Trainer-Assisted Generation
Jun 23, 2026Sijie Wang, Zhengyu Qing, Zhiqiang Tan +6Diffusion-Based Reinforcement Learning Methods Diffusion Language Models
School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen · Data Science and Analytics Thrust, The Hong Kong University of Science and Technology (Guangzhou) · Huawei Technologies Ltd.
HERALD: High-Throughput Block Diffusion LLM Serving via CPU-GPU Cooperative KV Cache Retrieval
Jun 19, 2026Omin Kwon, Doyeon Kim, Jongseok Park +3LLM Inference Optimization Diffusion Language Models
Seoul National University Seoul, Republic of Korea · UC Berkeley Berkeley, CA, USA
SSH-Net: A Deep Neural Network for Predicting Failure Time Distribution Functions under Competing Risks with Application to GPU Data
Jun 18, 2026Jie Min, Yueyao Wang, Mengkun ChenTime-To-Event Data Risk
Department of Mathematics & Statistics, University of South Florida, Tampa, FL · School of Statistics and Data Science, Zhejiang Gongshang University, Hangzhou, China · Department of Statistics,AbstractVirginia Tech, Blacksburg, VA
Detecting Hidden ML Training With Zero-Overhead Telemetry
Jun 17, 2026Robi Rahman, Sabiha TajdariGraphics Processing Unit Resources Nvidia
Machine Intelligence Research Institute · University of Virginia
ViBE: Co-Optimizing Workload Skew and Hardware Variability for MoE Serving
May 30, 2026Seokjin Go, Marko Scrbak, Ephrem Wu +2Mixture-Of-Expert Inference Inference Workloads
Georgia Institute of Technology Atlanta, GA, USA · Advanced Micro Devices, Inc. Austin, TX, USA · Advanced Micro Devices, Inc. Santa Clara, CA, USA +1
ScaleAcross Explorer: Exploring Communication Optimization for Scale-Across AI Model Training
May 23, 2026Minghao Li, Alicia Golden, Samuel Hsia +14Large Language Model Training Graphics Processing Unit Resources
1Harvard University · 2Meta Platforms, Inc.
Instant GPU Efficiency Visibility at Fleet Scale
May 20, 2026Connor Pedersen, Dong H. Ahn, Michel Migdal +2Graphics Processing Unit Resources High-Performance Computing
NVIDIA
GEM: GPU-Variability-Aware Expert to GPU Mapping for MoE Systems
May 19, 2026Sourish Wawdhane, Avinash Kumar, Poulami DasMixture-Of-Expert Inference Mixture-Of-Experts
The University of Texas at Austin
Data Driven Optimization of GPU efficiency for Distributed LLM-Adapter Serving
Feb 27, 2026Ferran Agullo, Joan Oliveras, Chen Wang +5Cpu-Gpu Hybrid Designs Graphics Processing Unit Resources
Barcelona Supercomputing Center (BSC), Barcelona, Spain · Universitat Politècnica de Catalunya - BarcelonaTech (UPC), Barcelona, Spain · IBM Research, New York, USA
TensorCommitments: A Lightweight Verifiable Inference for Language Models
Feb 13, 2026Oguzhan Baser, Elahe Sadeghi, Eric Wang +5LLM Inference Optimization Cryptographic Merkle Tree Commitments
Electrical and Computer Engineering, The University of Texas at Austin · Theseus AI Labs · Electrical and Computer Engineering, McGill University +1
Saving GPU Hours in LLM Inference System Development and Online Workloads with Simulation and DBMS-Inspired Cache Replacement Policies
Nov 12, 2024Kyoungmin Kim, Jiacheng Li, Kijae Hong +3LLM Inference Optimization Graphics Processing Unit Resources
EPFL, Switzerland · NUS, Singapore · POSTECH, South Korea
NGD-SLAM: Towards Real-Time Dynamic SLAM without GPU
May 12, 2024Yuhao Zhang, Mihai Bujanca, Mikel LujánOrb-Slam2 Simultaneous Localization And Mapping
University of Manchester. 2University of Cambridge. · 3Qualcomm Technologies XR Labs, Austria.

Sep 29, 2026, cs.AIJ/K move · Enter open · S save

Decode-Latency Feedback Prefill: A Model-Free Controller and Its Generalization Limits

Gaurav Agarwal, Ashish Garg, Isha Singhal

Time-To-First-Token Graphics Processing Unit Resources Autoregressive Model Feedback Model-Free

Open paper AskPDF arXiv

Concurrent autoregressive inference creates a fundamental interference problem: prefilling a newly arrived long prompt can delay tokens for requests that are already decoding. Fixed prefill chunks reduce this interference, but the best chunk size depends on the model, hardware, load, and latency objective. We introduce Decode-Latency Feedback Prefill (DLFP), a model-free controller that changes only prefill work that overlaps active decodes. After a guarded scheduling cycle, DLFP uses the observed interval as proportional feedback to resize the next prefill chunk; isolated prefills remain unrestricted. We implement DLFP in vLLM and evaluate it with open-loop Poisson arrivals, exact token accounting, raw request traces, and NVIDIA telemetry. On Qwen3-0.6B in BF16 on one A100 80 GB GPU, three paired 100-request trials reduce P99 inter-token latency by 24.8%, 30.1%, and 28.2% (mean 27.7%, paired 95% confidence interval 21.0% to 34.3%) with exact output agreement, no failures, and unchanged SLO compliance. The benefit is not free: mean P99 time to first token increases 34.8% while remaining inside the declared SLO. Crucially, the mechanism does not generalize to Qwen3-8B, Qwen3-32B, or a two-GPU tensor-parallel configuration. We trace the failure to an asynchronous scheduler-call interval that is only a proxy for completed GPU iteration time. This negative result defines the boundary of the contribution and motivates a completion-timed controller for concurrent CPU and on-device inference. We do not claim mobile-device performance; the present work is a reproducible proof-of-concept and generalization study.

Momentum

Related topics

Latest papers 30