Intensive Operator

Intensive operator research focuses on optimizing the execution of computationally demanding operations within deep learning models, primarily aiming to improve inference speed and efficiency on resource-constrained hardware like GPUs and edge devices. Current efforts concentrate on parallelizing operator execution, intelligently scheduling operator launch order to minimize resource contention, and developing hardware-aware optimization techniques including quantization and operator replacement. This research significantly impacts the deployment of large-scale deep learning models in various applications, from natural language processing and advertising systems to resource-limited embedded systems, by reducing latency and energy consumption.

Papers

December 16, 2023

Opara: Exploiting Operator Parallelism for Expediting DNN Inference on GPUs
Aodong Chen, Fei Xu, Li Han, Yuan Dong, Li Chen, Zhi Zhou, Fangming Liu
Single GPU DNN Inference Inference Workload Intensive Operator

November 28, 2023

Data-efficient operator learning for solving high Mach number fluid flow problems
Noah Ford, Victor J. Leon, Honest Mrema, Jeffrey Gilbert, Alexander New
Supervised Baseline Surface Parameterization Intensive Operator

October 26, 2023

ZeroQuant-HERO: Hardware-Enhanced Robust Optimized Post-Training Quantization Framework for W8A8 Transformers
Zhewei Yao, Reza Yazdani Aminabadi, Stephen Youn, Xiaoxia Wu, Elton Zheng, Yuxiong He
Neural Network Post Training Quantization Quantization Technique Dynamic Quantization Transformer XL W4A8 Quantization Intensive Operator

September 20, 2023

Grassroots Operator Search for Model Edge Adaptation
Hadjer Benmeziane, Kaoutar El Maghraoui, Hamza Ouarnoughi, Smail Niar
Deep Learning Hardware Aware Neural Architecture Search Efficient Neural Network Architecture Intensive Operator

April 10, 2023

Deploying Machine Learning Models to Ahead-of-Time Runtime on Edge Using MicroTVM
Chen Liu, Matthias Jobst, Liyuan Guo, Xinyue Shi, Johannes Partzsch, Christian Mayr
Machine Learning Code Generation Extreme Edge Hand Gesture Recognition Runtime Analysis AI Application Phantom 2D Accelerator Intensive Operator

September 26, 2022

FeatureBox: Feature Engineering on GPUs for Massive-Scale Ads Systems
Weijie Zhao, Xuewu Jiao, Xinsheng Luo, Jingxue Li, Belhal Karimi, Ping Li
Single GPU Click Through Rate Prediction Feature Engineering Click Through Rate Intensive Operator

July 31, 2022

Building an Efficiency Pipeline: Commutativity and Cumulativeness of Efficiency Operators for Transformers
Ji Xin, Raphael Tang, Zhiying Jiang, Yaoliang Yu, Jimmy Lin
Transformer Megatron Decepticons Efficient Pipeline Efficient Method Cumulative Water Heater Intensive Operator

Intensive Operator

Papers

Opara: Exploiting Operator Parallelism for Expediting DNN Inference on GPUs

Data-efficient operator learning for solving high Mach number fluid flow problems

ZeroQuant-HERO: Hardware-Enhanced Robust Optimized Post-Training Quantization Framework for W8A8 Transformers

Grassroots Operator Search for Model Edge Adaptation

Deploying Machine Learning Models to Ahead-of-Time Runtime on Edge Using MicroTVM

FeatureBox: Feature Engineering on GPUs for Massive-Scale Ads Systems

Building an Efficiency Pipeline: Commutativity and Cumulativeness of Efficiency Operators for Transformers