Bit Weight

Bit weight research focuses on reducing the precision of numerical representations in neural networks, aiming to improve efficiency and reduce resource consumption without significant accuracy loss. Current efforts concentrate on exploring different quantization techniques (e.g., INT4, INT8, FP8) for weights and activations within various architectures, including transformers (like BERT) and convolutional neural networks (like ResNet), often employing quantization-aware training or post-training quantization methods. This work is significant because it enables deployment of large models on resource-constrained devices (e.g., edge devices, embedded systems) and accelerates inference speed, impacting both the efficiency of deep learning applications and the accessibility of advanced AI technologies.

Papers

May 1, 2024

When Quantization Affects Confidence of Large Language Models?
Irina Proskurina, Luc Brun, Guillaume Metzler, Julien Velcin
Language Model Quantization Operator High Confidence Quantization Loss Bit Weight

March 31, 2023

FP8 versus INT8 for efficient deep learning inference
Mart van Baalen, Andrey Kuzmin, Suparna S Nair, Yuwei Ren, Eric Mahurin, Chirag Patel, Sundar Subramanian, Sanghyuk Lee, Markus Nagel, Joseph Soriaga, Tijmen Blankevoort
Deep Learning Device Inference Personality Classification Deep Learning Inference Artificial Intelligence Inference Bit Weight FP8 Format

January 31, 2023

Quantized Neural Networks for Low-Precision Accumulation with Guaranteed Overflow Avoidance
Ian Colbert, Alessandro Pappalardo, Jakoba Petri-Koenig
Neural Network Directed Accumulator Bit Weight

November 13, 2022

FullPack: Full Vector Utilization for Sub-Byte Quantized Inference on General Purpose CPUs
Hossein Katebi, Navidreza Asadi, Maziar Goudarzi
Sub Byte High Resolution Representation Bit Weight

July 15, 2022

Low-bit Shift Network for End-to-End Spoken Language Understanding
Anderson R. Avila, Khalil Bibi, Rui Heng Yang, Xinlin Li, Chao Xing, Xiao Chen
Convolutional Neural Network Deep Neural Network Low Bit Quantization End to End Spoken Language Bit Weight Power of Two Quantization Bit Shift Network

July 13, 2022

Sub 8-Bit Quantization of Streaming Keyword Spotting Models for Embedded Chipsets
Lu Zeng, Sree Hari Krishnan Parthasarathi, Yuzong Liu, Alex Escott, Santosh Kumar Cheekatmalla, Nikko Strom, Shiv Vitaladevuni
Quantization Operator Keyword Spotting Embedded System Quantization Technique Neural Network Accelerator Bit Weight Linear Quantization 8 Bit Quantization

April 7, 2022

ShiftNAS: Towards Automatic Generation of Advanced Mulitplication-Less Neural Networks
Xiaoxuan Lou, Guowen Xu, Kangjie Chen, Guanlin Li, Jiwei Li, Tianwei Zhang
Neural Network Automatic Generation Bit Weight Bit Shift Network

March 25, 2022

MKQ-BERT: Quantized BERT with 4-bits Weights and Activations
Hanlin Tang, Xipeng Zhang, Kai Liu, Jianchen Zhu, Zhanhui Kang
Language Model BERT Model Ticket BERT Targeted Activation Penalty Bit Weight

February 23, 2022

Bitwidth Heterogeneous Federated Learning with Progressive Weight Dequantization
Jaehong Yoon, Geon Park, Wonyong Jeong, Sung Ju Hwang
Heterogeneous Federated Learning Low Bit Efficient Dequantization Binary Weight Bit Weight

January 25, 2022

Bit-serial Weight Pools: Compression and Arbitrary Precision Execution of Neural Networks on Resource Constrained Processors
Shurui Li, Puneet Gupta
Neural Network Linear Compression Network Compression Bit Weight

December 23, 2021

Training Quantized Deep Neural Networks via Cooperative Coevolution
Fu Peng, Shengcai Liu, Ning Lu, Ke Tang
Deep Neural Network Training Data Cooperative Coevolution Bit Weight

Bit Weight

Papers

When Quantization Affects Confidence of Large Language Models?

FP8 versus INT8 for efficient deep learning inference

Quantized Neural Networks for Low-Precision Accumulation with Guaranteed Overflow Avoidance

FullPack: Full Vector Utilization for Sub-Byte Quantized Inference on General Purpose CPUs

Low-bit Shift Network for End-to-End Spoken Language Understanding

Sub 8-Bit Quantization of Streaming Keyword Spotting Models for Embedded Chipsets

ShiftNAS: Towards Automatic Generation of Advanced Mulitplication-Less Neural Networks

MKQ-BERT: Quantized BERT with 4-bits Weights and Activations

Bitwidth Heterogeneous Federated Learning with Progressive Weight Dequantization

Bit-serial Weight Pools: Compression and Arbitrary Precision Execution of Neural Networks on Resource Constrained Processors

Training Quantized Deep Neural Networks via Cooperative Coevolution