Memory-Efficient Integer-Only Training of Deep Neural Networks

Hazem Hesham Yousef Shalby; Fabrizio Pittorino; Manuel Roveri

cs.AIOct 12, 2026

Memory-Efficient Integer-Only Training of Deep Neural Networks

Authors: Hazem Hesham Yousef Shalby, Fabrizio Pittorino, Manuel Roveri

Organizations: Dipartimento di Elettronica, Informazione e Bioingegneria (DEIB), Politecnico di Milano, Milan, Italy

Abstract

Training neural networks in reduced precision can yield significant efficiency gains, yet all existing methods that scale to standard computer vision benchmarks retain floating-point in at least one stage, typically the weight update or optimizer state. We present a training algorithm that scales to modern deep neural networks, in which every tensor (weights, gradients, and activations) is stored as INT8, and every operation (forward pass, backward pass, weight update, optimizer update) is performed in integer arithmetic. Specifically, tensors are stored as signed integer mantissas paired with explicit power-of-two exponents, on which integer-only operations are performed using five custom composable building blocks (products, sums, divide-by-constant, nonlinear functions, and weight updates). This allows complete implementation of trainable integer layers, including convolutions, batch and layer normalization, softmax, and GELU. A per-group velocity mechanism with independent exponent tracking preserves gradient precision across neurons and channels with diverse magnitudes. As every tensor fits in INT8, training memory is roughly 4x smaller than the FP32 baseline. We validate our training algorithm on CNNs (ResNet-18/34/50/101, MobileNetV2) and Vision Transformers (ViT-Ti / B), demonstrating for the first time that training neural networks with INT8 tensors and integer arithmetic can achieve competitive accuracy on CIFAR-10, CIFAR-100 and ImageNet at a reduced memory budget.

Figures & tables

Category	Method	Fwd/Bwd int	Wt update int	Wt storage int	Momentum + LR	CNN @ INet	ViT
Low-precision training	Mixed-Prec. Micikevicius et al. (2018)	✗	✗	✗	✓	✓	✓
	FP8 Micikevicius et al. (2022)	✗	✗	✗	✓	✓	✓
	F8Net Jin et al. (2022)	✗	✗	✗	✓	✓	✗
	DQT Shalby et al. (2026)	✗	✗	✗	✓	✓	✗
Integer fwd/bwd passes	Banner et al. (2018)	✓	✗	✗	✓	✓	✗
	Zhu et al. (2020)	✓	✗	✗	✓	✓	✗

Table 1: Comparison of low-precision training methods. ✓ = fully integer / supported; ✗ = floating-point / not supported;

\scriptstyle\sim

= partially integer (some float remains). CNN @ INet means ResNet-18–scale or CNNs trained/evaluated on ImageNet; ViT means training or fine-tuning of a Vision Transformer.

Method	Weights	Gradient	Velocity	Activations
FP32	FP32	FP32	FP32	FP32
FP16 Micikevicius et al. (2018)	FP32	FP32	FP32	FP16
FP8 NVIDIA Micikevicius et al. (2022)	FP32	FP32	FP32	FP16
FP8 IBM / HFP8 Sun et al. (2019)	FP16	FP16	FP16	FP8
INT8 fwd/bwd Jin et al. (2022)	FP32	FP32	FP32	FP32
Ghaffari et al. (2022)	FP32	FP32	INT16	FP32

Table 2: Per-tensor storage format and arithmetic precision.

Panel of Figure 1 — Figure 1: Method overview. All training tensors are integer $(C,e)$ pairs (see Eq. ( 1 )) with no floating-point storage or arithmetic at any stage. (a) Forward and backward of every layer. (b) Integer-only SGD with momentum and per-group velocity.

Figure 2: Accuracy vs training memory on CIFAR-10, CIFAR-100 and ImageNet. Each point is one (method, model) entry from Table 3 .

Dataset	Model	Method	Acc. (%)	Training	Precision
Dataset	Model	Method	Acc. (%)	mem. (GB)	Precision
CIFAR-10	ResNet-18	FP32 Ghaffari et al. (2022)	95.3	1.36	FP32
		Ghaffari et al. (2022)	94.8	1.33	INT8/16 + FP32
		Ours	94.7	$0.34$	INT8
	ResNet-34	FP32 Mert Bülent Sarıyıldız (2018)	95.5	2.16	FP32
	ResNet-34	Ours	95.1	$0.55$	INT8

Table 3: Test accuracy (%) and training memory (GB) on CIFAR-10, CIFAR-100 and ImageNet. Training details and hyperparameters are reported in Appendix D , while the memory computation is detailed in Appendix C .

Method	Metric	INT8	INT7	INT6	INT5	INT4	Precision ( $\text{B}\in\{4,\ldots,8\}$ )
	Acc (%)	94.8	94.7	94.47	88.5	Diverge
Ghaffari et al. (2022)	Mem (GB)	1.33	1.33	1.33	1.33	1.33	intB + FP32, velocity INT16
	Acc (%)	94.7	94.1	91.8	83.3	70.5
Ours	Mem (GB)	0.34	0.30	0.26	0.22	0.17	intB, velocity INT8

Table 4: Accuracy (%) and training memory (GB) on ResNet-18 / CIFAR-10 (batch 256) across bit-widths

\text{B}\in\{8,7,6,5,4\}

Dataset	Model	FP32	w/o per-group velocity		w/ per-group velocity
Dataset	Model	(Reference)	Acc.	Exp. mem.	Acc.	Exp. mem.
CIFAR-10	ResNet-18	95.3	93.8 ( $-1.5$ )	$0.02$ KB	94.7 ( $-0.6$ )	$4.7$ KB
	MobileNetV2	94.7	89.3 ( $-5.4$ )	$0.05$ KB	91.5 ( $-3.2$ )	$16.7$ KB
	ViT-Ti	90.9	85.3 ( $-5.6$ )	$0.05$ KB	88.0 ( $-2.9$ )	$20.5$ KB

Table 5: Effect of per-group velocity on test accuracy (%) and exponent memory.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Layer / operator	Primitives used	Ref.
Convolution (incl. 1 $\times$ 1, depthwise)	Product	Equations 5 and 3
Linear / fully-connected	Product	Equations 5 and 3
Patch embedding (strided conv)	Product	Equations 5 and 3
Residual add, gradient merge	Sum	Equation 6
Average pool ( $K{\times}K$ )	Divide-by-constant	Equations 7 and 8
ReLU / ReLU6	Clip	Section B.1

Appendix

Table 6: Layer and operator instantiations used by our framework.

Activation	Forward (on $C$ )	Backward (grad)	LUT
ReLU	$\max(C,0)$	$dY\cdot\mathbf{1}[C>0]$	—
ReLU6	$\textsc{clip}(C,0,c),\;c=\min(6\gg e,q_{\max})$	$dY\cdot\mathbf{1}[0<C<c]$	—
GELU	$\textsc{LUT}_{\mathrm{fwd}}[e,C]$ ( Equation 19 )	$(dY\cdot\textsc{LUT}_{\mathrm{bwd}}[e,C])\gg B_{G}$	INT8 $[30,256]$

Appendix

Table 7: Pointwise activations used in our models. All operations are integer on the int

B

mantissa

C

; the exponent

e

is carried through unchanged.

dY

denotes the upstream gradient.

LUT	Entry	Domain	Scale	Shape	Size
$\textsc{LUT}_{\exp}$	$\textsc{round}\!\bigl(\exp(k\,2^{e_{x}})\cdot 2^{L}\bigr)$	$k\in[-255,0]$ , ${\sim}30$ exponents	$L{=}7$	$[30,256]$	${\sim}8$ KB
$\textsc{LUT}_{\log}$	$\textsc{round}\!\bigl(-\log(p/2^{B})\cdot 2^{L_{\ell}}\bigr)$	$p\in[1,2^{B}{-}2]$	$L_{\ell}{=}5$	$[256]$	${\sim}256$ B

Appendix

Table 8: Precomputed INT8 tables for the output head. Built once at initialization and shared across every output instance in the model.

Configuration	$P$ (M)	$A$ (M/sample)	$N$
ResNet-18 / CIFAR-10 (32 2 )	11.2	1.29	256
MobileNetV2 / CIFAR-10 (32 2 )	2.24	1.88	256
ViT-Ti / CIFAR-10 (32 2 )	5.36	7.91	256
ViT-B / CIFAR-10 fine-tune (224 2 )	86.0	35.1	64
ResNet-18 / CIFAR-100 (32 2 )	11.2	1.29	256
ResNet-34 / CIFAR-100 (32 2 )	21.3	2.01	256

Appendix

Table 9: Per-configuration inputs to Equation 28 : parameters

P

, saved-for-backward activations

A

per sample, batch size

N

	RN-18 C10/C100	MNv2 C10	ViT-Ti C10	ViT-B C10	RN-18 INet
Batch size	256	256	256	64	256
Epochs	200	200	400	50	90
Base LR	0.1	0.05	0.05	0.001	0.1
Warmup epochs	5	5	5	0	5
Momentum $\beta$	0.9	0.9	0.9	0.9	0.9
Weight decay	$10^{-4}$	$10^{-4}$	$10^{-4}$	$10^{-4}$	$10^{-4}$

Appendix

Table 10: Per-model hyperparameters used for Table 3 . All runs: SGD with momentum, cosine LR with linear warmup.

Figure 3: ResNet-18 / CIFAR-10: integer-only (INT8) vs. FP32 baseline, 200 epochs, step LR at 60/120/180.

Softmax	GELU	LayerNorm	Linear	Acc. (%)	$\Delta$ vs. float
float	float	float	float	99.1	(float baseline)
float	float	float	INT8	95.9	$-3.2$
float	float	INT8	INT8	95.8	$-3.3$
float	INT8	INT8	INT8	95.9	$-3.2$
INT8	INT8	INT8	INT8	96.1	$-3.0$
Ghaffari et al. [2022] (float softmax, FP32 weights)				98.8	$-0.3$

Appendix

Table 11: Operator layer ablation on ViT-B fine-tuning (CIFAR-10). All rows use INT8 weights and activations.

Low-Precision Training

See all 24 papers

True 4-Bit Quantized Convolutional Neural Network Training on CPU: Achieving Full-Precision Parity
Mar 14, 2026Shivnath TatheLow-Precision Training 4-Bit Quantization
Independent Researcher, Pune, India
Enabling Fully Integer-Only Inference for Lightweight Detection Transformers
Jul 27, 2026Thanh Cong Le, Michal Szczepanski, Martyna PorebaEfficient Transformer Inference Efficient Inference
Université Paris-Saclay, CEA, List, Palaiseau, F-91120, France
FlashOptim: Optimizers for Memory-Efficient Training
Feb 26, 2026Jose Javier Gonzalez Ortiz, Abhay Gupta, Christopher Rinard +1Memory-Efficient Training Mixed-Precision Quantization
Databricks AI Research

Mar 14, 2026J/K move · Enter open · S save

True 4-Bit Quantized Convolutional Neural Network Training on CPU: Achieving Full-Precision Parity

Shivnath Tathe

Independent Researcher, Pune, India

Low-Precision Training 4-Bit Quantization Neural Network Quantization Quantization-Aware Training Convolutional Neural Networks

Open paper AskPDF arXiv

Low-precision neural network training has emerged as a promising direction for reducing computational costs and democratizing access to deep learning research. However, existing 4-bit quantization methods either rely on expensive GPU infrastructure or suffer from significant accuracy degradation. In this work, we present a practical method for training convolutional neural networks at true 4-bit precision using standard PyTorch operations on commodity CPUs. We introduce a novel tanh-based soft weight clipping technique that, combined with symmetric quantization, dynamic per-layer scaling, and straight-through estimators, achieves stable convergence and competitive accuracy. Training a VGG-style architecture with 3.25 million parameters from scratch on CIFAR-10, our method achieves 92.34% test accuracy on Google Colab's free CPU tier -- matching full-precision baseline performance (92.5%) with only a 0.16% gap. We further validate on CIFAR-100, achieving 70.94% test accuracy across 100 classes with the same architecture and training procedure, demonstrating that 4-bit training from scratch generalizes to harder classification tasks. Both experiments achieve 8x memory compression over FP32 while maintaining exactly 15 unique weight values per layer throughout training. We additionally validate hardware independence by demonstrating rapid convergence on a consumer mobile device (OnePlus 9R), achieving 83.16% accuracy in only 6 epochs. To the best of our knowledge, no prior work has demonstrated 4-bit quantization-aware training achieving full-precision parity on standard CPU hardware without specialized kernels or post-training quantization.

Abstract

Low-Precision Training

Explore similar work

True 4-Bit Quantized Convolutional Neural Network Training on CPU: Achieving Full-Precision Parity

Enabling Fully Integer-Only Inference for Lightweight Detection Transformers

FlashOptim: Optimizers for Memory-Efficient Training