PQuantML: A Tool for End-to-End Hardware-aware Model Compression

Roope Niemi; Anastasiia Petrovych; Arghya Ranjan Das; Enrico Lupi; Chang Sun; Dimitrios Danopoulos; Marlon Joshua Helbing; Mia Liu

cs.LGMar 27, 2026

PQuantML: A Tool for End-to-End Hardware-aware Model Compression

Authors: Roope Niemi, Anastasiia Petrovych, Arghya Ranjan Das, Enrico Lupi, Chang Sun, Dimitrios Danopoulos, Marlon Joshua Helbing, Mia Liu, +4 more

Organizations: European Center for Nuclear Research (CERN), CH-1211 Geneva, Switzerland · Purdue University, West Lafayette, IN 47907, USA · California Institute of Technology, Pasadena, CA 91125, United States · University of Padova, Italy · Physikalisches Institut, Heidelberg University, Germany · SLAC National Accelerator Laboratory, Menlo Park, USA · Institute of Physics Belgrade, Serbia

Abstract

PQuantML is a new open-source, hardware-aware neural network model compression library tailored to end-to-end workflows. Motivated by the need to deploy performant models to environments with strict latency constraints, PQuantML simplifies training of compressed models by providing a unified interface to apply pruning and quantization, either jointly or individually. The library implements multiple pruning methods with different granularities, as well as fixed-point quantization with support for High-Granularity Quantization. We evaluate PQuantML on representative tasks such as the jet substructure classification, so-called jet tagging, an on-edge problem related to real-time LHC data processing. Using various pruning methods with fixed-point quantization, PQuantML achieves substantial parameter and bit-width reductions while maintaining accuracy. The resulting compression is further compared against existing tools, such as QKeras and HGQ.

Figures & tables

Figure 1: Examples of pruning strategies: original network (left), unstructured pruning with irregular weight removal (middle), and structured pruning removing entire channels or filters (right).

Figure 2: The generic training loop of PQuantML handles the different stages of various compression methods.

	AP	AutoSparse	CS	DST	MDMM	PDP	Wanda
Training stages
Pre-training						✓	✓
Training	✓	✓	✓	✓	✓	✓	✓
Fine-tuning			✓		✓	✓
Granularity
Unstructured		✓	✓	✓	✓	✓	✓

Table 1: Training stages and pruning granularities for supported pruning methods.

Model	Acc.	DSP	LUT	FF	Latency[cycles]	F max (MHz)	II [cycles]
HLF JSC (OpenML)
PQuantML (no pruning)	76.9%	1175	60,089	21,201	11(54.6ns)	201.3	2
PQuantML (DST t )	76.3%	147	6,298	4,034	10(42.3ns)	236.3	1
PQuantML (DST w )	76.3%	154	3,895	2,899	10(47.2ns)	211.8	1
PQuantML (PDP t )	75.89%	90	5,540	4,464	12(47.1ns)	254.8	1
PQuantML (PDP c )	76.2%	205	4,605	3,991	11(46.5ns)	236.8	1

Table 2: Resource usage, latency, and model accuracy across PQuantML and existing hardware-aware compression frameworks. In all PQuantML results, the bit-widths of weights and activations stay the same during training, but integer bits and fractional bits of weights are shared per-tensor ( t ), per-channel ( c ) or per-weight ( w ). The F max of the model marked with * is cited from [ 12 ] . For the model marked with ** the F max is based on the HLS target clock period. hls4ml is used in all the results except for those marked with ***, where da4ml is used.

Model	$\beta$	Acc.	DSP	LUT	FF	Latency[cycles]	F max	II [cycles]
PQuantML + HGQ	$1\cdot 10^{-6}$	76.5%	33	10,966	4,016	9(40.7ns)	220	1
HGQ	$1\cdot 10^{-6}$	76.5%	28	11,372	4,906	9(39.5ns)	228	1
PQuantML + HGQ	$3\cdot 10^{-6}$	76.0%	15	5,339	2,244	9(36.9ns)	244	1
HGQ	$3\cdot 10^{-6}$	75.9%	23	5,435	2,444	9(38.2ns)	235	1
PQuantML + HGQ	$5\cdot 10^{-6}$	75.6%	16	4,169	1,738	8(28.3ns)	283	1
HGQ	$5\cdot 10^{-6}$	75.7%	10	3,975	1,603	8(29.6ns)	270	1

Table 3: Comparison of HGQ-based models trained with PQuantML and native HGQ. The models were trained using the same hyperparameter settings with varying

\beta

values. The resource usage is reported after place-and-route.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Explanation
pretraining_epochs	Number of epochs during the pretraining stage
epochs	Number of epochs during the main training stage
fine_tuning_epochs	Number of epochs during the fine-tuning stage
rounds	Number of pruning/quantization rounds during training
rewind	When to rewind the weights: never , every-round , or post-training-stage
save_weights_epoch	Epoch at which weights are saved for rewinding during the first round

Appendix

Table 4: Training parameters defining the generic PQuantML training function.

Parameter	Explanation
default_data_keep_negatives	Default $k$ value for data quantization (0 or 1)
default_data_integer_bits	Default integer bit-width $i$ for data quantization
default_data_fractional_bits	Default fractional bit-width $f$ for data quantization
default_weight_keep_negatives	Default $k$ value for weight quantization (0 or 1)
default_weight_integer_bits	Default integer bit-width $i$ for weight quantization
default_weight_fractional_bits	Default fractional bitw-idth $f$ for weight quantization

Appendix

Table 5: Quantization parameters of the configuration object.

Model	Dataset	Pruning parameters	Epochs
DST t	OpenML	$\alpha:1\cdot 10^{-4}$	0/1500/500
DST w	OpenML	$\alpha:1.5\cdot 10^{-4}$	0/200/100
PDP t	OpenML	sparsity: 0.94	100/2000/200
PDP c	OpenML	sparsity: 0.94	100/500/100
FITCompress c	OpenML	compression goal: 0.0075	50/300/0
PDP t	CERN Box	sparsity: 0.93	100/500/500

Appendix

Table 6: Hyperparameters used in the results shown in Table 2 . The learning rate is fixed to

1\cdot 10^{-3}

for all experiments.

Explore similar work

APQF: Agentic Profiling-Guided Structured Pruning and Mixed-Precision Quantization with Adaptive Fine-Tuning
Aug 6, 2026Sadegh Jafari, Mohiuddin Bilwal, Fan Zhou +2Model Compression Mixed-Precision Quantization
Department of Computer Science Iowa State University Ames, IA, USA · Department of Agricultural and Biosystems Engineering Iowa State University Ames, IA, USA
OneComp: One-Line Revolution for Generative AI Model Compression
Mar 30, 2026Yuma Ichikawa, Keiji Kimura, Akihiro Yoshida +11Model Compression Neural Network Compression
Fujitsu Limited · RIKEN Center for AIP · Institute of Science Tokyo +2
Prune-Quantize-Distill: An Ordered Pipeline for Efficient Neural Network Compression
Apr 5, 2026Longsheng Zhou, Yu ShenNeural Network Compression Efficient Neural Network Inference
Institute of Advanced Technology University of Science and Technology of China Hefei, China · Supercomputing Center University of Science and Technology of China Hefei, China

Aug 6, 2026J/K move · Enter open · S save

APQF: Agentic Profiling-Guided Structured Pruning and Mixed-Precision Quantization with Adaptive Fine-Tuning

Sadegh Jafari, Mohiuddin Bilwal, Fan Zhou, Brian Gelder+1

Department of Computer Science Iowa State University Ames, IA, USA · Department of Agricultural and Biosystems Engineering Iowa State University Ames, IA, USA

Model Compression Mixed-Precision Quantization Quantization-Aware Training Neural Network Pruning

Open paper AskPDF arXiv

Modern deep neural networks achieve strong performance, but their scale makes them costly and slow, especially on resource-constrained edge devices. Pruning and quantization address this, but rely on manual, expert choices and on algorithms that are hard to apply across architectures. Uniform settings also ignore how differently individual layers respond to compression, which costs accuracy. We introduce APQF, an agentic profiling-guided framework that combines structured pruning, mixed-precision quantization-aware training, and accuracy recovery in one automated pipeline. A profiling agent measures how cost is distributed across the model and how sensitive each part is to pruning, and this evidence drives per-layer pruning ratios, per-layer bit-widths, and the recovery strategy, all proposed by LLM planners and validated before execution. To our knowledge, APQF is the first framework to combine LLM-guided, profiling-grounded decisions with a fully training-aware pruning and quantization pipeline for both CNNs and vision transformers. We evaluate APQF on ResNet, VGG7, ViT, DeiT, and Swin using ImageNet-1k and CIFAR-10. On ImageNet it cuts compute to 5.6-7.7 percent of the original bit-operations, a 13-18x reduction, while keeping accuracy close to the baseline, and under a 200K-image budget it stays roughly 17 points higher in Top-1 than existing joint pruning and quantization methods. On CIFAR-10 it compresses further than that method on four of five architectures. On VGG7 it reaches 93.15 percent using only 0.41 percent of baseline bit-operations, the only method at that compression level to improve on its full-precision baseline. Ablations show that uniform compression loses the most accuracy at matched compute, and that withholding profiling data from the planner hurts every model. Six LLM planners, including free open-weight ones, all reach 97.4-97.9 percent on Swin-Tiny.

Abstract

Explore similar work

APQF: Agentic Profiling-Guided Structured Pruning and Mixed-Precision Quantization with Adaptive Fine-Tuning

OneComp: One-Line Revolution for Generative AI Model Compression

Prune-Quantize-Distill: An Ordered Pipeline for Efficient Neural Network Compression