Quantize by Doing: Task-Guided Mixed-Precision Quantization of Vision-Language-Action Models

Jiuyi Xu; Xiao Hu; Meida Chen; Song Wang; Yang Ye; Yangming Shi

cs.ROOct 28, 2026

Quantize by Doing: Task-Guided Mixed-Precision Quantization of Vision-Language-Action Models

Authors: Jiuyi Xu, Xiao Hu, Meida Chen, Song Wang, Yang Ye, Yangming Shi

Organizations: Colorado School of Mines · Northeastern University · USC Institute for Creative Technologies · University of Central Florida

Abstract

Post-training quantization (PTQ) reduces the memory footprint of vision-language-action (VLA) models, but lowering bit widths while maintaining task performance remains challenging. Existing work has used sensitivity estimates from calibration data to assign bit widths to different model components. However, using task success directly to guide mixed-precision allocation is underexplored because closed-loop evaluations are costly and the effect of quantizing one component depends on the precision of other components. To this end, we propose QbD-VLA, a task-guided mixed-precision quantization method that uses task success from short closed-loop evaluations to assign weight and activation bit widths while accounting for interactions between model components. Experiments covering four VLA models and all four LIBERO suites show strong task performance at average precisions of approximately 4 bits for weights and fewer than 6 bits for activations. On SIMPLER, the gain over uniform quantization is small. On a Franka Research 3 robot arm, simulation-selected allocations show no detected success-rate loss on X-VLA but fail on $π_{0.5}$ . These findings support using task success to guide mixed-precision VLA quantization, while highlighting the need for real-world evaluation before deployment. Code is available at github.com/jiuyixu25/QbD-VLA.

Figures & tables

Figure 1: The overview of QbD-VLA. A VLA policy (X-VLA) with one weight / activation bit-width per group and the allocation shown has an effective W4.16 / A5.90 and is deployed unchanged on the FR3 arm. A group is lowered by one level, the policy is rolled out in simulation, and the move is kept only if its success estimate stays above the floor. The search repeats until no move passes.

Policy	Configuration	Spatial	Object	Goal	Long	Avg.
$\pi_{0.5}$	Full precision	99.0	99.0	94.0	95.0	96.8
	Uniform W4A4	0.0	0.0	0.0	0.0	0.0
	Uniform W4A6	97.5	99.5	93.5	95.5	96.5
	QbD-VLA W4A5.74	98.5	99.0	94.0	93.5	96.2
OpenVLA-OFT	Full precision	100.0	97.5	96.0	92.5	96.5
	Uniform W4A4	0.0	0.0	0.0	0.0	0.0

Table 1: Success rate (%) on the four LIBERO suites. QbD-VLA allocations are selected on LIBERO-Spatial and applied unchanged to the other suites. Their bit widths are parameter-weighted nominal values over all layer groups.

Configuration	W / A	Spatial	Object	Goal	Long	Avg.
FP	16 / 16	99.0	99.0	94.0	95.0	96.8
ActQuant	3.09 / 16	98.5	99.0	97.0	89.0	95.9
HoloQ-VLA	4 / 4	99.0	100.0	96.5	96.5	98.0
QbD-VLA	4.00 / 5.74	98.5	99.0	94.0	93.5	96.2

Table 2: VLA quantization methods on

\pi_{0.5}

under our evaluation on the four LIBERO suites, using their released code or weights. The ActQuant checkpoint averages 3.09 weight bits.

Figure 2: $\pi_{0.5}$ rollouts of the LIBERO-Goal task put the wine bottle on the rack from the same initial state. Rows, top to bottom: full precision, uniform W4A4, uniform W4A6, and the QbD-VLA allocation. Uniform W4A4 never approaches the bottle and uniform W4A6 drops it short of the rack, whereas the selected allocation completes the task.

Allocation method	$\pi_{0.5}$	OpenVLA-OFT	X-VLA	$\pi_{0}$
QbD-VLA	98.5	97.5	97.0	74.5
Action error	27.5	97.5	96.0	76.0
Activation kurtosis	0.0	0.0	0.0	51.0
Reconstruction error	52.0	0.0	91.5	60.0
UFR	99.0	99.0	87.0	68.0

Table 3: Success rate (%) on LIBERO-Spatial. The three sensitivity-based rows share a greedy allocator and the target cost of the QbD-VLA allocation of each policy. UFR reaches its own cost: W4.00 / A5.90, W8.00 / A8.00, W4.00 / A5.62, and W4.00 / A4.54.

X-VLA configuration	W / A	Coke can	Move near	Drawer	Put in drawer	Overall
FP	16 / 16	90.7	84.2	65.7	29.6	75.0
Uniform W4A8	4 / 8	94.0	83.8	64.4	28.7	75.6
Uniform W4A6	4 / 6	87.0	75.4	52.3	4.6	64.8
Uniform W4A4	4 / 4	0.0	1.7	13.9	0.0	3.9
QbD-VLA, 24 tasks, $\tau=0.90$	4.03 / 5.33	87.7	65.8	44.4	0.0	59.8
QbD-VLA, 40 tasks, $\tau=0.95$	4.00 / 5.88	89.3	84.2	43.5	12.0	66.8

Table 4: X-VLA on the SIMPLER visual-matching split.

Model	Configuration	Success	Close cmd.
X-VLA	FP	11/30	28/30
X-VLA	QbD-VLA	13/30	29/30
$\pi_{0.5}$	FP	6/6	6/6
$\pi_{0.5}$	QbD-VLA	0/6	0/6

Table 5: Transfer of frozen allocations to the FR3 arm.

Configuration	Success	Close cmd.
Uniform W4A6	0/6	0/6
Uniform W4A8	1/6	1/6
Uniform W8A6	0/6	0/6
Uniform W8A8	5/6	6/6

Table 6: Uniform-precision diagnostics for

\pi_{0.5}

on the FR3 arm.

Model	Group	Layers	Params (M)	Share (%)	QbD-VLA
$\pi_{0.5}$	Vision encoder	162	411.1	14.6	W4A6
	Language backbone	126	1981.8	70.2	W4A6
	Action head: attention projections	72	84.9	3.0	W4A4
	Action head: pre-attention modulators	18	56.6	2.0	W4A6
	Action head: post-attention and final modulators	18	56.6	2.0	W4A4
	Action head: MLP layers	54	226.5	8.0	W4A4

Table 7: Layer groups used for bit allocation, with the number of quantizable linear layers in each group, their parameter count, their share of the model’s quantizable parameters, and the level QbD-VLA selects for the group on LIBERO-Spatial. BF16 and FP32 mark groups kept at the released precision. The All groups rows give the parameter-weighted nominal bit widths used in Table 1 .

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

	$\pi_{0.5}$	OFT	X-VLA	$\pi_{0}$
Released precision	BF16	BF16	FP32	BF16
Render size	360	256	256	360
Action mode	delta	delta	absolute	delta
$H_{\mathrm{alloc}}$	120	120	300	120
$H_{\mathrm{eval}}$ , Spatial	280	220	800	280
$H_{\mathrm{eval}}$ , Object	280	280	800	280

Appendix

Table 8: Per-model settings.

H_{\mathrm{alloc}}

is the episode limit during allocation and

H_{\mathrm{eval}}

the limit of each model’s official evaluation code, used for every reported number. Allocation episodes and wall-clock time are for the primary LIBERO-Spatial runs.

Mixed-Precision Quantization

Mixed-precision quantization assigns different numeric precisions across a model to balance accuracy with memory, speed, energy, and hardware constraints.

See all 171 papers

Jun 17, 2026

Mix-QVLA: Task-Evidence-Aware Mixed-Precision Quantization of Vision-Language-Action Models

We propose Mix-QVLA, a task-evidence-aware mixed-precision PTQ framework for VLA models. Mix-QVLA anchors each quantized variant to the full-precision action-token reference decision and evaluates whether quantization preserves task-relevant evidence across key VLA functional boundaries. It computes normalized gradient-weighted task-evidence maps from boundary activations and compares full-precision and quantized maps using evidence-mass and attribution-distribution distortion, capturing changes in both the strength and allocation of decision-supporting evidence. A soft-bottleneck objective aggregates boundary-level degradation into layer-wise sensitivity scores. Mix-QVLA further models sensitivity throughout task execution, capturing phase-dependent shifts in layer importance rather than assuming a fixed sensitivity profile. The resulting evidence- and time-aware scores guide mixed-precision bit allocation under model-size and BitOps budgets. Extensive evaluations on OpenVLA-style policies show that Mix-QVLA improves the accuracy-efficiency trade-off of low-bit VLA deployment. On LIBERO, Mix-QVLA reduces OpenVLA-OFT memory from 15.4 GB to 4.1 GB, retains 96.3 average success compared with 97.1 for the BF16 model, and achieves a 1.52x inference speedup.

Navin Ranjan, Andreas Savakis

Rochester Institute of Technology Rochester, New York 14623, USA

Mixed-Precision Quantization VLM Quantization Efficient VLA Model Inference Vision-Language-Action Models Post-Training Quantization

Ask PDF

May 19, 2026

ActQuant: Sub-4-bit Action-Guided Quantization for Vision-Language-Action Models

Vision-Language-Action (VLA) models exhibit remarkable action generation for embodied intelligence, but their heavy compute make deployment on edge platforms impractical. Aggressive, sub-4-bit weight quantization is the natural solution, yet existing post-training quantization (PTQ) methods suffer severe performance degradation in this regime. To address this, we introduce ActQuant, an action-guided mixed-precision PTQ framework that operates in two stages: (1) an inter-tensor bit allocator that assigns each weight matrix a single bit-width based on how much it contributes to predicting the agent's actions; (2) an intra-tensor scale optimizer tunes per-block quantization scales using action-aware curvature, so that dynamic range is concentrated on the weights most influential for control. To deliver the on-device benefits of our aggressive quantization, we further introduce OmniModel.cpp, an agentic conversion pipeline that ports architectures into a native C/C++ runtime with efficient low-bit kernels. We evaluate ActQuant both in simulation and on a real-world 6-DoF UR3 arm, with all models deployed through OmniModel.cpp. On the LIBERO benchmark, ActQuant is the only method that operates at or below 3 bits-per-weight, retaining 95.0% on OpenVLA-OFT and 94.8% on

π_{0.5}

. Pushed further, ActQuant reaches 2.5 bpw at 90.1% on OpenVLA-OFT, compressing the backbone from 14.3 GB to 2.7 GB (5.3

\times

). On the physical UR3 arm,

π_{0.5}

quantized with ActQuant retains the baseline's success rate while reducing the memory footprint by 2.5

\times

Arash Akbari, Arman Akbari, Masih Eskandar +11

1Northeastern University · University of Georgia · 3Cisco Systems

Neural Network Quantization VLM Quantization Efficient VLA Model Inference Low-Bit Quantization Mixed-Precision Quantization Efficient VLA Models+4

Ask PDF

Sep 21, 2026

VLAQuantBench: Closed-Loop Evaluation of Post-Training Quantization for Vision-Language-Action Models

Post-training quantization reduces the memory requirements of vision-language-action (VLA) models, but precision selection must account for the interaction between layer scope, numerical format, and calibration. We introduce VLAQuantBench, a controlled evaluation with 409 runs and 94,574 simulation episodes: four models on LIBERO, with X-VLA additionally evaluated on three simulation benchmark families. Under uncalibrated W4A4 round-to-nearest quantization, expanding a

π_{0.5}

action-head subset from 126 to 167 layers raises success from 7.0% to 70.5%. Fixed-observation replay confirms a corresponding numerical recovery. Two-episode calibration removes the severe joint failures in the tested subsets, whereas the same smoothing-and-clipping recipe lowers

π_0

success and does not recover OpenVLA-OFT end-to-end. For OpenVLA-OFT, protecting one 28,672-parameter output projection instead restores near-baseline success: the remaining 441 eligible linear layers retain W3 on LIBERO-Long or eight-bit activations across all four suites. Task-clustered intervals support the large failure and recovery contrasts. These results establish recipe-dependent interactions and identify concrete precision assignments, rather than universal layer-sensitivity rules. Real-kernel and physical-robot measurements complement the accuracy analysis. Code, configurations, and episode records are publicly available at https://github.com/jiuyixu25/VLAQuantBench.

Jiuyi Xu, Qing Jin, Meida Chen +3

Colorado School of Mines · Independent Researcher · University of Central Florida +2

Mixed-Precision Quantization Closed-Loop Control of VLA Models Efficient VLA Model Inference Vision-Language-Action Models Post-Training Quantization VLA Model Post-Training

Ask PDF

Abstract

Mixed-Precision Quantization

Explore similar work

Mix-QVLA: Task-Evidence-Aware Mixed-Precision Quantization of Vision-Language-Action Models

ActQuant: Sub-4-bit Action-Guided Quantization for Vision-Language-Action Models

VLAQuantBench: Closed-Loop Evaluation of Post-Training Quantization for Vision-Language-Action Models