cs.AINov 17, 2026

Why Sub-4-Bit Vision CNNs Collapse and Where Mixed-Precision Recovery Stops

Authors: Michael Schroeder

Organizations: GE Appliances, a Haier company

Abstract

Deep vision models operating in resource-constrained environments depend heavily on low-bit quantization, yet compressing weights below 4 bits frequently induces catastrophic accuracy collapse. While mixed-precision recovery strategies aim to rescue accuracy by selectively retaining high-precision parameters, the fundamental mechanics of where and why these models collapse remain poorly understood. In this study, we systematically analyze the anatomy of quantization collapse across standard Vision CNNs (including ResNet and EfficientNet backbones) evaluated on the full 50,000-image ImageNet-1k validation set. First, we map the exact progression of degradation, demonstrating that collapse is highly non-linear and concentrated in late-stage pointwise projections and depthwise layers. Second, we introduce a structured, multi-step recovery waterfall that evaluates selective integer precision (INT4/INT8) and selective floating-point scales. We show that keeping less than 5% of critical layer activations in 8-bit precision recovers up to 92% of the accuracy lost during aggressive sub-4-bit compression. Finally, we bridge the hardware reality gap by evaluating deployment performance on standard ARM and x86 CPU architectures. We show that while low-bit quantization reduces memory storage requirements, standard runtime software bit-unpacking introduces a 1.48x to 21.5x execution slowdown depending on batch size, highlighting a critical trade-off between model footprint and actual edge throughput. Our findings provide a practical blueprint for navigating the limits of sub-4-bit deployment without sacrificing hardware performance.

Figures & tables

Explore similar work

CardsList
  1. RAMP: Robust Adaptive Mixed-Precision Quantization for Edge CPU Vision Models

    Sep 23, 2026David Población-Criado, Dario Garcia-Gasulla, Eduardo QuinonesMixed-Precision QuantizationEdge Inference

  2. Audit at the Precision You Deploy: Quantization Reshapes a Subliminally Learned Preference

    Oct 31, 2026Md Rayhanul Masud, Mst Suraiya Akter ShefaQuantization RobustnessLLM Quantization