Small hand-tracking networks are deployed as 4- and 8-bit codes, and they should keep adapting to a user's camera after deployment, on the device and within roughly the memory of the model itself. Quantization-aware training cannot do this cheaply, because it keeps a 32-bit master copy of every weight together with its optimizer state, about 96 bits per weight. Training the codes directly with stochastic rounding needs no copy but injects rounding noise at every step, and the methods that remove this noise keep a residual buffer for every weight. We introduce sensitivity-gated accumulator caching (SGAC), which keeps residual state for only a small and changing set of weight groups. A set-associative cache of 8-bit accumulators collects the sub-step updates of the groups with the largest Fisher-weighted squared code step and writes them into the codes as whole steps; an evicted line is flushed by stochastic rounding, and a group without a line does not move until it is admitted. We prove that the write-back is unbiased and show, under a local quadratic model, that rounding a sub-step update costs loss of the same order as the update's own gain, so that rounding pays only where the expected gradient is large relative to the curvature. On a 383k-parameter hand-keypoint network, SGAC trains with 8.1 bits of state per weight. Scored on the 4/8-bit network that the device runs after adaptation, it has the highest mean accuracy of nine deployed low-bit networks on two simulated cameras. On twelve ImageNet-C corruptions it has the highest average of corrupted and clean accuracy of seven deployed methods, ahead of the PEQA-style, ECO-style and direct low-bit baselines with confidence intervals that exclude zero, and it matches an 8-bit accumulator kept for every weight with 56% of that state.
Figures & tables
Figure 1: Overview of the two-hand system (top) and of SGAC (bottom); each panel is described in the section named in its title. For each hand, PalmNet-K turns the frame It into keypoints PtR or PtL , GestureTCN turns them into gesture tokens ytR or ytL , and the intent grammar G turns tokens into actions given the scene state st . SGAC updates the deployed 4/8-bit codes in place through a small cache of accumulators.
Figure 2: The compass gradient. A CompassConv block is trained with six branches and folded into one 3×3 kernel for deployment (top). On the device, the gradient g of the folded kernel is projected onto the three masks and the two stencils of ( 3 ) and summed (bottom), which gives the update direction the six branches would have produced, without storing a branch.
Figure 3: SGAC memory layout. The low-bit codes (“ROM”) are the only copy of the convolution weights; a set-associative cache (“SRAM”) holds 8-bit accumulators for a few groups, which write whole code steps back; uncached groups wait.
Figure 4: Wait-or-write-back for one weight group. While the group has no line, its codes do not move and its gradients only update Fg (wait). Once admitted, its sub-step updates di accumulate in 8 bits and the codes move by whole steps as the accumulator crosses ±1 (commit). On eviction, the remainder (0.6 of a step in this example) is rounded stochastically into the codes (flush), so nothing is lost in expectation.
Figure 5: Qualitative results and the keypoint network. (a)–(b) keypoints of the final PalmNet-K; (c) training crops; (d)–(e) the same evaluation hands through two simulated cameras before adaptation and after SGAC (PalmNet-K 0 ), with the number of joints within [email protected] (images enhanced for display; keypoints are network outputs); (f) architecture.
Figure 6: Training state against the accuracy of the deployed network, averaged over both cameras (mean ± std over seeds). The dashed line is the frontier of the networks that end in 4/8-bit codes: the best accuracy reachable with at most that much state. Full precision and QAT both use 96 bits and are drawn slightly apart.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Detection vs. false activations as the gesture threshold varies; stars mark the FDR-calibrated thresholds.
Post-training weight-only quantization to 4 bits is widely used to reduce the memory and compute costs of large language model inference. Existing PTQ methods, such as AWQ and GPTQ, improve how weights are mapped onto a fixed 4-bit grid through scaling, clipping, or error compensation. To further improve accuracy, methods such as OmniQuant and QuIP# uses gradient-assisted algorithms at the cost of hours of quantization time. In this work, we propose AAAC (Activation-Aware Adaptive Codebooks), a lightweight method for 4-bit LLM weight quantization. AAAC replaces the fixed scalar codebook used in standard quantization with two small learned scalar codebooks (64 bytes) per layer. Each group of weights selects the codebook that minimizes activation-weighted reconstruction error, encoding the choice in the unused sign bit of the group's positive scale and adding zero storage overhead. AAAC completes in 3--30 minutes on a single GPU, and adds no memory beyond the model itself. We evaluate against AWQ, GPTQ, IF4, GPTVQ, OmniQuant, SqueezeLLM, and QuIP# across model families. AAAC outperforms baselines at orders-of-magnitude less quantization time.
Beshr IslamBouli, David Jin
Massachusetts Institute of Technology, Cambridge, MA, USA.
Binary neural networks are very attractive for constrained deployment, enabling small footprint and low-power inference. For binary activations, the dot products become sign-controlled additions or subtractions, but the number of operations is unchanged. Indeed, every neuron or output channel still accumulates all of its input, even though only the sign will be retained, which is often wasteful. As the accumulation progresses, the running partial sum frequently drifts so far from zero that its final sign becomes highly predictable long before the last term is reached; every contribution evaluated after that point changes the value of the sum but not the final output activation. This paper turns this observation into a post-training early-stopping mechanism. We characterize the behavior of the running accumulations on the training dataset and use this information to predict the final sign as soon as possible. No model parameter is retrained. We count the number of operations under an idealized ordering of weights. On VGG11 applied to the CIFAR-10 dataset, the method removes 86.6% of the accumulation terms of the deepest convolution for a 0.37-point accuracy drop, and 25% of the full-network arithmetic when used on the three deepest convolutions simultaneously, for a 1.36-point drop.
Quentin Luquet de Saint-Germain, Massil Ait Abdeslam, Jean Pierre David
Polytechnique Montréal, Department of Electrical Engineering Montréal, Canada
Linear attention and recurrent models compress past tokens into a fixed-size state that is updated at every step. However, storing these states in high precision incurs substantial memory and bandwidth costs at large batch sizes, motivating low-bit state quantization. Unlike weights and key--value caches, a recurrent state is requantized after every update, so its quantization errors accumulate across steps. Our analysis shows that these errors are highly coherent across steps, so reducing per-step errors alone is insufficient. This coherence arises mainly from small updates that are lost in rounding. To preserve these updates, we intervene both before and at rounding. Before rounding, we propose Low-Rank Bypass (LRB), which accumulates small updates in a compact low-rank buffer and periodically merges its sufficiently large elements into the state. At rounding, we observe that a state element can even move opposite to its update, and such reversals prove particularly harmful. Therefore, we propose Bounded Directional Rounding (BDR), which rounds reversed elements toward their updates without additional storage. Although BDR enlarges the rounding error, it improves the model, suggesting that the direction of an update matters more than its rounding error at a single step. Across Gated DeltaNet, GLA, Mamba2, and RetNet, our training-free method substantially outperforms INT4, while reducing state memory by up to 5.3× and speeding up decoding by up to 1.9× relative to FP32 states.