When Calibration Data Matter in Sub-4-Bit Language Model Quantization
Organizations: Massachusetts Institute of Technology · Intel Corporation
Abstract
Low-bit post-training quantization reduces language-model memory but can sharply degrade downstream task accuracy. Calibration usually relies on generic text, but when task-informed data recover accuracy and which quantizers benefit remain poorly understood. We introduce FragCal-DC, which supplements generic text with training documents from task families with low retention of full-precision accuracy in a two-bit scan. We use a two- to four-bit sweep to locate functional regimes and four-seed calibration comparisons to evaluate six data-dependent methods across six dense models (1.7B-8B) and ten tasks. At their lowest tested functional bit widths, iterative SignRound V2 and OmniQuant improve at two bits, whereas single-pass GPTQ and AWQ show small mean responses at three bits. At two bits, SignRound V2 gains 1.83 percentage points in mean ten-task accuracy over generic-text calibration across six models. At 70 billion parameters, GPTQ and SignRound V2 both improve at two bits, with functional generic-calibration baselines. For two-bit SignRound V2, task-level controls show that recovery concentrates on families included in calibration, while incorrect answer designations retain about 90% of the mean accuracy gain. Changes in reconstruction error do not consistently track accuracy gains. These findings support task-informed calibration as a design principle for preserving downstream accuracy in low-bit quantization.
Figures & tables
| Method | Class | Evaluated implementation |
| RTN | Data-free reference | Round-to-nearest |
| ❍ RTN-opt | Single-pass | Importance-weighted scale search, zero iterations |
| ❍ AWQ | Single-pass | Activation-aware scale search |
| ❍ GPTQ | Single-pass | Hessian-based rounding corrections |
| ⚫ OmniQuant | Iterative | Learnable weight clipping (LWC), 20 epochs |
| ⚫ SignRound V2 | Iterative | Rounding and scale optimization, 200 iterations |
| Width | FP Ref | RTN | ❍ RTN-opt | ❍ AWQ | ❍ GPTQ | ⚫ OmniQuant | ⚫ SR-V2 | ⚫ AQLM |
|---|---|---|---|---|---|---|---|---|
| W2G128 | 0.643 | 0.314 ✗ | 0.386 ✓ | 0.308 ✗ | 0.306 ✗ | 0.388 ✓ | 0.536 ✓ | 0.596 ✓ |
| W3G128 | 0.643 | 0.550 ✓ | 0.594 ✓ | 0.603 ✓ | 0.576 ✓ | 0.604 ✓ | 0.623 | — |
| W4G128 | 0.643 | 0.627 | 0.631 | 0.635 | 0.638 | 0.634 | 0.633 | — |
| Method | Format | Pile-200 AVG | FragCal-DC AVG | (pp) | 95% CI | GC (%) | |
|---|---|---|---|---|---|---|---|
| ❍ RTN-opt | W2G128 | 6 | 0.3853 | 0.3847 | |||
| ❍ AWQ | W3G128 | 6 | 0.6053 | 0.6049 | |||
| ❍ GPTQ | W3G128 | 6 | 0.5747 | 0.5770 | |||
| ⚫ OmniQuant | W2G128 | 5 | 0.4225 | 0.4403 | |||
| ⚫ SR-V2 | W2G128 | 6 | 0.5367 | 0.5550 | |||
| ⚫ AQLM | W2-equiv. | 6 | 0.5906 | 0.5980 |
| Method | Type | W2G128 | W3G128 | W4G128 | |||
|---|---|---|---|---|---|---|---|
| (pp) | GC (%) | (pp) | GC (%) | (pp) | GC (%) | ||
| ❍ RTN-opt | Single-pass | ||||||
| ❍ AWQ | Single-pass | ||||||
| ❍ GPTQ | Single-pass | ||||||
| ⚫ OmniQuant | Iterative | ||||||
| ⚫ SR-V2 | Iterative | ||||||
| Method | Seeds | Pile-200 | FragCal | pp | GC (%) |
|---|---|---|---|---|---|
| ❍ RTN-opt | 4 | .3213 | .3239 | +0.26 | +0.6 |
| ❍ AWQ | 1 | .322 | .321 | ||
| ❍ GPTQ | 4 | .3665 | .3899 | +2.33 | +6.6 |
| ⚫ SR-V2 | 4 | .659 | .668 | +0.86 | +13.8 |
| Task group | pp | 95% interval | Positive models |
|---|---|---|---|
| Fragility-selected tasks | 5/5 | ||
| Strict non-targeted tasks | 4/5 | ||
| Within-model difference | 5/5 |
| Calibration | AVG | pp | 95% interval | Gain retained |
| ❍ RTN-opt FragCal | 0.3843 | — | ||
| ❍ RTN-opt Pile-200 | 0.3846 | — | — | — |
| ⚫ SR-V2 correct | 0.5551 | 100.0% | ||
| ⚫ SR-V2 wrong | 0.5535 | 90.0% | ||
| ⚫ SR-V2 no designation | 0.5478 | 54.9% | ||
| ⚫ SR-V2 Pile-200 | 0.5390 | — | — | — |
| Method | MSE (%) | accuracy (pp) | Mismatch | |
|---|---|---|---|---|
| ❍ AWQ | 6 | 4/6 | ||
| ❍ GPTQ | 6 | 2/6 | ||
| ⚫ OmniQuant | 5 | 3/5 | ||
| ⚫ SR-V2 | 6 | 2/6 | ||
| ⚫ AQLM | 6 | 5/6 |
| Model | W2G128 | W3G128 | W4G128 |
|---|---|---|---|
| Llama-2-7B | |||
| Qwen3-1.7B | |||
| Phi-3 |
Appendix figures & tables54 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | ❍ RTN-opt | ⚫ SR-V2 | ||
|---|---|---|---|---|
| AVG | AVG | Fragility-selected | Strict non-targeted | |
| Llama-2-7B | ||||
| Mistral-7B | ||||
| Q2.5-7B-Inst | ||||
| Qwen3-1.7B | ||||
| Qwen3-8B | ||||
| Method | Model | Format | Reference | Pile | FragCal | pp | GC |
|---|---|---|---|---|---|---|---|
| ❍ RTN-opt | Llama-2-7B | W2G128 | Pile-200 | 0.4130 | 0.4119 | ||
| ❍ RTN-opt | Llama-2-7B | W3G128 | Pile-200 | 0.5886 | 0.5883 | ||
| ❍ RTN-opt | Llama-2-7B | W4G128 | Pile-200 | 0.6032 | 0.6033 | ||
| ❍ RTN-opt | Mistral-7B | W2G128 | Pile-200 | 0.4423 | 0.4444 | ||
| ❍ RTN-opt | Mistral-7B | W3G128 | Pile-200 | 0.6346 | 0.6342 | ||
| ❍ RTN-opt | Mistral-7B | W4G128 | Pile-200 | 0.6584 | 0.6586 |
| Model | FP Ref | RTN | ❍ RTN-opt | ❍ AWQ | ❍ GPTQ | ⚫ OmniQuant | ⚫ SR-V2 | ⚫ AQLM |
|---|---|---|---|---|---|---|---|---|
| Qwen3-1.7B | .559 | .303 | .313 | .320 | .306 | .363 | .428 | .512 |
| Llama-2-7B | .608 | .321 | .413 | .294 | .313 | .343 | .528 | .571 |
| Mistral-7B | .664 | .352 | .447 | .298 | .306 | .301 | .564 | .620 |
| Q2.5-7B-Inst | .692 | .311 | .367 | .298 | .311 | .477 | .605 | .620 |
| Qwen3-8B | .662 | .290 | .385 | .319 | .302 | .455 | .572 | .627 |
| Phi-3 | .672 | .304 | .389 | .318 | .298 | .389 | .522 | .628 |
| Model | FP Ref | RTN | ❍ RTN-opt | ❍ AWQ | ❍ GPTQ | ⚫ OmniQuant | ⚫ SR-V2 |
|---|---|---|---|---|---|---|---|
| Qwen3-1.7B | .559 | .417 | .455 | .488 | .443 | .507 | .527 |
| Llama-2-7B | .608 | .580 | .589 | .577 | .570 | .583 | .590 |
| Mistral-7B | .664 | .623 | .634 | .634 | .620 | .625 | .645 |
| Q2.5-7B-Inst | .692 | .615 | .644 | .667 | .635 | .658 | .675 |
| Qwen3-8B | .662 | .570 | .623 | .623 | .605 | .631 | .649 |
| Phi-3 | .672 | .494 | .618 | .631 | .583 | .622 | .649 |
| Model | FP Ref | RTN | ❍ RTN-opt | ❍ AWQ | ❍ GPTQ | ⚫ OmniQuant |
|---|---|---|---|---|---|---|
| Qwen3-1.7B | .559 | .524 | .535 | .547 | .541 | .548 |
| Llama-2-7B | .608 | .603 | .603 | .606 | — † | .603 |
| Mistral-7B | .664 | .653 | .654 | .659 | .656 | .654 |
| Q2.5-7B-Inst | .692 | .678 | .676 | .682 | .680 | .682 |
| Qwen3-8B | .662 | .648 | .649 | .652 | .656 | .654 |
| Phi-3 | .672 | .653 | .668 | .664 | .655 | .664 |
| Task | Qwen3 1.7B | Llama-2 7B | Mistral 7B | Qwen2.5 7B-Instruct | Qwen3 8B | Phi-3 mini |
|---|---|---|---|---|---|---|
| ARC-C | .429 | .451 | .539 | .547 | .568 | .565 |
| ARC-E | .725 | .755 | .802 | .816 | .835 | .817 |
| BoolQ | .777 | .793 | .843 | .864 | .866 | .854 |
| HellaSwag | .604 | .762 | .812 | .805 | .750 | .775 |
| LAMBADA | .504 | .736 | .750 | .695 | .643 | .645 |
| MMLU | .554 | .417 | .600 | .717 | .730 | .699 |
| Model | Correct | No designation | Wrong | Wrong/correct |
| Llama-2-7B | 107.0% | |||
| Mistral-7B | 98.3% | |||
| Q2.5-7B-Inst | 56.2% | |||
| Qwen3-1.7B | 79.0% | |||
| Qwen3-8B | 91.2% | |||
| Phi-3-mini | +2.97 | +2.12 | +2.69 | 91% |
| Model | Iterative minus single-pass (pp) |
|---|---|
| Llama-2-7B | |
| Mistral-7B | |
| Phi-3-mini-4k-instruct | |
| Q2.5-7B-Inst | |
| Qwen3-1.7B | |
| Qwen3-8B |
| Width | Model | ❍ AWQ | ❍ GPTQ | ⚫ OmniQuant | ⚫ SR-V2 | ⚫ AQLM |
|---|---|---|---|---|---|---|
| W2G128 | Llama-2-7B | 0.295 † | 0.319 † | 0.382 | 0.522 | 0.568 |
| W2G128 | Mistral-7B | 0.299 † | 0.310 † | 0.320 † | 0.565 | 0.625 |
| W2G128 | Phi-3-mini | 0.317 † | 0.299 † | 0.390 | 0.525 | 0.624 |
| W2G128 | Q2.5-7B-Inst | 0.298 † | 0.313 † | 0.515 | 0.602 | 0.595 |
| W2G128 | Qwen3-1.7B | 0.320 † | 0.305 † | 0.366 | 0.435 | 0.503 |
| W2G128 | Qwen3-8B | 0.321 † | 0.317 † | 0.460 | 0.571 | 0.630 |
| Task | FP retention |
|---|---|
| MMLU | 78.2% |
| ARC-Challenge | 84.3% |
| HellaSwag | 86.0% |
| OpenBookQA | 89.7% |
| LAMBADA | 91.1% |
| TruthfulQA | 91.3% |
| Supplement | +16 documents | +32 documents | +64 documents | Best size |
|---|---|---|---|---|
| ARC/OBQA-train | +1.27 | +1.60 | +1.19 | 32 |
| MMLU-aux-train | +0.27 | +0.39 | +0.64 | 64 |
| HellaSwag-train | +0.77 | +0.30 | +0.49 | 16 |
| Supplement | MMLU | ARC-C | HellaSwag | Other (7) | AVG |
|---|---|---|---|---|---|
| MMLU 32 | +5.65 † | 0.61 | +0.76 | 0.27 | +0.39 |
| ARC/OBQA 32 | +6.87 | +2.81 † | +0.54 | +0.82 | +1.60 |
| HellaSwag 16 | +0.30 | +1.07 | +3.30 † | +0.43 | +0.77 |
| Component | Source | Documents | Purpose |
|---|---|---|---|
| Pile anchor | pile-10k | 128 | General coverage |
| MMLU supplement | auxiliary_train | 32 | Reasoning/knowledge coverage |
| ARC/OBQA supplement | ARC-C/E + OBQA train | 32 | Task-aligned QA coverage |
| HS supplement | HellaSwag train | 16 | Commonsense coverage |
| Total | 208 | Raw documents; effective sequence counts depend on tokenizer and loader |
| Method | Model | Seed | Pile | FragCal | pp |
|---|---|---|---|---|---|
| ❍ RTN-opt | Llama-2-7B | 42 | 0.4130 | 0.4125 | |
| ❍ RTN-opt | Llama-2-7B | 123 | 0.4150 | 0.4130 | |
| ❍ RTN-opt | Llama-2-7B | 456 | 0.4112 | 0.4101 | |
| ❍ RTN-opt | Llama-2-7B | 789 | 0.4129 | 0.4118 | |
| ❍ RTN-opt | Mistral-7B | 42 | 0.4467 | 0.4515 | |
| ❍ RTN-opt | Mistral-7B | 123 | 0.4474 | 0.4490 |
| Model | s42 | s123 | s456 | s789 | Mean | SD | CV |
|---|---|---|---|---|---|---|---|
| Qwen3-1.7B | 0.4431 | 0.4479 | 0.4547 | 0.4401 | 0.4465 | 0.0056 | 1.25% |
| Llama-2-7B | 0.5700 | 0.5710 | 0.5728 | 0.5694 | 0.5708 | 0.0013 | 0.22% |
| Mistral-7B | 0.6201 | 0.6228 | 0.6251 | 0.6189 | 0.6217 | 0.0024 | 0.38% |
| Q2.5-7B-Inst | 0.6345 | 0.6352 | 0.6378 | 0.6329 | 0.6351 | 0.0018 | 0.28% |
| Qwen3-8B | 0.6053 | 0.6021 | 0.6085 | 0.6019 | 0.6045 | 0.0028 | 0.46% |
| Phi-3 | 0.5829 | 0.5767 | 0.5799 | 0.5759 | 0.5788 | 0.0032 | 0.55% |
| Model | s42 | s123 | s456 | s789 | Mean | SD | CV |
|---|---|---|---|---|---|---|---|
| Qwen3-1.7B | 0.4552 | 0.4593 | 0.4622 | 0.4552 | 0.4580 | 0.0030 | 0.66% |
| Llama-2-7B | 0.5355 | 0.5357 | 0.5404 | 0.5310 | 0.5356 | 0.0038 | 0.71% |
| Mistral-7B | 0.5869 | 0.5880 | 0.5948 | 0.5857 | 0.5888 | 0.0041 | 0.70% |
| Q2.5-7B-Inst | 0.6075 | 0.6078 | 0.6074 | 0.6060 | 0.6072 | 0.0007 | 0.12% |
| Qwen3-8B | 0.5865 | 0.5867 | 0.5865 | 0.5834 | 0.5858 | 0.0014 | 0.24% |
| Phi-3 | 0.5564 | 0.5535 | 0.5510 | 0.5586 | 0.5549 | 0.0033 | 0.59% |
| Pile | FragCal | |||||||
|---|---|---|---|---|---|---|---|---|
| Model | Mean | SD | CV | Range | Mean | SD | CV | Range |
| Qwen3-1.7B | .3658 | .0039 | 1.07% | .0093 | .3847 | .0031 | 0.81% | .0072 |
| Llama-2-7B | .3820 | .0061 | 1.60% | .0141 | .3947 | .0049 | 1.24% | .0117 |
| Mistral-7B † | .3067 | .0005 | 0.17% | .0010 | .3031 | .0042 | 1.39% | .0084 |
| Q2.5-7B-Inst | .5147 | .0105 | 2.04% | .0212 | .5300 | .0051 | 0.96% | .0117 |
| Qwen3-8B | .4598 | .0022 | 0.48% | .0047 | .4830 | .0013 | 0.27% | .0032 |
| Method | W2G128 | W3G128 | W4G128 |
|---|---|---|---|
| ❍ GPTQ | — | 0.0024 | — |
| ⚫ OmniQuant | 0.0031 | 0.0023 | 0.0011 |
| ⚫ SR-V2 | 0.0030 | — | — |
| Method | Width | Recorded objective change (%) |
|---|---|---|
| ❍ RTN-opt | W2G128 | |
| ❍ AWQ | W3G128 | |
| ❍ GPTQ | W3G128 | |
| ⚫ OmniQuant | W2G128 | |
| ⚫ SR-V2 | W2G128 | |
| ⚫ AQLM | W2-equivalent |
| Model | Bit | Pile-200 Loss | FragCal Loss | (%) |
|---|---|---|---|---|
| Qwen3-1.7B | W2G128 | 8.892 | 2.639 | 70.3% |
| Qwen3-1.7B | W3G128 | 2.007 | 0.528 | 73.7% |
| Qwen3-1.7B | W4G128 | 0.463 | 0.122 | 73.7% |
| Llama-2-7B | W2G128 | 0.249 | 0.064 | 74.2% |
| Mistral-7B | W2G128 | 0.074 | 0.000 | 99.4% |
| Q2.5-7B-Inst | W2G128 | 0.640 | 0.173 | 73.0% |
| Model | Bit | Pile-200 Loss | FragCal Loss | (%) |
|---|---|---|---|---|
| Qwen3-1.7B | W3G128 | 2.007 | 0.528 | 73.7% |
| Llama-2-7B | W3G128 | 0.043 | 0.012 | 72.5% |
| Mistral-7B | W3G128 | 0.008 | 0.000 | 96.9% † |
| Q2.5-7B-Inst | W3G128 | 0.114 | 0.031 | 72.7% |
| Qwen3-8B | W3G128 | 0.162 | 0.058 | 64.4% |
| Phi-3 | W3G128 | 0.103 | 0.030 | 70.5% |
| Method | Model | MSE (%) | accuracy (pp) |
|---|---|---|---|
| ❍ AWQ | Qwen3-1.7B | ||
| Llama-2-7B | |||
| Mistral-7B | |||
| Qwen2.5-7B | |||
| Qwen3-8B | |||
| Phi-3 |
| Model | Bit | Pile-200 Final | FragCal Final | (%) |
|---|---|---|---|---|
| Qwen3-1.7B | W2G128 | 939.08 | 872.90 | 7.0% |
| Llama-2-7B | W2G128 | 2.16 | 2.52 | +16.7% |
| Mistral-7B | W2G128 | 0.11 | 0.11 | 0.0% |
| Q2.5-7B-Inst | W2G128 | 17.92 | 17.39 | 3.0% |
| Qwen3-8B | W2G128 | 76.80 | 73.07 | 4.9% |
| Model | Bit | Pile-200 Loss | FragCal Loss | (%) |
|---|---|---|---|---|
| Qwen3-1.7B | W2G128 | 741.5 | 629.6 | 15.1% |
| Llama-2-7B | W2G128 | 2.54 | 2.34 | 7.7% |
| Mistral-7B | W2G128 | 0.187 | 0.215 | +15.0% ∗ |
| Q2.5-7B-Inst | W2G128 | 19.6 | 17.3 | 11.7% |
| Qwen3-8B | W2G128 | 68.5 | 60.6 | 11.5% |
| Config | Model | Optimizer | Loss | Loss | AVG | AVG | GC% |
|---|---|---|---|---|---|---|---|
| W2G16 | Q3-1.7B | SignSGD | 465.8 | — | .5025 | — | — |
| DM-SignSGD | 360.3 | 22.6% | .5082 | +0.57 | +5.9% | ||
| W2G16 | Q2.5-7B | SignSGD | 9.72 | — | .6920 | — | — |
| DM-SignSGD | 6.83 | 29.7% | .6901 | 0.19 | 3.8% | ||
| W2G128 | Q3-1.7B | SignSGD | 790.1 | — | .4295 | — | — |
| DM-SignSGD | 639.8 | 19.0% | .4297 | +0.02 | +0.2% |
| Quantity | Pass 1 | Pass 2 |
|---|---|---|
| Mean reconstruction loss (relative) | ||
| Downstream AVG (pp), median over 20 cells | — | |
| Cells with AVG | — | 15/20 |
| Cells with AVG pp | — | 1/20 |
| Model | |||||
|---|---|---|---|---|---|
| Q3-1.7B | .290 .002 | .408 .003 | .419 .007 | .430 .004 | +.139 |
| Q2.5-7B | .322 .033 | .568 .004 | .589 .002 | .603 .002 | +.282 |
| Ll2-7B | .374 .033 | .502 .003 | .513 .002 | .526 .002 | +.153 |
| Mi-7B | .428 .016 | .541 .007 | .553 .005 | .560 .002 | +.132 |
| Q3-8B | .304 .021 | .547 .004 | .559 .002 | .570 .002 | +.266 |
| Phi-3 | .340 .028 | .494 .005 | .519 .001 | .521 .004 | +.181 |
| Model | Trend | ||||
|---|---|---|---|---|---|
| Q3-1.7B | +0.2 | +1.8 | +2.6 | +2.9 | monotonic |
| Q2.5-7B | +0.6 | +0.0 | 0.2 | +0.4 | flat |
| Ll2-7B | +2.2 | +1.5 | +1.8 | +0.9 | shrinks |
| Mi-7B | 3.5 | +2.6 | +2.8 | +3.0 | grows |
| Q3-8B | +1.8 | +1.2 | +1.6 | +1.6 | flat |
| Phi-3 | 2.0 | +2.7 | +2.7 | +3.4 | grows |
| Model | gain | gain | Gain share at (%) |
|---|---|---|---|
| Q3-1.7B | +.117 | +.022 | 84% |
| Q2.5-7B | +.247 | +.035 | 88% |
| Ll2-7B | +.129 | +.024 | 84% |
| Mi-7B | +.113 | +.020 | 85% |
| Q3-8B | +.243 | +.023 | 91% |
| Phi-3 | +.154 | +.027 | 85% |
| Model | Type | Pile-128 † | FragCal | FP Ref | (pp) | GC |
|---|---|---|---|---|---|---|
| Mistral-7B | Base | .5603 | .5888 | .6623 | +2.85 | +27.9% |
| Mistral-7B-Instruct | Instruct | .5650 | .5817 | .6397 | +1.68 | +22.5% |
| Llama-2-7B | Base | .5263 | .5356 | .6080 | +0.93 | +11.4% |
| Llama-2-7b-chat | Instruct | .5382 | .5536 | .6073 | +1.54 | +22.3% |
| Method | Bit | Pile-128 | Pile-200 | FragCal-DC |
|---|---|---|---|---|
| ❍ RTN-opt | W2G128 | — | .0006 | .0008 |
| ❍ RTN-opt | W3G128 | — | .098 | .097 |
| ⚫ SR-V2 | W2G128 | .056 | .058 | .048 |
| ⚫ SR-V2 | W3G128 | .139 | .135 | .142 |
| Model | W2G128 | W3G128 | W4G128 |
|---|---|---|---|
| Q3-1.7B | +0.54pp | +0.32pp | 0.06pp |
| Llama-2-7B | 0.52pp | 0.43pp | +0.04pp |
| Mistral-7B | +0.17pp | +0.13pp | +0.15pp |
| Q2.5-7B-Inst | 0.33pp | 0.11pp | +0.03pp |
| Q3-8B | 0.12pp | +0.19pp | 0.04pp |
| Mean |
| W2G128 | W3G128 | |||||
|---|---|---|---|---|---|---|
| Model | FP Ref | Pile-200 | FragCal | Pile-200 | ||
| Qwen3-1.7B | 25.3 | 70.9 | 63.4 | +7.6 | 31.7 | +0.9 |
| Llama-2-7B | 7.8 | 12.4 | 12.4 | 0.0 | 8.6 | +0.1 |
| Mistral-7B | 7.6 | 13.1 | 12.5 | +0.6 | 8.3 | +0.1 |
| Qwen2.5-7B-Instruct | 11.1 | 23.1 | 23.0 | +0.1 | 12.9 | 0.1 |
| Qwen3-8B | 14.6 | 29.8 | 28.7 | +1.1 | 16.5 | 0.0 |
| Method | Bit | Model | FP Ref | Pile | PPL |
|---|---|---|---|---|---|
| ❍ AWQ | W3G128 | Qwen3-1.7B | 25.3 | 34.2 | 0.5 |
| ❍ AWQ | W3G128 | Llama-2-7B | 7.8 | 9.0 | 0.0 |
| ❍ AWQ | W3G128 | Mistral-7B | 7.6 | 8.7 | 0.0 |
| ❍ AWQ | W3G128 | Qwen2.5-7B-Instruct | 11.1 | 13.4 | 0.1 |
| ❍ AWQ | W3G128 | Qwen3-8B | 14.6 | 16.9 | +0.2 |
| ❍ GPTQ | W3G128 | Qwen3-1.7B | 25.3 | 53.0 | 18.5 |
| Model | Cross-entropy reduction | Accuracy change (pp) |
|---|---|---|
| Llama-2-7B | ||
| Mistral-7B | ||
| Q2.5-7B-Inst | ||
| Qwen3-1.7B | ||
| Qwen3-8B |
| Method | Mean FragCal minus Pile-200 | Paired cells |
|---|---|---|
| ❍ AWQ | 48 | |
| ❍ GPTQ | 72 | |
| ⚫ OmniQuant | 62 | |
| ⚫ AQLM | 20 |
| Pile-200 | FragCal | Pile-128 | |||||||
| Task | s42 | s123 | s456 | s789 | s42 | s123 | s456 | s789 | s42 |
| ARC-C | .543 | .550 | .543 | .524 | .562 | .567 | .558 | .550 | .537 |
| ARC-E | .813 | .817 | .813 | .814 | .810 | .819 | .802 | .815 | .808 |
| BoolQ | .819 | .845 | .850 | .821 | .836 | .831 | .831 | .831 | .841 |
| HellaSwag | .770 | .773 | .772 | .768 | .790 | .792 | .795 | .787 | .762 |
| LAMBADA | .760 | .753 | .758 | .750 | .749 | .749 | .761 | .753 | .764 |
| Task | BF16 Ref | RTN W2G128 |
|---|---|---|
| ARC-C | .613 | .235 |
| ARC-E | .851 | .285 |
| BoolQ | .872 | .400 |
| HellaSwag | .857 | .264 |
| LAMBADA | .793 | .003 |
| MMLU | .763 | .231 |
| Pile-200 | FragCal | |||||||
| Task | s42 | s123 | s456 | s789 | s42 | s123 | s456 | s789 |
| ARC-C | .220 | .213 | .230 | .221 | .219 | .224 | .227 | .226 |
| ARC-E | .334 | .341 | .344 | .327 | .348 | .343 | .344 | .342 |
| BoolQ | .502 | .498 | .500 | .494 | .498 | .515 | .503 | .513 |
| HellaSwag | .283 | .276 | .281 | .280 | .282 | .281 | .282 | .279 |
| LAMBADA | .054 | .057 | .060 | .054 | .058 | .056 | .057 | .057 |
| Pile-200 | FragCal | |||||||
| Task | s42 | s123 | s456 | s789 | s42 | s123 | s456 | s789 |
| ARC-C | .271 | .267 | .273 | .257 | .311 | .276 | .285 | .271 |
| ARC-E | .392 | .399 | .435 | .412 | .460 | .476 | .509 | .444 |
| BoolQ | .534 | .562 | .536 | .520 | .502 | .497 | .557 | .562 |
| HellaSwag | .405 | .428 | .427 | .390 | .481 | .466 | .481 | .446 |
| LAMBADA | .147 | .166 | .141 | .137 | .201 | .205 | .196 | .146 |
| ❍ AWQ | ❍ GPTQ | ⚫ SR-V2 | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Task | Pile | FragCal | Pile | FragCal | Pile | FragCal | |||
| ARC-C | .584 | .597 | .573 | .585 | .603 | .624 | |||
| ARC-E | .824 | .834 | .815 | .819 | .838 | .843 | |||
| BoolQ | .850 | .847 | .823 | .815 | .864 | .852 | |||
| HellaSwag | .832 | .833 | .818 | .815 | .827 | .837 | |||
| LAMBADA | .768 | .763 | .755 | .748 | .790 | .787 | |||
| Five tasks | Four tasks, excluding HS | |||||
|---|---|---|---|---|---|---|
| Method | Ours | Pub. | Ours | Pub. | ||
| Unquantized | 68.59 | 64.66 | 66.76 | 66.50 | ||
| ❍ AWQ | 35.89 | 34.74 | 38.34 | 37.00 | ||
| ❍ GPTQ | 39.08 | 41.56 | 40.76 | 43.80 | ||
| ⚫ OmniQuant | 45.22 | 46.98 | 45.51 | 48.65 | ||
| ⚫ SR-V2 | 64.73 | 57.88 | 63.51 | 60.30 | ||
| Model | Layer | Module | |||
|---|---|---|---|---|---|
| Qwen3-1.7B | 0 (early) | gate_proj | 15.2% | 0.995 | 0.943 |
| Qwen3-1.7B | 0 (early) | q_proj | 14.8% | 0.988 | 0.983 |
| Qwen3-1.7B | 13 (mid) | gate_proj | 28.2% | 0.980 | 0.249 |
| Qwen3-1.7B | 13 (mid) | q_proj | 16.6% | 0.994 | 0.489 |
| Qwen3-1.7B | 27 (final) | gate_proj | 23.0% | 0.987 | 0.896 |
| Qwen3-1.7B | 27 (final) | q_proj | 30.4% | 0.977 | 0.112 |
| Width | Runs | Final CV | CV | Top-5 overlap |
|---|---|---|---|---|
| W2G128 | 4 | 0.0281 0.0015 | 0.0246 0.0015 | 0.75 |
| W3G128 | 4 | 0.0261 0.0018 | 0.0226 0.0018 | 0.85 |
| W4G128 | 4 | 0.0234 0.0017 | 0.0200 0.0017 | 0.85 |
| Condition | LAMBADA | FragCal minus Pile |
|---|---|---|
| — | ||
| , seed 42 | 0.6499 | — |
| — | ||
| Pile control | — |
| Probe | Observation | Interpretation boundary |
|---|---|---|
| Wrong designations | Most of the gain remains | Options and task exposure remain |
| RTN-opt | Small response versus SR-V2’s gain | Iteration remains coupled to other method differences |
| Same W3 | Pairwise GC favors SR-V2 over AWQ | GC uses method-specific controls. Pile-200 pp class interval spans zero |
| 70B | Both GPTQ and SR-V2 improve at W2 | Extends sensitivity. Scale and headroom remain coupled |
| Scale flattening | Fixed-code fidelity degrades | Code reoptimization remains untested |
| Hessian spot check | Calibration statistics differ | Rounding-decision changes are unmeasured |
| Alternative | pp | 95% interval | Positive models |
|---|---|---|---|
| Diversity-DC | 4/5 | ||
| LSA-DC | 4/5 | ||
| ArXiv-OOD | 5/5 |
| Model | Random QA minus Pile | FragCal minus Pile |
|---|---|---|
| Llama-2-7B | ||
| Mistral-7B | ||
| Q2.5-7B-Inst | ||
| Qwen3-1.7B | ||
| Qwen3-8B | ||
| Phi-3-mini |
| Model | Mean pp | Positive seeds |
|---|---|---|
| Llama-2-7B | 4/4 | |
| Mistral-7B | 3/4 | |
| Qwen2.5-7B-Instruct | 4/4 | |
| Qwen3-1.7B | 4/4 | |
| Qwen3-8B | 4/4 |
| Seed | MMLU | ARC-C | ARC-E | OBQA | HellaSwag |
|---|---|---|---|---|---|
| 42 | 1 | 1 | 1 | 3 | 2 |
| 123 | 0 | 4 | 1 | 7 | 0 |
| 456 | 0 | 2 | 1 | 6 | 0 |
| 789 | 3 | 3 | 2 | 6 | 0 |
| Model | Format | pp | 95% interval |
|---|---|---|---|
| Phi-3 | W2A8G128 | ||
| Mistral-7B | W2A8G128 | ||
| Phi-3 | MXINT W2A8G32 | ||
| Mistral-7B | MXINT W2A8G32 |
LLM Quantization
Large language model quantization reduces numerical precision to lower memory use and accelerate training or inference.
See all 434 papers