Language models are being explored as the interface through which space robots interpret operator commands and select among verified skills. Running them onboard calls for low-bit compression, and flight use adds requirements that generic post-training quantization leaves implicit: a fixed memory budget and storage format, a static artifact under configuration control, and behavior close to the full-precision model validated on the ground. We treat compression as a ground-segment step and study QSelect, which composes one static low-bit model from compatible AWQ, GPTQ, and AutoRound checkpoints by assigning each linear module to one quantizer family and scoring every substitution at the final next-token distribution. The default fidelity mode needs no labels or tuned thresholds and accepts a substitution only when the forward KL divergence from the BF16 reference decreases; an optional target-adapted mode adds answer likelihood on a small labeled set. On DeepSeek-R1-Distill-Qwen-7B, Llama-3.1-8B, and Llama-3.2-3B at 3 and 4 bits, composed models reach lower mean held-out perplexity than every single-family model in all six settings, and the fidelity mode lowers perplexity and KL further. In a matched scaffold, final-output scoring gives lower held-out KL than representation-similarity scoring in all 12 paired runs. With labels from training or development splits of the evaluated task families, the target-adapted mode raises 3-bit pooled accuracy on the two Llama models by 4.3 and 4.5 points over the best single-family model. At 3 bits, unquantized embedding and output layers make up 43-46% of the packed artifact. All evaluations use general-domain benchmarks on a ground GPU; space-domain tasks and flight hardware are left to future work.
Figures & tables
Fig. 1: Overview of QSelect, with Llama-3.2-3B at W4 as the running example. The three W4 candidates share one packed format. Step 1 gives module ℓ of the current model F(a) to family r and scores the result by the forward KL from the cached reference distribution; the lowest-KL family is kept only if it lowers the current KL (right). Step 2 commits the choice and moves on, in one shallow-to-deep pass over all 196 modules. The static artifact is uplinked after acceptance tests; onboard use is not evaluated here. Bar heights and grid colors are schematic; counts, token budgets, and times are measured values (Sections V and VI ).
Requirement
Design choice in QSelect
Evidence in this paper
R1 Fixed memory budget and a storage format the onboard runtime supports
All candidates share bit width, group size, and packed integer format; selection never changes the bit width
Table VII , Fig. 3
R2 Static artifact under configuration control
One candidate per module, fixed after selection; no runtime selector, routing, or ensemble; the assignment vector and checkpoint hashes identify the artifact
By construction (Section IV-F )
R3 Behavior close to the ground-validated reference
Candidates are scored by the forward KL from the reference next-token distribution on calibration text; acceptance can check the same quantity on held-out text
Tables II – III
R4 Few or no mission labels
Fidelity mode needs only unlabeled text; the target-adapted mode uses a small labeled set
Tables II and III
R5 Bounded, one-time ground compute
Decision granularity and pass count trade ground time for fidelity
Fig. 2 , Table VI
TABLE I: Deployment requirements for an onboard language model and how they are addressed
Setting
Configuration
Δ PPL
Δ KL
Δ Micro
Δ Macro
DS W3
QSelect-KL
-4.8%
-7.5%
-0.15
+0.75
Local-MSE
+8.0%
+1.7%
-2.16
-1.20
DS W4
QSelect-KL
-1.7%
-13.8%
+0.02
-0.45
Local-MSE
+2.9%
-13.5%
+0.04
+0.32
Llama-3.1 W3
QSelect-KL
-2.1%
-4.3%
-0.90
-1.20
Local-MSE
-0.7%
+4.3%
-3.60
-3.30
TABLE III: Fidelity mode and a local reconstruction criterion, both relative to the module-wise QSelect-TA mean. PPL and KL are relative changes of three-replicate means; accuracy values are percentage points. Negative PPL/KL and positive accuracy favor the listed configuration. Local-MSE was run on the 7B/8B settings only.
Setting
Δ PPL
Δ KL
Δ Micro
Δ Macro
DS W3
-1.42%
-9.96%
+0.74
+1.17
DS W4
+2.03%
-5.07%
-0.03
-0.46
Llama-3.1 W3
-6.26%
-19.14%
+5.45
+4.29
Llama-3.1 W4
-0.81%
-17.14%
+0.68
+0.88
TABLE IV: CKA versus guarded final-distribution KL (0.1% margin) inside the same sequential layer-wise scaffold. Deltas are final-KL minus CALM-reimpl.; PPL/KL are relative percentages and accuracy values are percentage points.
Setting
AWQ
GPTQ
AutoRound
DS W3
0
3
0
DS W4
0
3
0
Llama-3.1 W3
0
3
0
Llama-3.1 W4
0
3
0
Llama-3.2 W3
0
0
3
Llama-3.2 W4
0
2
1
TABLE V: Whole-model family choices made by the same KL-shortlist and answer-NLL rule. Counts are over calibration-bank replicates 17, 23, and 41.
Fig. 2: Ground selection time versus fidelity and task accuracy on DS-R1-Qwen-7B W3; labels in (c) apply to all panels. Filled circles use the final output (layer-, component-, and module-wise QSelect-TA, the guarded layer-wise KL selector, and module-wise QSelect-TA with three passes); hollow squares are proxy criteria (Local-MSE and CALM-reimpl. with CKA). All values come from Tables II , III , VI , XI , and XVI . Local-MSE and three-pass metrics are derived from reported relative changes. Times for Local-MSE and the layer, component, and module scans are the approximate values reported for the DS settings (about 0.8, 18, 35–40, and 210 minutes); the bar on the component point spans 35–40 minutes. The triangle is module-wise QSelect-KL, derived from Table III ; its time is not reported, so it is drawn at the module-wise QSelect-TA time, which bounds it from above.
Artifact reduction
Peak
Drift
Setting
Measured
Est. †
BF16 share †
CUDA
Δ PPL
Δ KL
DS W3
68.74%
68.79%
45.9%
26.3%
-0.029%
-0.089%
DS W4
63.37%
63.40%
39.1%
24.2%
-0.028%
+0.024%
Llama-3.1 W3
69.70%
69.78%
43.3%
27.9%
+0.033%
-1.314%
Llama-3.1 W4
64.26%
64.30%
36.7%
25.9%
-0.277%
-0.232%
TABLE VII: Packed export of the calibration-replicate-17 assignments (QSelect-TA). Artifact and peak-CUDA entries are reductions relative to the BF16 reference; Δ PPL and Δ KL are packed-versus-fake-quantized drift. † From the analytic storage model (Section VII -A), not measured.
Fig. 3: Analytic composition of the serialized artifact. Green segments are the BF16 embedding, output head, and normalization layers; the percentages give their share of the quantized artifact. Each label compares the estimated reduction with the measured one in Table VII .