cs.CVOct 15, 2026
SaveMJev-Spatial: System One Spatial Intelligence via Native Attention Matching
Abstract
Spatial intelligence lets an agent perceive and act in the physical world, and many of its judgments must be made at interaction rates: which spatial relation holds, how far an object is, or where to place it. Vision-language models (VLMs) produce these answers by free-form text generation, paying for autoregressive decoding and answer parsing even though the output space is compact: one of a few options, a scalar, or a 2D point. In light of this, we introduce MJev-Spatial, a System 1 multimodal decision model that tackles all three types of spatial understanding tasks within a unified classification framework. Specifically, MJev-Spatial treats each candidate as a class and scores it using the backbone's pretrained query and key projections, without adding parameters. The candidates are the given options for choice tasks, or value bins and image regions refined through coarse-to-fine classification for numerical and point prediction, respectively. At inference, each classification decision takes a single forward pass and generates no answer tokens. To train on heterogeneous spatial data efficiently, we supervise all coarse-to-fine stages jointly with teacher forcing in one forward pass, and combine this with length-aware global batching and cross-GPU load balancing, raising training throughput by 30.8% over the original batching strategy. Built on Qwen3.5-4B and trained on 12.3M question-answer pairs converted from existing spatial datasets, MJev-Spatial achieves the highest overall score (65.64%) among eight tested models, ranks first on seven of twelve benchmarks, and outperforms its unmodified autoregressive base model on all twelve, while running 2.5 times faster than the fastest autoregressive baseline and 5.6 times faster than the base model.
Figures & tables
Figure 2: \captionlead Task formulation and coarse-to-fine inference. Every stage uses a shared candidate scorer. Choice tasks select a supplied option. Numerical estimation selects a coarse interval and then a fine interval, returning its midpoint. Point prediction uses three region selections, re-encoding crops between stages and returning the final cell center.
| \mindlabtablehead Output | \mindlabtablehead Candidates per stage | \mindlabtablehead Stages | \mindlabtablehead Decoded answer |
|---|---|---|---|
| Choice | Supplied option contents | 1 | The selected option, mapped back through the recorded option permutation. |
| Length (m) | 32 roots; 4 or 8 fine cells | 1–2 | Zero, overflow, or fine-cell midpoint. |
| Count | 32 coarse groups; 32 integers | 2 | , covering 0–1023 exactly. |
| Angle (deg) | 14 roots; 32 fine cells | 1–2 | Zero, overflow, or degree interval midpoint. |
| Point | 9 spatial cells at each level | 3 | Center of the final cell in original image coordinates. |
Table 1: Output types and their decision structure. Numerical tasks share the scalar interface but keep separate codebooks. Ordinary pointing produces one point; the probability tree is an evaluation extension for multi-point counting.
Figure 3: \captionlead Architecture of MJev-Spatial. Each candidate is followed by the same End marker. After contextualization by the language backbone, the state at each End position is passed through the same pretrained to form a separate candidate key; the final Decide state supplies the query through . Their similarities determine the selected candidate, with no added prediction head or new parameters. Native normalization, rotary transformations, and head averaging are retained but omitted from the diagram. Probability bars are schematic.
| \mindlabtablehead Model | \mindlabtablehead SAT | \mindlabtablehead CV- Bench | \mindlabtablehead Robo- VQ | \mindlabtablehead BLINK | \mindlabtablehead MMSI | \mindlabtablehead Ref Spatial | \mindlabtablehead Where 2Place | \mindlabtablehead Robo- POI | \mindlabtablehead Point- Bench | \mindlabtablehead ERQA | \mindlabtablehead Emb Spatial | \mindlabtablehead VSI full | \mindlabtablehead Overall |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MJev-Spatial | 79.33 | 88.82 | 79.06 | 67.17 | 45.60 | 58.50 | 67.96 | 55.74 | 64.38 | 46.75 | 78.46 | 55.93 | 65.64 |
| Molmo2-ER | 80.00 | 88.31 | 62.01 | 72.17 | 42.90 | 52.00 | 46.64 | 26.23 | 53.47 | 46.50 | 77.99 | 60.55 | 59.06 |
| Qwen3.5-4B | 43.00 | 85.56 | 56.17 | 63.11 | 33.90 | 46.00 | 55.98 | 50.00 | 62.65 | 45.50 | 75.82 | 39.93 | 54.80 |
| Qwen3-VL-8B-Instruct | 83.33 | 86.37 | 74.72 | 61.76 | 31.40 | 49.00 | 67.07 | 59.02 | 61.54 | 43.25 | 78.90 | 57.07 | 62.79 |
| Spatial-SSRL-Qwen3VL-4B | 79.67 | 85.52 | 65.26 | 60.39 | 26.70 | 48.75 | 66.54 | 61.48 | 60.60 | 41.25 | 78.10 | 46.65 | 60.08 |
| SenseNova-SI Qwen3-VL-8B | 79.67 | 83.15 | 75.68 | 56.08 | 35.30 | 33.00 | 45.92 | 45.90 | 52.34 | 41.00 | 75.33 | 59.38 | 56.90 |
Table 2: Performance comparison of MJev-Spatial with seven autoregressive VLMs on 12 spatial understanding benchmarks . Best and second-best scores are bold and underlined, respectively.
| \mindlabtablehead Metric | \mindlabtablehead Linear | \mindlabtablehead KEV | \mindlabtablehead Native Q/K |
|---|---|---|---|
| SAT real | 78.33 | 76.33 | 78.00 |
| CV-Bench | 83.80 | 85.31 | 85.38 |
| RefSpatial | 49.00 | 45.50 | 51.00 |
| Where2Place | 67.99 | 61.02 | 59.98 |
| RoboSpatial pointing | 65.57 | 63.11 | 62.30 |
| RoboSpatial VQA | 60.34 | 68.54 | 68.06 |
Table 3: Comparison results on pretrained candidate scoring. “Linear” and “KEV” are abbreviation for linear classifier and KEV pointer, respectively.
Table 4: Numerical prediction ablations. (a) Fine-stage candidate count with fixed coarse intervals and argmax decoding. (b) Fine-stage decoding using the same 216-cell model, with coarse-stage argmax. Quantization error uses training length labels; prediction metrics use the VST-300 dev set. Shading marks the adopted settings.
| \mindlabtablehead Metric | \mindlabtablehead Crop | \mindlabtablehead Original | \mindlabtablehead ViT replay | \mindlabtablehead KV replay |
|---|---|---|---|---|
| SAT real | 78.00 | 74.67 | 73.33 | 78.00 |
| CV-Bench | 85.31 | 85.14 | 84.56 | 84.52 |
| RoboSpatial VQA | 68.06 | 68.75 | 65.01 | 63.38 |
| BLINK val | 63.04 | 62.56 | 61.07 | 62.74 |
| MMSI-Bench | 27.80 | 25.80 | 26.90 | 26.80 |
| RefSpatial | 51.00 | 46.50 | 47.00 | 45.50 |
Table 5: Matched training for alternative visual inputs. All four models use L32 mean, 72K QA, and three nine-way decisions. Each alternative is trained from the same pretrained base.
| \mindlabtablehead Layout | \mindlabtablehead QA/s | \mindlabtablehead Padding (%) | \mindlabtablehead Step CV (%) | \mindlabtablehead Tail (ms) |
|---|---|---|---|---|
| Original order | 99.06 | 35.17 | 2.17 | 39.5 |
| Local length | 116.91 | 12.36 | 4.13 | 157.6 |
| Global balanced | 129.54 | 1.77 | 6.07 | 85.5 |
| Global + buffer 128 | 129.39 | 1.03 | 19.77 | 95.8 |
Table 6: Training efficiency on the historical 72K corpus: eight GPUs, global batch 256, microbatch 8, four accumulation steps. Each condition runs 100 updates; the last 92 updates contain the same 23,552 measured QA. Tail includes communication and waiting. No quality evaluation was performed.
Appendix figures & tables25 assets
Supplementary material from the paper’s appendix.
Appendix
| \mindlabtablehead Source | \mindlabtablehead Choice | \mindlabtablehead Point | \mindlabtablehead Length | \mindlabtablehead Angle | \mindlabtablehead Count | \mindlabtablehead Total |
|---|---|---|---|---|---|---|
| RefSpatial ( Zhou et al., 2025a ) | 6.1M | 3.5M | 1.3M | 108K | 0 | 11M |
| SAT ( Ray et al., 2025 ) | 168K | 0 | 0 | 0 | 0 | 168K |
| SenseNova-SI ( Cai et al., 2026 ) | 424K | 0 | 34K | 2 | 25K | 484K |
| VSI-590K-image ( Yang et al., 2025a ) | 203K | 0 | 27K | 12K | 13K | 255K |
| VST-P ( Yang et al., 2025b ) | 218K | 0 | 170K | 0 | 0 | 388K |
| Total | 7.1M | 3.5M | 1.6M | 120K | 38K | 12.3M |
Table 7: Final r12 corpus after content-only option cleanup. Counts denote original QA, not the number of decision stages. Source names refer to training annotations, including adaptations of existing data.
| \mindlabtablehead Task | \mindlabtablehead QA | \mindlabtablehead Share (%) | \mindlabtablehead Decision rounds |
| Choice | 7.1M | 57.65 | 1 |
| Point | 3.5M | 28.34 | 3 |
| Length | 1.6M | 12.71 | 1–2 |
| Angle | 120K | 0.98 | 1–2 |
| Count | 38K | 0.31 | 2 |
Table 8: Natural task distribution used by the final one-pass training recipe. No task rebalancing or repeated oversampling is applied.
| \mindlabtablehead Item | \mindlabtablehead Configuration |
|---|---|
| Initialization | Official post-trained Qwen3.5-4B; fresh from base |
| Readout | L32 native Q/K mean; no layer fusion |
| Data | r12 content-only; 12.3M QA; each exactly once |
| Global / per-GPU micro / accumulation | 512 / 8 / 8 on eight GPUs |
| Updates / last real batch | 24,032 / 186 QA |
| Backbone / readout LR | / |
Table 9: Recorded configuration for training on the 12.3M-QA corpus.
| \mindlabtablehead Benchmark | \mindlabtablehead Requests | \mindlabtablehead Score and protocol |
|---|---|---|
| SAT real ( Ray et al., 2025 ) | 300 | Mean accuracy over both orders of 150 real-image questions. |
| CV-Bench ( Tong et al., 2024 ) | 2.6K | ( Zhou et al., 2017 ; Lin et al., 2014 ; Brazil et al., 2023 ) ; counting is included in the overall score. |
| RoboSpatial VQA ( Song et al., 2025 ) | 228 | Mean of configuration (123) and compatibility (105) accuracies; RGB input with the current annotations. |
| BLINK val ( Fu et al., 2024 ) | 1.9K | Macro average of all 14 public validation tasks; no test split. Use the paired Hugging Face options and labels. |
| MMSI-Bench ( Yang et al., 2026 ) | 1K | Exact-option accuracy over all 11 types; all 2–10 images in original order; not circular accuracy. |
| RefSpatial ( Zhou et al., 2025a ) | 200 | Binary-mask mean point accuracy; location 100 and placement 100; the overlapping unseen subset is not added again. |
Table 10: Historical 72K pilot evaluation scope. Counts are execution requests.
| \mindlabtablehead Model | \mindlabtablehead SAT | \mindlabtablehead CV-Bench | \mindlabtablehead Robo-VQ | \mindlabtablehead BLINK | \mindlabtablehead MMSI | \mindlabtablehead RefSpatial |
|---|---|---|---|---|---|---|
| MJev-Spatial | 155.1 | 76.5 | 88.9 | 106.6 | 116.0 | 204.8 |
| Spatial-SSRL- Qwen3VL-4B | 237.8 | 127.7 | 123.8 | 124.0 | 139.4 | 682.0 |
| SpatialThinker-7B | 266.4 | 120.8 | 152.3 | 172.4 | 236.8 | 458.8 |
| Qwen3-VL-8B- Instruct | 250.8 | 128.5 | 111.9 | 128.6 | 151.0 | 855.5 |
| SenseNova-SI Qwen3-VL-8B | 223.3 | 115.4 | 102.7 | 105.0 | 114.0 | 858.7 |
| Molmo2-ER | 343.3 | 146.5 | 226.1 | 280.3 | 348.5 | 373.3 |
Table 11: Final-model inference efficiency on the complete twelve-benchmark evaluation sets. Entries are mean single-query latency in milliseconds on one NVIDIA B200, batch size one. Timing includes image I/O, preprocessing, host-to-device transfer, inference/decision, and decoding; warmup and model loading are excluded. The two panels contain the same eight models.
| \mindlabtablehead Metric | \mindlabtablehead AR | \mindlabtablehead Linear | \mindlabtablehead KEV | \mindlabtablehead Mean | \mindlabtablehead Concat | \mindlabtablehead Merge |
|---|---|---|---|---|---|---|
| SAT real | 43.00 | 78.33 | 76.33 | 78.00 | 75.67 | 72.00 |
| CV-Bench | 85.56 | 83.80 | 85.31 | 85.38 | 85.12 | 85.22 |
| RoboSpatial VQA | 56.64 | 60.34 | 68.54 | 68.06 | 66.84 | 65.27 |
| BLINK val | 62.98 | 61.71 | 62.99 | 63.04 | 62.69 | 61.97 |
| MMSI-Bench | 34.20 | 27.70 | 25.10 | 27.70 | 28.90 | 27.10 |
| RefSpatial | 46.00 | 49.00 | 46.00 | 51.00 | 51.00 | 50.00 |
Table 12: Completed Qwen experiments on the fixed evaluation union. All adapted models use the 72K-QA recipe and the historical length codebook; these are not the 12.3M-QA model. Higher is better except VST MAE. Point-Bench is recomputed as the five-category macro average from retained category scores. Original run provenance and differing sequence budgets are retained.
| \mindlabtablehead Category | \mindlabtablehead AR | \mindlabtablehead Linear | \mindlabtablehead KEV | \mindlabtablehead Mean | \mindlabtablehead Concat | \mindlabtablehead Merge |
|---|---|---|---|---|---|---|
| \mindlabtablesection SAT real | ||||||
| action_conseq | 74.32 | 83.78 | 78.38 | 78.38 | 75.68 | 77.03 |
| ego_movement | 8.70 | 95.65 | 100.00 | 100.00 | 100.00 | 67.39 |
| goal_aim | 48.53 | 95.59 | 92.65 | 94.12 | 91.18 | 94.12 |
| obj_movement | 41.30 | 76.09 | 73.91 | 73.91 | 65.22 | 69.57 |
| perspective | 27.27 | 43.94 | 42.42 | 48.48 | 50.00 | 48.48 |
Table 13: Category-level main comparison. Percentages use each task's recorded scorer. Historical length codebook and root-pruned Point-Bench. Categories do not share a common difficulty scale.
Figure 4: \captionlead The complete depth/aggregation experiment, shown on three representative metrics. Early visual readouts are inadequate, while later layers exhibit task-dependent rather than monotonic gains. All 24 models’ benchmark values are provided in Section E.3 .
| \mindlabtablehead Metric | \mindlabtablehead L32 | \mindlabtablehead +L28 | \mindlabtablehead +L24 | \mindlabtablehead +L28+L24 |
|---|---|---|---|---|
| SAT real | 78.00 | 75.33 | 76.33 | 78.67 |
| CV-Bench | 85.31 | 84.66 | 83.46 | 85.31 |
| RoboSpatial VQA | 68.06 | 67.18 | 66.70 | 65.62 |
| BLINK val | 63.04 | 62.25 | 63.37 | 62.03 |
| MMSI-Bench | 27.80 | 27.00 | 27.00 | 27.70 |
| RefSpatial | 51.00 | 47.50 | 50.00 | 48.00 |
Table 14: Equal-weight native Q/K layer fusion. Each fusion is trained independently; L32 is re-evaluated with the same evaluator. Point-Bench uses the five-category macro average.
| \mindlabtablehead Readout | \mindlabtablehead SAT | \mindlabtablehead CV | \mindlabtablehead Robo VQA | \mindlabtablehead BLINK | \mindlabtablehead MMSI |
|---|---|---|---|---|---|
| L4 mean | 52.67 | 51.41 | 72.32 | 40.34 | 27.00 |
| L4 concat | 49.67 | 52.02 | 64.51 | 41.46 | 26.30 |
| L4 merge | 49.67 | 52.48 | 71.03 | 41.24 | 28.30 |
| L8 mean | 49.67 | 58.87 | 66.07 | 41.37 | 25.10 |
| L8 concat | 53.33 | 59.71 | 50.38 | 39.58 | 26.20 |
| L8 merge | 52.33 | 62.39 | 73.61 | 40.65 | 24.50 |
Table 15: Complete depth/readout sweep: classification metrics. Every model uses the same QA budget and aligned recipe. The retained parameter count varies with depth. Point-Bench is macro.
| \mindlabtablehead Readout | \mindlabtablehead RefSpatial | \mindlabtablehead W2P | \mindlabtablehead Robo point | \mindlabtablehead PB macro | \mindlabtablehead MAE (m) |
|---|---|---|---|---|---|
| L4 mean | 4.50 | 13.96 | 4.10 | 12.38 | 0.6480 |
| L4 concat | 3.00 | 13.96 | 4.92 | 12.07 | 0.6397 |
| L4 merge | 6.50 | 12.96 | 5.74 | 10.15 | 0.6417 |
| L8 mean | 7.00 | 14.95 | 17.21 | 17.58 | 0.4895 |
| L8 concat | 15.00 | 29.93 | 28.69 | 25.92 | 0.4833 |
| L8 merge | 16.00 | 33.91 | 31.15 | 28.58 | 0.4894 |
Table 16: Complete depth/readout sweep: pointing metrics. Every model uses the same QA budget and aligned recipe. The retained parameter count varies with depth. Point-Bench is macro.
Figure 5: \captionlead Centered candidate-logit RMS for each head, averaged per QA along the all-head trajectory. Values share a common scale and measure response amplitude. H02 is strongest and H11 weakest in all three task groups.
| \mindlabtablehead Head | \mindlabtablehead Class RMS | \mindlabtablehead Point RMS | \mindlabtablehead Length RMS | \mindlabtablehead Class pp | \mindlabtablehead Point pp | \mindlabtablehead Length m |
|---|---|---|---|---|---|---|
| H01 | 2.324 | 3.938 | 5.742 | -0.011 | +0.127 | +0.000000 |
| H02 | 2.394 | 4.065 | 5.954 | +0.009 | +0.127 | +0.000000 |
| H03 | 2.328 | 3.932 | 5.708 | +0.020 | +0.127 | +0.000000 |
| H04 | 2.368 | 4.022 | 5.917 | +0.009 | +0.127 | +0.000279 |
| H05 | 1.202 | 2.055 | 3.236 | +0.050 | +0.102 | -0.000346 |
| H06 | 1.180 | 2.032 | 3.232 | +0.002 | +0.102 | -0.003992 |
Table 17: All 16 single-head interventions. Positive effect means retaining the head helps: full-minus-masked score, or masked-minus-full length MAE. Classification/pointing effects are benchmark-equal diagnostic means; RMS values average per question.
| \mindlabtablehead Metric | \mindlabtablehead Crop | \mindlabtablehead Original only | \mindlabtablehead Original + cache |
|---|---|---|---|
| RefSpatial (%) | 51.00 | 9.00 | 9.00 |
| Where2Place (%) | 59.98 | 32.00 | 32.00 |
| RoboSpatial pointing (%) | 62.30 | 20.49 | 20.49 |
| Point-Bench macro (%) | 56.84 | 21.96 | 21.96 |
| Counting success (%) | 39.29 | 4.08 | 4.08 |
| Count MAE | 1.7296 | 4.0459 | 4.0561 |
Table 18: Fixed crop-trained native-mean checkpoint: removing new pixel crops, with and without a reusable original-image prefix. All three textual grid rounds remain. The crop control was not timed in this run.
| \mindlabtablehead Metric | \mindlabtablehead K argmax | \mindlabtablehead K comp. | \mindlabtablehead K task | \mindlabtablehead N argmax | \mindlabtablehead N comp. | \mindlabtablehead N task |
|---|---|---|---|---|---|---|
| RefSpatial | 38.50 | 35.14 | 38.50 | 45.50 | 43.65 | 45.50 |
| Where2Place | 67.96 | 64.43 | 64.43 | 60.91 | 58.99 | 58.99 |
| RoboSpatial pointing | 56.56 | 57.38 | 57.38 | 60.66 | 63.11 | 63.11 |
| Point-Bench (macro) | 46.46 | 26.81 | 49.12 | 50.76 | 32.71 | 54.63 |
| PB counting | 13.27 | 26.53 | 26.53 | 12.76 | 32.14 | 32.14 |
Table 19: Direct visual-token pointing with KEV (K) or native mean (N). Each model shares weights/logits across argmax, components, and task-based decoding. Task routing was specified after the first decoding experiment; Point-Bench is macro.
| \mindlabtablehead Metric | \mindlabtablehead KEV grid | \mindlabtablehead KEV visual | \mindlabtablehead Mean grid | \mindlabtablehead Mean visual |
|---|---|---|---|---|
| SAT real | 76.33 | 76.67 | 78.00 | 77.00 |
| CV-Bench | 85.31 | 83.95 | 85.38 | 84.52 |
| RoboSpatial VQA | 68.54 | 64.32 | 68.06 | 62.50 |
| BLINK val | 62.99 | 63.87 | 63.04 | 62.76 |
| MMSI-Bench | 25.10 | 25.50 | 27.70 | 30.20 |
| RefSpatial | 46.00 | 38.50 | 51.00 | 45.50 |
Table 20: Visual-token pointing versus hierarchical crop pointing. Visual-token results use task-based single/multiple-point routing established after the initial decoding experiment. This routing differs from the grid baseline on Where2Place and RoboSpatial; it is an exploratory comparison.
| \mindlabtablehead Metric | \mindlabtablehead Continue/stop | \mindlabtablehead Full tree | \mindlabtablehead Root-pruned |
|---|---|---|---|
| Official success (%) | 19.39 | 38.78 | 38.78 |
| Exact count (%) | 22.45 | 46.43 | 46.43 |
| Count MAE | 11.643 | 1.735 | 1.740 |
| Predicted points | 10.485 | 3.342 | 3.337 |
| Decision nodes | – | 91.00 | 51.10 |
| Evaluated cells | – | 729.00 | 405.83 |
Table 21: Point-Bench counting: all 196 questions, fixed L32 mean weights, no retraining. Full/pruned are an actual paired run; historical continue/stop quality uses a different execution setting and has no comparable timing here.
Figure 6: \captionlead Probability-tree example from the first request in the preselected 12-case gallery. Full expansion uses 91 nodes; root pruning uses 51 and marks unvisited regions in gray. Both return four points in this case. Example from Point-Bench ( Cheng et al., 2025 ) .
| \mindlabtablehead Peak threshold | \mindlabtablehead Full success | \mindlabtablehead Pruned success | \mindlabtablehead Full count | \mindlabtablehead Pruned count | \mindlabtablehead Full MAE | \mindlabtablehead Pruned MAE |
|---|---|---|---|---|---|---|
| 5% | 30.10 | 30.10 | 34.69 | 34.69 | 2.041 | 2.041 |
| 10% | 38.78 | 38.78 | 46.43 | 46.43 | 1.735 | 1.740 |
| 20% | 37.24 | 37.24 | 43.88 | 43.88 | 1.765 | 1.760 |
Table 22: Offline component-threshold sensitivity on the retained maps. Success and exact-count columns are percentages. The primary threshold is fixed at 10%.
| \mindlabtablehead VST-300 diagnostic | \mindlabtablehead Historical | \mindlabtablehead | \mindlabtablehead 4/8 argmax | \mindlabtablehead 4/8 exp. |
|---|---|---|---|---|
| MAE (m), lower | 0.335420 | 0.327732 | 0.320207 | 0.329955 |
| AbsRel (%), lower | 24.64 | 31.04 | 29.03 | 30.04 |
| m (%), higher | 56.67 | 58.33 | 59.00 | 55.67 |
| (%), higher | 69.46 | 69.46 | 68.12 | 70.81 |
Table 23: Fixed internal length diagnostic. 4/8 argmax is the decoder used by MJev-Spatial; 4/8 exp. decodes the same logits with a fine-stage conditional expectation. Coarse decoding is argmax in every condition.
| \mindlabtablehead Metric | \mindlabtablehead Historical | \mindlabtablehead | \mindlabtablehead |
|---|---|---|---|
| SAT real | 78.00 | 72.33 | 80.67 |
| CV-Bench | 85.39 | 85.08 | 85.21 |
| RoboSpatial VQA | 68.06 | 64.87 | 66.30 |
| BLINK val | 63.10 | 60.94 | 62.52 |
| MMSI-Bench | 27.70 | 25.40 | 27.60 |
| RefSpatial | 51.00 | 49.50 | 52.00 |
Table 24: Length-codebook ablation, independently trained on the same 72K QA. The historical checkpoint is re-evaluated within this experiment. All point and choice decoders are unchanged.
| \mindlabtablehead Representation | \mindlabtablehead MAE (m) | \mindlabtablehead P99 (m) | \mindlabtablehead Max (m) | \mindlabtablehead cm (%) | \mindlabtablehead rel. |
|---|---|---|---|---|---|
| Historical 200 m | 1.6548 | 1.2384 | 8320.10 | 59.33 | 80.29 |
| Uniform 960 | 6.4545 | 8.5012 | 8.53 | 0.06 | 1.12 |
| Log 384 | 0.0939 | 1.5758 | 43.10 | 47.53 | 52.36 |
| Log 960 | 0.0379 | 0.6371 | 47.14 | 69.86 | 99.92 |
| Quantile 960 | 4.2371 | 3.0380 | 3935.55 | 80.78 | 95.50 |
| Hierarchical log1p 960 | 0.0391 | 0.6657 | 21.39 | 69.42 | 99.84 |
Table 25: Training-label reconstruction on all 1.6M lengths, assuming the correct discrete cell is selected. The 960-leaf alternatives differ in allocation; only the hierarchical log1p proposal is tested in the subsequent matched model experiment. Task-specific routing is an unadopted proposal.
Figure 7: \captionlead Controlled scheduling comparison on the historical 72K corpus. Local grouping reduces padding but leaves more rank imbalance; global grouping and balancing deliver the largest throughput gain. A cross-update buffer reduces padding slightly further without increasing throughput.
| \mindlabtablehead Global | \mindlabtablehead Micro | \mindlabtablehead Accum. | \mindlabtablehead Outcome | \mindlabtablehead s/update | \mindlabtablehead QA/s | \mindlabtablehead Peak GB |
| 256 | 8 | 4 | Complete | 2.3007 | 111.27 | 242.03 |
| 512 | 8 | 8 | Complete | 3.7629 | 136.07 | 242.05 |
| 256 | 16 | 2 | OOM | – | – | – |
| 512 | 16 | 4 | OOM | – | – | – |
| 256 | 32 | 1 | OOM | – | – | – |
| 512 | 32 | 2 | OOM | – | – | – |
Table 26: Expanded-data memory feasibility. The original six cases use 4,096 warmup and 8,192 measured QA; the additional global384/micro12 trial also fails. Partial timings from failed configurations are excluded. The successful rows are resummarized on a common window in Table 27 .
| \mindlabtablehead Global | \mindlabtablehead Micro | \mindlabtablehead Accum. | \mindlabtablehead Timed steps | \mindlabtablehead s/update | \mindlabtablehead QA/s | \mindlabtablehead Peak GB |
| 256 | 8 | 4 | 30 | 2.3265 | 110.04 | 242.03 |
| 512 | 8 | 8 | 15 | 3.7791 | 135.48 | 242.05 |
| 768 | 8 | 12 | 10 | 5.4944 | 139.78 | 242.03 |
Table 27: New-data profiling with the 4/8 length codebook: a common 7,680-QA window after 4,608 QA warmup. The 12,800-QA pool is a stratified prefix sample, not the natural full-data distribution. Peak is the maximum per-rank allocated GPU memory, in decimal GB. Microbatch 16 and 32 failed with OOM in both original 256/512 conditions.