Readout is not Recovery: Dissociating Coordinate Emission from Visual-Corruption Repair in Vision-Language Models

Drandreb Earl Juanico

cs.AISep 26, 2026

Readout is not Recovery: Dissociating Coordinate Emission from Visual-Corruption Repair in Vision-Language Models

Authors: Drandreb Earl Juanico

Abstract

VLM bounding-box localization is both language generation and spatial commitment. Parseable fields such as bbox_2d make localization easy to score, but dimensions that emit coordinate tokens need not repair localization after visual evidence is damaged. We study this readout/recovery separation in Qwen3-VL-4B-Instruct on single-object COCO grounding. We compare clean coordinate-token readout rankings with corruption-derived repair rankings, using object-mask endpoint replacement for recovery and clean-input flooring for depth localization. In Qwen3-VL, coordinate-token rankings are inert through layer 24, load-bearing from layers 32-35, and peak at layer 34; corruption-derived rankings harm layers 16-24 but become beneficial near layer 35/final. A Kimi-VL-A3B diagnostic shows a matching output-proximal transition despite a different box format. Object-mask recovery separates rank budgets: $k=250$ shows necessity, $k=500$ shows Top- $k$ restoration above random, and $k=d/2$ is largely capacity-driven. Partial-occlusion sweeps reveal that high-overlap coordinate-token sets can hurt at $k=1000$ and help mainly at half-width, while population corruption-derived sets provide no reliable fixed repair set. Edge-attribution patching shows coordinate-token paths are high precision but low recall for detection recovery, and RMSNorm quasi-layer controls do not close the endpoint-repair gap. Endpoint coordinate triage is therefore a useful circuit prior, but occlusion recovery requires a separate benchmark.

Figures & tables

Panel of Figure 1 — Figure 1: Illustration of the two corruption settings. Object mask is the hard corruption: the queried object region is blanked, removing most local evidence for the target. Partial occlusion is the softer corruption: a patch is sampled inside the ground-truth box, leaving part of the queried object visible. This distinction motivates treating object mask as the hard repair benchmark and partial occlusion as the more practical occlusion-like sensitivity setting.

Layer	$n$	Base	$\Delta$ Full	$\Delta$ Top	$\Delta$ Comp	$z_{\mathrm{top}}$	$z_{\mathrm{comp}}$	Interpretation
$8$	$3741$	$.5871$	$-.0014$	$+.0022$	$-.0002$	$+1.34$	$-.07$	null; controls near baseline
$16$	$3741$	$.5871$	$-.0595$	$+.0014$	$-.0112$	$+.41$	$\mathbf{-2.19}$	complement harmful
$17$	$3741$	$.5871$	$-.0435$	$+.0036$	$-.0061$	$+.20$	$-1.95$	mild complement harm
$21$	$3741$	$.5871$	$\mathbf{-.2243}$	$+.0004$	$\mathbf{-.0540}$	$+.97$	$\mathbf{-3.70}$	complement-load-bearing
$24$	$2997$	$.5904$	$\mathbf{-.2692}$	$+.0101$	$\mathbf{-.0368}$	$+.87$	$\mathbf{-3.98}$	complement-load-bearing
$32$	$2984$	$.5899$	$\mathbf{-.4603}$	$\mathbf{-.0340}$	$+.0006$	$\mathbf{-11.84}$	$-.83$	top load-bearing; full collapse

Table 1: Clean-input depth triage on Qwen3-VL using coordinate-token rankings. The Qwen Top-1280 set is null or protective through layer 17, complement-load-bearing at layers 21–24, and top-load-bearing from layers 32–35, with peak specificity at layer 34. The schema mirrors Table 2 .

Layer	$n$	Base	$\Delta$ Full	$\Delta$ Top	$\Delta$ Comp	$z_{\mathrm{top}}$	$z_{\mathrm{comp}}$	Interpretation
$6$	$1502$	$.210$	$+.006$	$+.002$	$+.003$	$-.75$	$+.75$	early null
$12$	$1502$	$.210$	$+.006$	$+.008$	$+.004$	$+1.51$	$+.12$	mild/nonselective benefit
$13$	$1502$	$.210$	$+.004$	$+.005$	$+.003$	$+.70$	$-.44$	early null
$16$	$1502$	$.210$	$-.001$	$+.005$	$+.007$	$+.56$	$+.88$	nonselective helpful
$18$	$1502$	$.210$	$+.011$	$+.007$	$+.004$	$+1.18$	$-.25$	early null
$24$	$1502$	$.210$	$+.005$	$-.008$	$\mathbf{+.011}$	$-.94$	$\mathbf{+1.42}$	transition onset

Table 2: Clean-input depth triage on Kimi-VL-A3B using coordinate-token rankings. The Kimi Top-1024 set is the half-width analogue of Qwen’s Top-1280 set. The schema mirrors Table 1 ; entries are split-mean values.

Layer	$S$	$n$	Base	$\Delta$ Full	Parse(full)	$\Delta$ Top	$z_{\mathrm{top}}$	$\Delta$ Comp	$z_{\mathrm{comp}}$	Selectivity gap
$8$	$5$	$3741$	$.5871$	$-.0014$	$.974$	$+.0001$	$-1.26$	$+.0010$	$+.67$	$-.0009$
$16$	$5$	$3741$	$.5871$	$-.0595$	$.881$	$\mathbf{-.0731}$	$\mathbf{-8.78}$	$+.0019$	$+.60$	$\mathbf{-.0750}$
$17$	$5$	$3741$	$.5871$	$-.0435$	$.882$	$\mathbf{-.0623}$	$\mathbf{-8.02}$	$-.0001$	$-.28$	$\mathbf{-.0623}$
$21$	$4$	$2997$	$.5904$	$-.2279$	$.587$	$\mathbf{-.3412}$	$\mathbf{-44.68}$	$-.0015$	$+.68$	$\mathbf{-.3397}$
$24$	$3$	$2240$	$.5952$	$-.2704$	$.476$	$\mathbf{-.2590}$	$\mathbf{-20.69}$	$+.0006$	$+.88$	$\mathbf{-.2597}$
$32$	$4$	$2984$	$.5899$	$-.4603$	$.216$	$-.0043$	$\mathbf{-2.19}$	$+.0008$	$-.56$	$-.0051$

Table 3: Clean-input depth triage using population corruption-derived rankings. The Top-1280 set is null at layer 8, harmful across layers 16–24, transitional at layer 32, necessary before sufficient at layers 33–34, and dual-criterion load-bearing at layer 35; the final site retains a weaker sufficiency signal.

S

is the number of held-out splits contributing to the row. Parse(full) reports parseability under full flooring. The selectivity gap is

\Delta

Top

-\Delta

Comp.

Ranking source	$k$	Top	Random	Bottom	Complement	Comp-random	$z_{\mathrm{top}}$ / $z_{\mathrm{comp}}$	Interpretation
Coordinate-token	$250$	$3.27\pm 1.31$	$2.97\pm.76$	$2.02\pm.73$	$\mathbf{90.31\pm.81}$	$\mathbf{94.79\pm.96}$	$+.92$ / $\mathbf{-5.60}$	necessity only
Coordinate-token	$500$	$\mathbf{10.60\pm 2.05}$	$\mathbf{8.77\pm 1.67}$	$6.00\pm 1.57$	$90.60\pm.61$	$91.04\pm.64$	$\mathbf{+2.35}$ / $-.37$	Top- $k$ restoration emerges
Coordinate-token	$1280$	$76.67\pm 2.74$	$73.73\pm 2.41$	$72.15\pm 3.80$	$72.15\pm 3.80$	$74.44\pm 2.57$	$+1.19$ / $-1.00$	capacity-dominated
Corruption-derived	$250$	$2.89\pm 0.84$	$2.93\pm 0.72$	$3.58\pm 1.08$	$\mathbf{89.01\pm 0.87}$	$\mathbf{94.31\pm 0.96}$	$-.17$ / $\mathbf{-8.41}$	exclusion cost only
Corruption-derived	$500$	$\mathbf{10.95\pm 1.68}$	$\mathbf{8.76\pm 1.49}$	$9.40\pm 1.32$	$\mathbf{88.64\pm 0.77}$	$\mathbf{91.04\pm 0.57}$	$\mathbf{+2.62}$ / $\mathbf{-2.09}$	sufficiency and necessity
Corruption-derived	$1280$	$69.80\pm 2.75$	$69.00\pm 2.66$	$68.29\pm 4.27$	$68.29\pm 4.27$	$69.76\pm 2.45$	$+.31$ / $-.79$	capacity-dominated

Table 4:

k

-scaling endpoint recovery under clean endpoint replacement. Entries are recovery percentages relative to the object-mask baseline and full-recovery ceiling;

\pm

reports standard deviation across per-seed means where shown.

z_{\mathrm{top}}

and

z_{\mathrm{comp}}

compare Top-

k

and complement against matched random controls. For

k=1280=d/2

, Bottom-

k

and complement are identical.

Ranking source	$k$	Random	Fixed global	Split-local range	Sample-local readout	Sample-specific corruption	Bottom-ranked
Coordinate-token	$500$	$.1200\pm.0035$	$\mathbf{.1300\,(+2.87)}$	$\mathbf{.1283\text{--}.1316\,(+2.38\text{--}+3.35)}$	$.1236\,(+1.04)$	$\mathbf{.1501\,(+8.66)}$	$.1064\,(-3.91)$
Coordinate-token	$1000$	$.3542\pm.0600$	$\mathbf{.2423\,(-1.86)}$	$\mathbf{.2294\text{--}.2533\,(-2.08\text{--}-1.68)}$	$.2480\,(-1.77)$	$\mathbf{.4106\,(+0.94)}$	$.3124\,(-0.69)$
Coordinate-token	$1280$	$.4547\pm.0122$	$.4730\,(+1.49)$	$.4651\text{--}.4777\,(+0.85\text{--}+1.88)$	$.4671\,(+1.01)$	$.4685\,(+1.13)$	$.4483\,(-0.53)$
Corruption-derived	$500$	$.119\pm.0035$	$\mathbf{.130\,(+2.95)}$	$\mathbf{.128\text{--}.141\,(+2.34\text{--}+6.30)}$	$.124\,(+1.17)$	$\mathbf{.149\,(+8.64)}$	$.123\,(+0.92)$
Corruption-derived	$1000$	$.356\pm.0590$	$\mathbf{.401\,(+0.77)}$	$\mathbf{.391\text{--}.408\,(+0.59\text{--}+0.88)}$	$.253\,(-1.75)$	$\mathbf{.412\,(+0.94)}$	$.273\,(-1.40)$
Corruption-derived	$1280$	$.456\pm.0118$	$.465\,(+0.76)$	$.456\text{--}.471\,(-0.03\text{--}+1.28)$	$.472\,(+1.32)$	$.471\,(+1.25)$	$.449\,(-0.64)$

Table 5: Object-mask endpoint recovery with local index sets under the primary clean endpoint-replacement operator. Entries report

R_{50}

; parentheses give the

z

statistic against the matched random control. Fixed global means one Top-

k

set reused for all test samples; split-local means one fixed set per split; sample-local readout is recomputed from the current clean test trace; sample-specific corruption is recomputed from the current clean–corrupt pair; bottom-ranked is the Bottom-

k

set from the corresponding fixed ranking.

Panel A: coordinate-token ranking family and matched alternatives under partial occlusion
$k$	Random- $k$	Fixed global	Split-local mean	Sample-local readout	Sample-specific corruption	Bottom-ranked
$500$	$.5101$	$.5072\,(-1.00;\,J=1.000)$	$.5088\,(-.48;\,J=.968)$	$\mathbf{.5140\,(+1.32;\,J=.072)}$	$.5073\,(-1.05;\,J=.257)$	$.5078\,(-.78;\,J=.000)$
$1000$	$.5208$	$\mathbf{.5032\,(-2.53;\,J=1.000)}$	$\mathbf{.5022\,(-2.64;\,J=.982)}$	$.5232\,(+.37;\,J=.223)$	$\mathbf{.5108\,(-1.41;\,J=.398)}$	$.5241\,(+.50;\,J=.000)$
$1280$	$.5212$	$\mathbf{.5289\,(+2.42;\,J=1.000)}$	$\mathbf{.5279\,(+2.00;\,J=.982)}$	$\mathbf{.5313\,(+3.24;\,J=.314)}$	$.5202\,(-.43;\,J=.470)$	$.5227\,(+.85;\,J=.000)$

Table 6: Partial-occlusion local-set recovery with explicit index-set terminology. A rectangular mean-color patch covers

40\%

of the ground-truth box. Entries report mean

R_{50}

and mean within-split

z

against ten random controls; Panel A also reports Jaccard overlap

J

with the fixed global coordinate-token Top-

k

set. A fixed global or fixed population set is reused for every test sample; only the score used to construct that fixed set differs by panel.

$k$	Repair pre/post overlap	Post endpoint-repair	Pre endpoint-repair	Pre-post
$500$	$\mathbf{.942\pm.008}$	$.705\pm.013$	$.713\pm.015$	$\mathbf{+.008}$
$1000$	$\mathbf{.961\pm.004}$	$.843\pm.008$	$.843\pm.009$	$\mathbf{+.001}$
$1280$	$\mathbf{.970\pm.003}$	$.882\pm.006$	$.877\pm.004$	$\mathbf{-.004}$

Table 7: Quasi-layer RMSNorm control using bbox_2d readout rankings. The table compares repair rankings before/after making RMSNorm explicit, and endpoint-readout rankings against repair rankings in post-norm and transported pre-norm spaces.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Qwen coordinate-token depth triage, layer 34 ( $k=1280$ , $n\approx 750$ per split)
Experiment	Condition	$R_{50}$	Parse %	$R_{50}\mid\mathrm{parse}$	Evaluation cells
	Baseline (no flooring)	$.568$	$97.9$	$.580$	$3$ held-out splits
	Top coordinate-token floored	$\mathbf{.408}$	$\mathbf{89.5}$	$\mathbf{.456}$	$3$ held-out splits
	Complement floored	$.558$	$97.0$	$.575$	$3$ held-out splits
	Random, size-matched	$.568$	$97.9$	$.580$	$3$ held-out splits
Kimi-VL-A3B coordinate-token depth triage ( $k=1024$ , $n=735$ – $767$ )

Appendix

Table 8: Parseability decomposition for representative intervention regimes.

R_{50}

counts unparseable outputs as zero;

R_{50}\mid\mathrm{parse}

evaluates only parseable generations. Kimi-VL-A3B uses a bracketed normalized-coordinate format rather than Qwen’s bbox_2d JSON convention, so its baseline parseable rate is lower than Qwen’s in these diagnostics.

Pool comparison	$k=500$	$k=1000$	$k=1280$
main pool replica vs. full ranking	$\mathbf{.953}$	$\mathbf{.970}$	$\mathbf{.966}$
main pool vs. digit-only pool	$\mathbf{.244}$	$\mathbf{.410}$	$\mathbf{.502}$
main pool vs. coordinate-format pool	$\mathbf{.439}$	$\mathbf{.548}$	$\mathbf{.614}$
digit-only pool vs. coordinate-format pool	$\mathbf{.247}$	$\mathbf{.428}$	$\mathbf{.492}$

Appendix

Table 9: Negative-set sensitivity of coordinate-token rankings. Entries report Jaccard overlap between Top-

k

hidden-dimension sets produced by the main position-specific negative pool and rankings produced by alternative negative-token pools. The check uses one held-out split with

n=147

valid generations. The first row compares a small-sample replica of the main-pool ranking with the corresponding full-scale main-pool ranking, isolating negative-pool effects from sample-size noise.

Condition	$k=1000$			$k=1280$
	$20\%$	$40\%$	$60\%$	$20\%$	$40\%$	$60\%$
Fixed population corruption-derived	$-.004$	$+.007$	$+.077^{\dagger}$	$+.002$	$-.010^{\dagger}$	$+.010$
Fixed global coordinate-token	$-.032^{\dagger}$	$-.043^{\dagger}$	$-.097^{\dagger}$	$+.010$	$+.007$	$+.012$
Split-local population corruption-derived	$-.010$	$+.011$	$+.043^{\dagger}$	$+.002$	$-.009^{\ddagger}$	$+.008$
Split-local coordinate-token	$-.026^{\dagger}$	$-.043^{\dagger}$	$-.032^{\dagger}$	$+.015^{\ddagger}$	$+.007$	$+.001$
Sample-local readout	$+.016$	$+.006$	$+.088^{\dagger}$	$+.002$	$+.019$	$+.013$

Appendix

Table 10: Final partial-occlusion severity sweep comparing

k=1000

and

k=1280

. Entries are

\Delta R_{50}

offsets relative to the corresponding no-restoration partial-occlusion baseline over five held-out splits with

150

samples per split. The ablation varies both restoration budget and occlusion area fraction while using the same generic index-set scopes as Table 6 .

$n_{\mathrm{edges}}$	$\Delta R_{50}$ Coordinate-token Top- $k$	$\Delta R_{50}$ Complement	$\Delta R_{50}$ Random- $k$	$\Delta J_{\Delta}$ Coordinate-token Top- $k$	$\Delta J_{\Delta}$ Complement	$\Delta J_{\Delta}$ Random- $k$
$10$	$-.002$	$-.021$	$-.010$	$+3621$	$+3208$	$+3888$
$25$	$-.019$	$-.018$	$-.016$	$+2321$	$+2216$	$+3618$
$50$	$-.020$	$-.022$	$-.019$	$-5140$	$-3763$	$+2315$
$100$	$\mathbf{-.034}$	$-.029$	$-.032$	$-10906$	$-8799$	$-13078$
$200$	$-.097$	$\mathbf{-.192}$	$\mathbf{-.166}$	$\mathbf{-33548}$	$-16025$	$-15411$

Appendix

Table 11: Edge-attribution patching validation under

40\%

partial occlusion. The top-

n

edges from each endpoint objective are zero-ablated at inference on

300

held-out examples.

\Delta R_{50}

is relative to the unablated partial-occlusion run, and

\Delta J_{\Delta}

is the change in the corresponding endpoint-gap objective. Bold entries mark the most diagnostic cells discussed in the main text.

Panel of Figure 2 — Figure 2: Circuit heatmaps with boundary-path overlays for the edge-attribution patching diagnostic. Columns show coordinate-token Top- $k$ , complement, and random- $k$ endpoint objectives; the top row shows attention heatmaps and the bottom row shows MLP heatmaps. The six panels use the individual heatmap PDFs for each objective and component type, preserving their titles, axes, overlays, and colorbars. The shared colorscale is anchored to the coordinate-token Top- $k$ condition, making the complement and random objectives visibly dimmer when their absolute edge scores are lower.

$k$	Pre/post repair overlap	Post endpoint-repair overlap	Pre endpoint-repair overlap	$\Delta$ pre-post
$500$	$\mathbf{.942\pm.008}$	$.705\pm.013$	$.713\pm.015$	$\mathbf{+.008}$
$1000$	$\mathbf{.961\pm.004}$	$.843\pm.008$	$.843\pm.009$	$\mathbf{+.001}$
$1280$	$\mathbf{.970\pm.003}$	$.882\pm.006$	$.877\pm.004$	$\mathbf{-.004}$

Appendix

Table 12: Supplementary detail for the quasi-layer RMSNorm ranking control using bbox_2d readout rankings. The pre/post repair column compares repair-relevant rankings across the RMSNorm boundary. Endpoint-repair columns compare endpoint readout rankings with repair rankings before and after VJP transport. All values are means

\pm

standard deviations across five bootstrap seeds.

Ranking source	$k$	Random	Fixed global	Split-local range	Sample-local readout	Sample-specific corruption	Bottom-ranked
Coordinate-token	$500$	$.1003\pm.0034$	$.0943\,(-1.75)$	$.0910\text{--}.0970\,(-2.74\text{--}-0.96)$	$.0945\,(-1.71)$	$.1021\,(+0.54)$	$.0965\,(-1.13)$
Coordinate-token	$1000$	$.1734\pm.0046$	$\mathbf{.1447\,(-6.27)}$	$\mathbf{.1396\text{--}.1491\,(-7.38\text{--}-5.31)}$	$.1456\,(-6.07)$	$\mathbf{.2631\,(+19.65)}$	$.1637\,(-2.11)$
Coordinate-token	$1280$	$.3196\pm.0748$	$.2013\,(-1.58)$	$.1967\text{--}.2076\,(-1.64\text{--}-1.50)$	$.2963\,(-0.31)$	$.3925\,(+0.98)$	$.3441\,(+0.33)$
Corruption-derived	$500$	$.100\pm.0034$	$.098\,(-0.69)$	$.095\text{--}.099\,(-1.52\text{--}-0.44)$	$.094\,(-1.71)$	$.102\,(+0.53)$	$.100\,(-0.04)$
Corruption-derived	$1000$	$.174\pm.0046$	$.179\,(+1.20)$	$.171\text{--}.176\,(-0.48\text{--}+0.49)$	$.146\,(-6.07)$	$\mathbf{.263\,(+19.65)}$	$\mathbf{.191\,(+3.72)}$
Corruption-derived	$1280$	$.320\pm.0748$	$.376\,(+0.75)$	$.368\text{--}.393\,(+0.65\text{--}+0.98)$	$.297\,(-0.31)$	$.393\,(+0.98)$	$.250\,(-0.93)$

Appendix

Table 13: Supplementary-only thresholded clean-replacement ablation for local-set endpoint recovery at

\theta=-0.5

. Entries report

R_{50}

; parentheses give the

z

statistic against the matched random control. Short column headers follow the same index-set definitions as Tables 5 and 6 . These results are not the primary endpoint-recovery results in Table 5 .

Vision-Language Grounding

Vision-Language Grounding links words and phrases to the objects, regions, actions, or moments they describe in images, video, and scenes.

See all 420 papers

LOCUS: Locate Once, Check Until Settled. Draft-and-Verify Visual Grounding with a Box Head and Coordinate-Token Verification
Oct 26, 2026Muhammad TalhaVision-Language Grounding Visual Grounding
Independent Researcher
Hi-Token: Hierarchical Coordinate Tokenization for Generative Visual Grounding
Aug 4, 2026Xiuyuan Zhu, Ke Lu, Kun Dong +6Vision-Language Grounding
University of Chinese Academy of Sciences, Beijing, China · State Key Laboratory of Communication Content Cognition, Beijing, China · Peng Cheng Laboratory, Shenzhen, Guangdong, China +1
What Looks Like a Capability Limit in Vision-Language Models Is a Readout Limit
Sep 23, 2026Alfredo F. Frontera Del ValleVLM Evaluation Vision-Language Grounding
Columbia University

Oct 26, 2026J/K move · Enter open · S save

LOCUS: Locate Once, Check Until Settled. Draft-and-Verify Visual Grounding with a Box Head and Coordinate-Token Verification

Muhammad Talha

Independent Researcher

Vision-Language Grounding Visual Grounding Efficient VLM Inference Speculative Decoding

Open paper AskPDF arXiv

Vision-language models (VLMs) ground a referring expression in one of two ways. They decode the box as coordinate tokens, which is precise but takes one sequential step per token, or they regress it from a hidden state with a small head, which takes one step but loses accuracy at high IoU. The two fail differently: a head usually finds the right object but places its edges loosely, while coordinate tokens place edges precisely. We therefore use the regression head as a speculative drafter for the model's own coordinate tokens. Locus writes the head's box as coordinate tokens and checks them with the same backbone by Jacobi iteration over a shared image prefix. The fixed point is the model's greedy coordinate-token answer, reached in about 5.60 sequential passes on average instead of 18.46 decoding steps; head and verifier are one set of weights, trained jointly. On the 8 RefCOCO, RefCOCO+ and RefCOCOg splits, verification raises Locus-2B's [email protected] from 54.87 for its own head to 59.13, and the one-pass head is ahead at [email protected] of every published model with at most 3B parameters except Florence-2-L. Against Jag, the strongest open regression-head model, at its own size and training budget, Locus ties at [email protected] and, with typical acceptance, gains +4.52 points at [email protected], with paired significance tests. Exact answers come 1.5x faster than vLLM greedy-decoding the same coordinate tokens with the same weights; a pure regression head remains faster still.

Abstract

Vision-Language Grounding

Explore similar work

LOCUS: Locate Once, Check Until Settled. Draft-and-Verify Visual Grounding with a Box Head and Coordinate-Token Verification

Hi-Token: Hierarchical Coordinate Tokenization for Generative Visual Grounding

What Looks Like a Capability Limit in Vision-Language Models Is a Readout Limit