Seeing, Saying, but Not Using: From Reportable Spatial Facts to Usable States in Multimodal Large Language Models

Jinchang Zhang; Guoyu Lu

cs.CVOct 2, 2026

Seeing, Saying, but Not Using: From Reportable Spatial Facts to Usable States in Multimodal Large Language Models

Authors: Jinchang Zhang, Guoyu Lu

Organizations: Intelligent Vision and Sensing (IVS) Lab Indiana University Bloomington Bloomington, IN, USA

Abstract

A multimodal large language model that correctly reports a spatial fact does not necessarily use that fact in subsequent reasoning. To study this distinction, we introduce SpaceConflict, a benchmark of 23{,}196 inputs for the construction and use of spatial state. Under a unified Supported/Contradictory/Unknown judgment interface, it covers local fact binding (L1), relational composition (L2), cross-observation consistency (L3), and state judgment under transformation (L4). Posing a direct-state query, a full-transformation query, and an explicit-initial-state query on the same world reveals an availability--utilization gap: models recover the initial state from visual evidence yet fail when that state must drive a transformation. For Qwen3.5-9B, 50 of 100 sequences with a correctly recovered initial state fail the full transformation, and supplying the state explicitly repairs all 50; the gap narrows with scale but does not close. We therefore propose Operational State Supervision (OSS), which supervises task-relevant spatial states and their transformation trajectories and aligns shared facts across contexts. OSS improves paired accuracy on matched judgments most on L3 and L4, the levels that depend on organizing and using state. Evaluating multimodal spatial reasoning thus requires asking not only whether a model can see and state a spatial fact, but whether that fact becomes a usable state in subsequent computation.

Figures & tables

Figure 1: Overview of SpaceConflict in a shared visual world. Multiple observations of the same underlying office scene instantiate four distinct spatial reasoning requirements.

Level	Pairs	Unknown	Inputs
L1	7,559	1,480	16,598
L2	1,212	200	2,624
L3	1,041	520	2,602
L4	1,136	300	2,572
Total	10,948	2,500	24,396

Table 1: Composition of the evaluation release by reasoning level.

Figure 2: From multi-view evidence to an operational world state. Multiple views of the same world provide visual evidence for constructing the initial spatial state $S_{0}$ in a room-centric reference frame. The state is then transformed without moving any object: $A_{1}$ re-centers the reference frame on the cabinet, and $A_{2}$ aligns the forward axis with the cabinet-to-TV direction, producing the cabinet-centered, TV-aligned state $S_{2}$ . The table is then queried in this transformed reference frame.

Model	L1	L2	L3	L4	Overall
Model	Claim/Pair Acc	Claim/Pair Acc	Claim/Pair Acc	Claim/Pair Acc	Claim/Pair Acc
Qwen2.5-VL-7B	23.38/0.85	21.86/0.17	30.75/0.00	26.59/5.81	24.36/1.21
Qwen2.5-VL-32B	54.43/32.85	51.98/43.15	52.50/24.88	56.18/28.61	54.16/32.79
Qwen3.5-4B	52.60/34.53	83.66/71.20	47.08/20.56	59.18/28.61	55.82/36.65
Qwen3.5-9B	55.45/39.08	89.81/84.57	53.65/39.58	60.73/30.81	59.26/43.30
Qwen3.5-27B	58.62/46.41	91.71/87.71	57.84/49.47	65.05/44.10	62.53/51.03

Table 3: Performance comparison across L1–L4 and overall evaluation.

Condition	4B	9B	27B
Direct state ( $s_{0}$ from vision)	65.00	62.50	60.00
Full visual transition	27.50	35.00	58.75
Explicit $s_{0}$ + visual context	64.38	100.00	100.00
Explicit $s_{1}$ + visual context	76.88	65.00	86.25
Symbolic full program	87.50	100.00	100.00
Symbolic second-action suffix	100.00	100.00	100.00

Table 4: Behavioral and program controls on the common count panel. Values are accuracy (%) over 160 sequences from 80 worlds.

Qwen3.5	Direct	Full	Explicit	Sham	$\Delta$ (pp)	95% CI
4B	35/47	12/47	9/47	10/47	-2.13	[-19.15, 14.89]
9B	42/47	11/47	37/47	9/47	59.57	[40.43, 76.60]
27B	41/47	4/47	47/47	14/47	70.21	[57.45, 82.98]

Table 5: Non-count coordinate-transformation results. Accuracy cells are correct worlds/47.

Model / Method	L1	L2	L3	L4	Overall
Base (Qwen3.5-9B)	53.23 / 38.91	92.64 / 87.88	48.57 / 39.91	62.60 / 31.56	56.53 / 44.49
Answer-SFT Natural	42.98 / 58.33	70.02 / 59.96	11.43 / 8.77	74.32 / 50.67	42.60 / 51.19
Answer-SFT Balanced	50.74 / 78.20	95.24 / 92.43	38.68 / 69.96	62.31 / 38.67	53.37 / 73.97
CoT-SFT Partial	20.50 / 13.85	37.23 / 12.99	24.54 / 31.80	23.94 / 7.56	22.90 / 15.25
OSS-L4	50.74 / 77.88	92.86 / 87.88	44.22 / 92.11	73.16 / 56.23	55.14 / 78.26
OSS (Ours)	53.53 / 86.94	96.10 / 94.81	45.29 / 96.05	82.36 / 68.44	58.26 / 86.79

Table 6: SFT results on SpaceConflict (test).

Appendix figures & tables37 assets

Supplementary material from the paper’s appendix.

Appendix

Source	Grouping unit	Construction role
SPAR ( Zhang et al., 2025 )	Image or source scene	Object relations and source-annotated multiview relations.
CA-VQA / CA-1M ( Daxberger et al., 2025a ; Lazarow et al., 2024 )	Capture	Reference/support frames, instance annotations, and scoped counts or relations.
VSI-Bench ( Yang et al., 2024 )	Parent video	Video-scoped spatial facts, counts, and appearance or observation order.
STI-Bench ( Li et al., 2025c )	Video and interval	Temporally scoped qualitative facts; continuous-valued regression targets are outside this release.
OmniSpatial ( Jia et al., 2025 )	Item or clip	Source-grounded spatial question–answer records. Unselected answer options are not treated as refutations.
Hypo3D ( Mao et al., 2025 )	Base scene and change	Native transformation descriptions and source post-state question–answer records.

Appendix

Table 7: Source roles in construction. World units follow the dataset-specific contract; supplementary grounding resources contribute annotations for existing scenes rather than a separate Controlled media source.

Operation	Edited dependency	Pairs
SUBJECT_OBJECT_SWAP	Subject/object assignment	5,284
COUNT_CLASS_REBIND	Category associated with a count	2,577
RELATION_GRAPH_UNSAT	Composed relation consistency	1,212
VISIBLE_CLASS_REBIND	Category associated with visibility	498
APPEARANCE_ORDER_ERROR	Temporal appearance order	195
SUBJECT_OBJECT_REBIND	Entities in a scoped relation	46

Appendix

Table 8: Realized L1–L3 semantic edits in the original release. Each row counts retained pairs, not candidate generations or individual claims.

Group	Recorded edit	Pairs
SPAR multiview relations	Subject/object swap	800
Appearance-order relations	Appearance-order error	195
Interval-scoped object relations	Subject/object rebinding	46
Total		1,041

Appendix

Table 9: Released L3 construction groups. The SPAR support-view audit and its 1,000-candidate denominator are reported separately in Table 14 .

Origin	Subtype	Core	Calibration	Total
Native	Direct	183	18	201
Native	Aggregated	214	1	215
Controlled		630	90	720
Total		1,027	109	1,136

Appendix

Table 10: L4 paired composition. Core and Calibration are construction-time dependency categories. Counts are pairs; the additional 300 Unknown inputs are reported in Table 1 .

Level	Evidence and context	Supported claim	Contradictory claim
L1	Window chair is right of the table; door chair is left of it. Left/right are exclusive in the named frame.	The window chair is right of the table.	The door chair is right of the table.
L2	$A$ is left of $B$ ; $B$ is left of $C$ . Strict transitivity is authorized.	$A$ is left of $C$ .	$C$ is left of $A$ .
L3	First appearance: chair at frame 2, table at frame 7, in the same ordered sequence.	The chair appears before the table.	The table appears before the chair.
L4	Initially three chairs; the stated transformation removes one identified chair.	After the change, two chairs remain.	After the change, three chairs remain.

Appendix

Table 11: Schematic construction examples. Relation rules and scopes are explicit in the evidence column. The examples illustrate the definitions; release counts refer to the actual retained data, not these teaching examples.

Artifact group	Information recorded
claim_graphs/	Canonical supported and contradictory propositions.
evidence_subgraphs/	Supporting/refuting facts and media references.
construction_traces/	Source references and hashes, operator, edit, and preserved fields.
certificates/	Proof nodes/edges, required facts, and applicable minimality information.
Verifier outputs and release/gold	Check outcomes, retained membership, and evaluation targets.

Appendix

Table 12: Organization of the linked L1–L3 provenance and proof bundle. Fields are resolved across associated artifacts; this table is not a list of mandatory keys in every certificate JSON.

Path	Main validation object	Scope of the recorded evidence
L1	Local support/refutation under authorized rules	Claim-level atomicity; the pair can use distinct deciding facts.
L2	Two-premise relation proof and premise-removal checks	Dependency of the annotated conflict justification.
L3 SPAR	Source-annotated support views and object references	Annotation-level support-view audit; local pixel validation is separately marked.
L4 Native	Source change and official post-state answer	Agreement with the source post-state record.
L4 Controlled	Source pre-state, DSL action, and computed post-facts	Agreement between two separately implemented transition procedures.
L4 dependency category	Core/Calibration and associated dependency fields	Construction-time categorization, with independent ablations tracked separately.

Appendix

Table 13: Validation objects and their scope. Checks are applied according to source, level, and label type. Construction records distinguish symbolic or annotation-level checks from visual observability and model-response tests.

Audit	Recorded counts	Interpretation
SPAR L2 candidate verification	2,000 candidates; 1,311 accepted; 689 rejected	65.55% candidate acceptance. The full retained L2 count is 1,212 pairs (Table 1 ).
SPAR L3 support-view audit	1,000 pairs checked; 1,000 pair passes; 1,273 ablation cases	Source-annotation support-view checks. The final multiview quota retains 800 pairs (Table 9 ).

Appendix

Table 14: Available construction-time audit records. Candidate acceptance, annotation-level checks, and final quota-selected membership have distinct denominators.

Control	Status	Recorded result or scope
Text-only label prediction	Executed	Accuracy 0.5; AUC 0.5 for the recorded diagnostic.
Equivalent inversion	Executed	13,474 cases; 0 failures.
Paired media-only input	Static input audit	Exact paired-media equality passed.
First/last frame, frame shuffle, media shuffle, wrong view	Manifests prepared	No model scores in this construction audit.

Appendix

Table 15: Recorded shortcut-control scope for the L1–L3 v10 construction audit. Static input checks and unscored manifests are not model performance measurements.

Unit	Train	Development	Test	Total
Registered source-world IDs	2,682	627	1,175	4,484
Binary-pair media groups	4,589	1,084	1,446	7,119
Transformation branches	634	173	225	1,032
Supported/Contradictory pairs	7,470	1,707	1,771	10,948
Supported/Contradictory inputs	14,940	3,414	3,542	21,896
Unknown inputs	282	152	2,066	2,500

Appendix

Table 16: Historical composite release statistics. The world row uses registered source-world IDs; it is not the cross-source world grouping used in the experimental summaries.

Split	Inputs	World groups in this audit
Train	13,476	2,221
Development	3,267	571
Test	5,608	1,124

Appendix

Table 17: Frozen training protocol after the cross-source identity audit. Test inputs are unchanged from the historical release; training and development membership follow the later audited protocol.

Qwen3.5	Metric	Numerator/denominator	Worlds	Estimate	95% CI
4B	ClaimAcc	13505/24396	4284	55.36	[55.09, 56.56]
4B	BinaryClaimAcc	12315/21896	4238	56.24	[55.50, 56.97]
4B	PairAcc	4012/10948	4238	36.65	[35.60, 37.66]
4B	C recall	4924/10948	4238	44.98	[43.86, 46.12]
4B	U recall	1190/2500	858	47.60	[48.76, 54.55]
4B	World macro	2153.7463005609006/4284	4284	50.27	[49.50, 51.04]

Appendix

Table 20: Available overall Qwen3.5 intervals. Percent estimates and intervals use their original estimands. The world-macro numerator is not a count of entirely correct worlds.

Qwen3.5	Level	Claim correct/N	Claim [CI]	Pairs correct/N	Pair [CI]
4B	L1	8730/16598	52.60 [51.71, 53.45]	2610/7559	34.53 [33.45, 35.64]
4B	L2	2028/2624	83.66 [80.88, 86.40]	863/1212	71.20 [66.56, 75.92]
4B	L3	1225/2602	47.08 [45.07, 49.04]	214/1041	20.56 [17.82, 23.44]
4B	L4	1522/2572	59.18 [57.70, 60.61]	325/1136	28.61 [25.94, 31.36]
9B	L1	9204/16598	55.45 [54.57, 56.30]	2954/7559	39.08 [38.01, 40.16]
9B	L2	2177/2624	89.81 [87.56, 91.97]	1025/1212	84.57 [81.49, 87.59]

Appendix

Table 22: Level-specific counts and 95% intervals for the three diagnostic-family checkpoints. Estimates and intervals are in percent.

ID	Actual question or intervention
B00	Historical full-task request (reused output)
B01	Canonical cumulative two-action visual task; query $s_{2}$
B02	Independent visual initial-fact query; query $s_{0}$
B03	Both actions retained; query $s_{1}$
B04	Ask which named state is the input to action 2
B05	B01 plus the correct initial count before both actions

Appendix

Table 24: Condition registry for the count diagnostics. The sample panel has 160 sequence-level entries per standard condition and an 80-world shared initial-protection supplement. B07 is not a matched control for B05.

Condition	Correct/N	Accuracy	95% CI	Worlds
B00	47/160	29.38	[21.88, 37.50]	80
B01	44/160	27.50	[21.88, 33.12]	80
B02	104/160	65.00	[53.75, 75.00]	80
B03	67/160	41.88	[32.50, 51.25]	80
B04	25/160	15.62	[10.62, 21.25]	80
B05	103/160	64.38	[56.25, 72.50]	80

Appendix

Table 25: Complete Qwen3.5-4B behavioral bridge. Accuracy and 95% world-cluster intervals are in percent. Counts follow the original condition-level scoring, including null/invalid decisions.

Condition	Correct/N	Accuracy	95% CI	Worlds
B00	53/160	33.12	[26.25, 40.62]	80
B01	56/160	35.00	[26.88, 43.12]	80
B02	100/160	62.50	[51.25, 72.50]	80
B03	46/160	28.75	[20.62, 36.88]	80
B04	160/160	100.00	[100.00, 100.00]	80
B05	160/160	100.00	[100.00, 100.00]	80

Appendix

Table 26: Complete Qwen3.5-9B behavioral bridge. Accuracy and 95% world-cluster intervals are in percent. Counts follow the original condition-level scoring, including null/invalid decisions.

Condition	Correct/N	Accuracy	95% CI	Worlds
B00	97/160	60.62	[51.25, 70.00]	80
B01	94/160	58.75	[48.75, 68.75]	80
B02	96/160	60.00	[50.00, 70.00]	80
B03	111/160	69.38	[59.98, 78.77]	80
B04	160/160	100.00	[100.00, 100.00]	80
B05	160/160	100.00	[100.00, 100.00]	80

Appendix

Table 27: Complete Qwen3.5-27B behavioral bridge. Accuracy and 95% world-cluster intervals are in percent. Counts follow the original condition-level scoring, including null/invalid decisions.

Qwen3.5	Contrast	Rescue/errors	Harm/successes	Net $\Delta$	95% CI
4B	B05–B01	74/116	15/44	36.88	[25.62, 47.50]
4B	B06–B01	86/116	7/44	49.38	[42.50, 56.25]
4B	B07–B01	5/116	7/44	-1.25	[-5.62, 3.12]
4B	B08–B01	96/116	0/44	60.00	[53.75, 66.25]
4B	B09–B01	116/116	0/44	72.50	[66.88, 78.12]
9B	B05–B01	104/104	0/56	65.00	[56.88, 73.12]

Appendix

Table 28: Paired changes relative to B01. Every row has 160 matched sequences over 80 worlds. Rescue is divided by baseline errors; harm is divided by baseline successes. Net differences and intervals are percentage points.

Qwen3.5	Conditional quantity	Events/eligible	Worlds	Rate	95% CI
4B	$\Pr(T=0\mid F=1)$	64/104	52	61.54	[55.77, 68.27]
4B	$\Pr(O=1\mid F=1,T=0)$	35/64	51	54.69	[42.19, 66.67]
4B	$\Pr(\mathrm{B06}=1\mid\mathrm{B03}=1,T=0)$	35/44	34	79.55	[65.85, 91.11]
9B	$\Pr(T=0\mid F=1)$	50/100	50	50.00	[40.00, 60.00]
9B	$\Pr(O=1\mid F=1,T=0)$	50/50	37	100.00	[100.00, 100.00]
9B	$\Pr(\mathrm{B06}=1\mid\mathrm{B03}=1,T=0)$	14/23	23	60.87	[39.13, 78.26]

Appendix

Table 31: Complete archived conditional diagnostics. Percent estimates and intervals are preserved from the conditional-statistics artifact. The last condition concerns an independently queried intermediate state, not an internal trajectory.

Condition	Signature	4B	9B	27B
B01	$s_{0}$	31/116; W=31	17/104; W=17	1/66; W=1
B01	$s_{1}$	35/116; W=31	17/104; W=17	19/66; W=19
B01	Action-1 amount	30/116; W=27	17/104; W=16	11/66; W=11
B01	Action-2 amount	21/116; W=20	33/104; W=30	22/66; W=15
B01	$\tau_{2}(s_{0})$	10/116; W=10	5/104; W=5	10/66; W=6
B01	$\tau_{1}(s_{1})$	17/116; W=16	21/104; W=21	9/66; W=9

Appendix

Table 34: Non-exclusive numerical matches among errors. Each cell gives matches/errors and the number of affected worlds. Multiple signatures can match the same output and the rows must not be summed.

Qwen3.5	Target	Wording	Correct/N	Accuracy	95% CI
4B	S0	T1	15/96	15.62	[12.50, 18.75]
4B	S0	T2	96/96	100.00	[100.00, 100.00]
4B	S0	T3	96/96	100.00	[100.00, 100.00]
4B	S0	T4	96/96	100.00	[100.00, 100.00]
4B	S1	T1	37/96	38.54	[31.25, 44.79]
4B	S1	T2	96/96	100.00	[100.00, 100.00]

Appendix

Table 35: Complete target-by-wording results. Every row contains 96 programs and 16 program-family clusters; accuracy and confidence intervals are percentages. Extreme empirical intervals are not population guarantees.

Qwen3.5	Target	Contrast	$\Delta$ (pp)	95% CI
4B	S0	T2–T1	84.38	[81.25, 87.50]
4B	S0	T3–T1	84.38	[81.25, 87.50]
4B	S0	T4–T1	84.38	[81.25, 87.50]
4B	S1	T2–T1	61.46	[55.21, 68.75]
4B	S1	T3–T1	61.46	[55.21, 68.75]
4B	S1	T4–T1	61.46	[55.21, 68.75]

Appendix

Table 36: Paired wording contrasts. Every comparison has 96 program pairs and 16 families. Differences and intervals are percentage points; all targets are retained.

Qwen3.5	Condition	Terminal match	Correct	Wrong state	Other	Null	Invalid
4B	T1_S0	43/96	15/96	75/96	6/96	0/96	0/96
4B	T1_S1	48/96	37/96	48/96	11/96	0/96	0/96
4B	T1_S2	68/96	68/96	24/96	4/96	0/96	0/96
4B	T2_S0	33/96	96/96	0/96	0/96	0/96	0/96
4B	T2_S1	0/96	96/96	0/96	0/96	0/96	0/96
4B	T2_S2	69/96	69/96	11/96	16/96	0/96	0/96

Appendix

Table 37: Output distributions for all target/wording conditions. All fractions have denominator 96. Terminal matching is non-exclusive; the remaining response categories are mutually exclusive. Each row has 16 program families.

Qwen3.5	Condition	Correct/worlds	Accuracy	95% CI
4B	Direct $s_{0}$	35/47	74.47	[61.70, 87.23]
4B	Full transformation	12/47	25.53	[12.77, 38.30]
4B	Explicit $s_{0}$	9/47	19.15	[8.51, 31.91]
4B	Matched sham	10/47	21.28	[10.64, 34.04]
9B	Direct $s_{0}$	42/47	89.36	[78.72, 97.87]
9B	Full transformation	11/47	23.40	[12.77, 36.17]

Appendix

Table 38: Complete non-count condition scores. Each row contains 47 worlds. Accuracy and confidence intervals are percentages.

Qwen3.5	Family	Condition	Correct/worlds	Accuracy	95% CI
4B	Horizontal	Direct	30/34	88.24	[76.47, 97.06]
4B	Horizontal	Full	5/34	14.71	[2.94, 26.47]
4B	Horizontal	Explicit	8/34	23.53	[8.82, 38.24]
4B	Horizontal	Sham	4/34	11.76	[2.94, 23.53]
4B	Vertical	Direct	5/6	83.33	[50.00, 100.00]
4B	Vertical	Full	1/6	16.67	[0.00, 50.00]

Appendix

Table 39: Non-count family-by-condition results. The family analysis is post-hoc descriptive. Estimates and intervals are percentages; each row uses the family-specific world count in its denominator.

Qwen3.5	Reader	Block	Correct/N	Accuracy	95% CI
4B	Majority	—	10/32	31.25	[12.50, 56.25]
4B	Surface	—	22/32	68.75	[46.88, 87.50]
4B	Residual	0	12/32	37.50	[12.50, 62.50]
4B	Residual	4	18/32	56.25	[31.25, 81.25]
4B	Residual	9	22/32	68.75	[43.75, 87.50]
4B	Residual	13	20/32	62.50	[37.50, 87.50]

Appendix

Table 42: Prespecified B01/S1/checkpoint probing slice. Every evaluated row uses 32 sequences from 16 worlds. Accuracy and intervals are percentages. Surface denotes the combined word/number-bag-plus-length baseline.

Block	Anchor	Info $\Delta$ LP	Sham $\Delta$ LP	Specific $\Delta$ LP	95% CI
0	A1 end	-0.001610	-0.004306	0.002696	[-0.0290, 0.0312]
0	A2 end	-0.001107	0.015729	-0.016835	[-0.0394, 0.0051]
0	Context end	-0.007092	-0.012296	0.005204	[-0.0211, 0.0328]
0	Query	-0.012741	0.010724	-0.023465	[-0.0569, 0.0078]
5	A1 end	0.000041	-0.013612	0.013653	[-0.0121, 0.0376]
5	A2 end	-0.009554	-0.007460	-0.002094	[-0.0347, 0.0301]

Appendix

Table 49: All 32 I1 localization windows. LP differences are nats, world-averaged over the same 25 Type A sequences/20 worlds. No multiplicity or selection adjustment is applied to these exploratory intervals.

Block	Anchor	Info rescue	Success-donor match	Info null	Info invalid
0	A1 end	1/25	10/34	1/25	0/25
0	A2 end	1/25	10/34	0/25	0/25
0	Context end	1/25	11/34	1/25	0/25
0	Query	0/25	10/34	0/25	0/25
5	A1 end	1/25	10/34	1/25	0/25
5	A2 end	1/25	10/34	1/25	0/25

Appendix

Table 50: Generation endpoints for all localization windows. Informative rescue/null/invalid use 25 sequences/20 worlds; success-donor absolute matching uses 34 sequences/27 worlds. Absolute matches need not be induced by patching.

Variable	Reader/block	Checkpoint: count [CI]	Sel.	Query: count [CI]	Sel.	$U$
$Z$	Constant training label
$s_{0}$	Majority	22/32 [43.75, 87.50]	62.50	—	—	2
$s_{0}$	Surface	21/32 [43.75, 87.50]	40.62	—	—	2
$s_{0}$	0	16/32 [25.00, 75.00]	37.50	19/32 [37.50, 81.25]	43.75	2
$s_{0}$	4	18/32 [31.25, 81.25]	37.50	15/32 [25.00, 68.75]	43.75	2
$s_{0}$	9	22/32 [43.75, 87.50]	43.75	19/32 [37.50, 81.25]	43.75	2

Appendix

Table 58: Qwen3.5-4B B1/B01 complete fixed-layer ledger. Residual rows are indexed by actual block; baseline rows have no block. Correct/N and confidence intervals are retained for each available anchor; selection values are descriptive. Evaluation: 32 sequences/16 worlds.

Variable	Reader/block	Checkpoint: count [CI]	Sel.	Query: count [CI]	Sel.	$U$
$Z$	Constant training label
$s_{0}$	Majority	22/32 [43.75, 87.50]	62.50	—	—	2
$s_{0}$	Surface	21/32 [43.75, 87.50]	71.88	—	—	2
$s_{0}$	0	26/32 [62.50, 100.00]	75.00	19/32 [34.38, 81.25]	43.75	2
$s_{0}$	4	30/32 [81.25, 100.00]	93.75	15/32 [25.00, 68.75]	40.62	2
$s_{0}$	9	30/32 [81.25, 100.00]	93.75	24/32 [56.25, 93.75]	56.25	2

Appendix

Table 59: Qwen3.5-4B B1/B06 complete fixed-layer ledger. Residual rows are indexed by actual block; baseline rows have no block. Correct/N and confidence intervals are retained for each available anchor; selection values are descriptive. Evaluation: 32 sequences/16 worlds.

Variable	Reader/block	Checkpoint: count [CI]	Sel.	Query: count [CI]	Sel.	$U$
$Z$	Constant training label
Entity	Label not supplied
Protected	Label not supplied
$s_{0}$	Majority	0/18 [0.00, 0.00]	0.00	—	—	2
$s_{0}$	Surface	11/18 [16.67, 83.33]	16.67	—	—	2
$s_{0}$	0	4/18 [0.00, 50.00]	16.67	2/18 [0.00, 16.67]	16.67	2

Appendix

Table 60: Qwen3.5-4B B2/FINAL complete fixed-layer ledger. Residual rows are indexed by actual block; baseline rows have no block. Correct/N and confidence intervals are retained for each available anchor; selection values are descriptive. Evaluation: 18 sequences/3 program families.

Variable	Reader/block	Checkpoint: count [CI]	Sel.	Query: count [CI]	Sel.	$U$
$Z$	Constant training label
$s_{0}$	Majority	22/32 [43.75, 87.50]	62.50	—	—	2
$s_{0}$	Surface	21/32 [43.75, 87.50]	40.62	—	—	2
$s_{0}$	0	16/32 [25.00, 75.00]	43.75	20/32 [37.50, 87.50]	37.50	2
$s_{0}$	4	16/32 [25.00, 75.00]	37.50	19/32 [34.38, 81.25]	43.75	2
$s_{0}$	9	18/32 [31.25, 81.25]	43.75	19/32 [37.50, 81.25]	37.50	2

Appendix

Table 61: Qwen3.5-9B B1/B01 complete fixed-layer ledger. Residual rows are indexed by actual block; baseline rows have no block. Correct/N and confidence intervals are retained for each available anchor; selection values are descriptive. Evaluation: 32 sequences/16 worlds.

Variable	Reader/block	Checkpoint: count [CI]	Sel.	Query: count [CI]	Sel.	$U$
$Z$	Constant training label
$s_{0}$	Majority	22/32 [43.75, 87.50]	62.50	—	—	2
$s_{0}$	Surface	21/32 [43.75, 87.50]	71.88	—	—	2
$s_{0}$	0	26/32 [62.50, 100.00]	87.50	20/32 [37.50, 87.50]	37.50	2
$s_{0}$	4	30/32 [81.25, 100.00]	93.75	21/32 [43.75, 87.50]	50.00	2
$s_{0}$	9	30/32 [81.25, 100.00]	93.75	20/32 [37.50, 87.50]	43.75	2

Appendix

Table 62: Qwen3.5-9B B1/B06 complete fixed-layer ledger. Residual rows are indexed by actual block; baseline rows have no block. Correct/N and confidence intervals are retained for each available anchor; selection values are descriptive. Evaluation: 32 sequences/16 worlds.

Variable	Reader/block	Checkpoint: count [CI]	Sel.	Query: count [CI]	Sel.	$U$
$Z$	Constant training label
Entity	Label not supplied
Protected	Label not supplied
$s_{0}$	Majority	0/18 [0.00, 0.00]	0.00	—	—	2
$s_{0}$	Surface	11/18 [16.67, 83.33]	16.67	—	—	2
$s_{0}$	0	6/18 [0.00, 66.67]	16.67	4/18 [0.00, 50.00]	16.67	2

Appendix

Table 63: Qwen3.5-9B B2/FINAL complete fixed-layer ledger. Residual rows are indexed by actual block; baseline rows have no block. Correct/N and confidence intervals are retained for each available anchor; selection values are descriptive. Evaluation: 18 sequences/3 program families.

Variable	Reader/block	Checkpoint: count [CI]	Sel.	Query: count [CI]	Sel.	$U$
$Z$	Constant training label
$s_{0}$	Majority	22/32 [43.75, 87.50]	62.50	—	—	2
$s_{0}$	Surface	21/32 [43.75, 87.50]	40.62	—	—	2
$s_{0}$	0	16/32 [25.00, 75.00]	25.00	20/32 [37.50, 87.50]	34.38	2
$s_{0}$	9	18/32 [31.25, 81.25]	43.75	20/32 [37.50, 87.50]	37.50	2
$s_{0}$	18	22/32 [43.75, 87.50]	37.50	22/32 [43.75, 87.50]	31.25	2

Appendix

Table 64: Qwen3.5-27B B1/B01 complete fixed-layer ledger. Residual rows are indexed by actual block; baseline rows have no block. Correct/N and confidence intervals are retained for each available anchor; selection values are descriptive. Evaluation: 32 sequences/16 worlds.

Variable	Reader/block	Checkpoint: count [CI]	Sel.	Query: count [CI]	Sel.	$U$
$Z$	Constant training label
$s_{0}$	Majority	22/32 [43.75, 87.50]	62.50	—	—	2
$s_{0}$	Surface	21/32 [43.75, 87.50]	71.88	—	—	2
$s_{0}$	0	26/32 [62.50, 100.00]	68.75	18/32 [31.25, 81.25]	34.38	2
$s_{0}$	9	30/32 [81.25, 100.00]	93.75	20/32 [37.50, 87.50]	46.88	2
$s_{0}$	18	30/32 [81.25, 100.00]	93.75	22/32 [43.75, 87.50]	65.62	2

Appendix

Table 65: Qwen3.5-27B B1/B06 complete fixed-layer ledger. Residual rows are indexed by actual block; baseline rows have no block. Correct/N and confidence intervals are retained for each available anchor; selection values are descriptive. Evaluation: 32 sequences/16 worlds.

Variable	Reader/block	Checkpoint: count [CI]	Sel.	Query: count [CI]	Sel.	$U$
$Z$	Constant training label
Entity	Label not supplied
Protected	Label not supplied
$s_{0}$	Majority	0/18 [0.00, 0.00]	0.00	—	—	2
$s_{0}$	Surface	11/18 [16.67, 83.33]	16.67	—	—	2
$s_{0}$	0	14/18 [66.67, 83.33]	16.67	6/18 [0.00, 83.33]	16.67	2

Appendix

Table 66: Qwen3.5-27B B2/FINAL complete fixed-layer ledger. Residual rows are indexed by actual block; baseline rows have no block. Correct/N and confidence intervals are retained for each available anchor; selection values are descriptive. Evaluation: 18 sequences/3 program families.

Multimodal Spatial Reasoning

See all 44 papers

Aug 5, 2026

Trace, Verify, and Correct: A Training-Free Framework for Spatial Reasoning in Multimodal LLMs

Although Multimodal Large Language Models (MLLMs) have made substantial progress, their spatial reasoning may still produce intermediate judgments inconsistent with the input image, allowing errors to propagate through the reasoning chain and affect the final answer. Existing methods mainly improve spatial reasoning through training or additional spatial information, without considering whether the reasoning process itself is faithful to the model input. Our study shows that unfaithful reasoning chains significantly reduce final-answer accuracy. To address this issue, we propose a modular and training-free framework for spatial reasoning verification and correction. The framework constructs a Spatial Evidence Graph (SEG), which associates atomic spatial evidence extracted from Chain-of-Thought reasoning with visual entities, spatial relations, source steps, and visual evidence. Spatial Evidence Reliability Assessment (SERA) evaluates the reliability of visual evidence based on object existence, localization, and geometric measurements. The framework then identifies the earliest spatial evidence unit contradicted by reliable visual evidence and guides the original MLLM to revise the subsequent reasoning and final answer. Across 15 model-dataset settings, our method achieves an average accuracy of 68.94%, outperforming the compared baselines by 8.55 percentage points on average. Our code will be open-sourced.

Yang Yang, Jiawei Chen, Tairan Chen +1

Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University · Zhongguancun Academy · Stevens Institute of Technology

Multimodal Grounding Visual Spatial Reasoning Multimodal Large Language Models Multimodal CoT Reasoning Spatial Reasoning in Language Models Multimodal Spatial Reasoning

Ask PDF

Oct 9, 2026

S-Space: Exploring Spatial Workspace in Multimodal Language Models

Multimodal language models (MLLMs) exhibit promising spatial reasoning capabilities, yet the internal mechanisms supporting them remain poorly understood. We uncover S-Space, a linear representation subspace in which object-token representations encode continuous coordinates along horizontal, vertical, and distance axes. Although we identify these axes by differentiating spatial-answer scores with respect to object-token activations, S-Space is intrinsic rather than induced by specific data or prompts, remaining readable under non-spatial questions. We show that S-Space functions as an internal spatial workspace of MLLMs. Perception encodes object locations into it, forming spatial representations that can be directly read out to support accurate spatial judgments. Internal reasoning transforms it for changing viewpoints. Verbal responses selectively access it, evidenced by causal interventions that alter spatial judgments while largely preserving non-spatial responses. Case studies further show that S-Space assigns locations to unseen entities and the observer, and connects physical directions with abstract linguistic meanings, suggesting both the capabilities and limitations of large-scale language training. Finally, we use S-Space as a diagnostic tool for spatial reasoning failures. Direction-word readouts reveal systematic coupling between cardinal and image-relative reference frames. More importantly, a perception-computation decomposition exposes a key bottleneck: MLLMs can form capable spatial representations, yet fall short in the spatial transformations required to reason reliably across viewpoint changes.

Zhiyuan Shi, Xiang Li, Xingzhuo Guo +8

Spatial Reasoning in Language Models Spatial Reasoning Multimodal Spatial Reasoning Multimodal Large Language Models Multimodal Reasoning Multimodal Model Interpretability+1

Ask PDF

Jun 30, 2026

Spatial Reasoning via Modality Switching Between Language and Symbolic Representations

Human reasoning is inherently multimodal: when problems become difficult, we rarely think in words alone. We often externalize our reasoning by sketching diagrams or drawing grids to understand the underlying conceptual structure and avoid mistakes. Building on this premise, our research investigates: (a) whether grounding multi-hop textual-spatial stories into geometry-aware modalities, such as layouts or grids, improves reasoning compared to natural language-based inference; and (b) whether a model can decide when to rely on natural language reasoning and when to switch to a structured modality. We address these questions by introducing a switching metric based on trustworthiness and complexity signals, which estimates when grounding a spatial story into structure is likely to improve performance. This takes a first step toward principled modality selection in Large Language Model (LLM) reasoning. Across our settings, switching from natural language-based reasoning to a grid-based representation improves LLM performance by up to 42%, highlighting the importance of modality choice in shaping reasoning outcomes.

Shreya Rajpal, Tanawan Premsri, Parisa Kordjamshidi

Department of Computer Science and Engineering Michigan State University Michigan, USA

Multimodal Reasoning Spatial Reasoning LLM Reasoning Multimodal Grounding Structured Reasoning Spatial Reasoning in Language Models+1

Ask PDF

Abstract

Multimodal Spatial Reasoning

Explore similar work

Trace, Verify, and Correct: A Training-Free Framework for Spatial Reasoning in Multimodal LLMs

S-Space: Exploring Spatial Workspace in Multimodal Language Models

Spatial Reasoning via Modality Switching Between Language and Symbolic Representations