Rotated, but How Far? Diagnosing and Improving Object-Rotation Reasoning in VLMs

Zhaochen Wang; Yujun Cai; Huangbo Zou; Hower Yang; Naipeng Dong; Miao Xu; Haibin Ling

cs.CVOct 5, 2026

Rotated, but How Far? Diagnosing and Improving Object-Rotation Reasoning in VLMs

Authors: Zhaochen Wang, Yujun Cai, Huangbo Zou, Hower Yang, Naipeng Dong, Miao Xu, Haibin Ling

Organizations: The University of Queensland · Westlake University

Abstract

Vision-language models (VLMs) can detect that an object has rotated across views, but cannot reliably tell by how much. We introduce OR-Bench, a fine-grained benchmark for object-rotation reasoning with eight tasks covering rotation detection, rotation magnitude estimation, and multi-view rotation reasoning. Across 12 VLMs, the gap is stark: the strongest models approach 100% accuracy on detection, yet even coarse magnitude estimation is near chance. When asked for exact angles, models place 91.8--100% of their predictions on just $0^\circ$ , $90^\circ$ , and $180^\circ$ , a failure we term canonical-angle collapse. This collapse persists even without visual input. Representation probing shows that missing information is only part of the explanation. Although rotation information becomes less recoverable at finer granularity, substantial coarse-grained information remains, and a simple linear probe outperforms the models' generated answers. This suggests that VLMs underuse rotation information they already encode. We therefore propose RotationCue, a lightweight decoder that recovers coarse rotation information from the VLM's own frozen representations and feeds it back to the model as intermediate textual context. Across three VLMs, RotationCue improves every model--task combination on OR-Bench, raising macro-average accuracy by 7.9--12.6 points while preserving general capabilities.

Figures & tables

Figure 1: Object-rotation reasoning in OR-Bench. Left: two views of the same object under azimuth rotation. Right: VLMs detect rotation reliably, but performance drops sharply once rotation magnitude must be estimated, even coarsely, leaving a large gap to humans.

Figure 2: Overview of OR-Bench. Eight tasks span two-view rotation understanding (orange) and multi-view rotation use (purple), covering multiple levels of answer granularity, from detecting whether a rotation occurred to specifying its exact angle.

Figure 3: Canonical-angle collapse across input and response settings. Left: share of predictions in the canonical set

\{0^{\circ},90^{\circ},180^{\circ}\}

; darker cells indicate stronger collapse. Right: Qwen3-VL-8B’s Exact Angle (MCQ) predictions concentrate primarily at

90^{\circ}

Figure 4: Linear probe accuracy at L8, L16, and the final VE layer, with the model’s own answer as a reference. Recoverability declines from Rotation Detection to Coarse Angle to Exact Angle.

Figure 5: Probe vs. LM head at

h_{\mathrm{pre}}

. Probe (filled) and LM head (open) are two linear heads reading the same pre-answer representation. The gap is largest on Coarse Angle.

Figure 6: RotationCue. The relation representation is read from vision layer L8 and passed to a lightweight decoder, without modifying the frozen VLM. Three learnable rotation queries predict a three-field rotation context, which is reinjected into the VLM as text only.

Level	Task	Qwen3-VL-8B	Qwen2.5-VL-7B	InternVL3.5-8B	Mean $\Delta$
Two-view	Rotation Detection	100.0 (+0.4)	99.0 (+13.0)	100.0 (+0.8)	+4.7
	Coarse Angle	66.7 (+26.0)	63.4 (+21.8)	60.5 (+28.4)	+25.4
	$90^{\circ}$ Threshold	55.6 (+18.6)	57.2 (+19.8)	44.0 (+6.6)	+15.0
	Exact Angle (MCQ)	32.9 (+8.2)	33.7 (+11.5)	28.0 (+5.8)	+8.5
	Exact Angle (Open)	34.6 (+9.1)	27.2 (+4.1)	27.2 (+3.7)	+5.6
Multi-view	View Matching	67.6 (+2.3)	57.4 (+1.4)	50.9 (+1.8)	+1.8

Table 2: Task-level RotationCue results on OR-Bench across three VLMs. Main values are scores (%); smaller lower-right values in parentheses are gains over Raw (

\Delta

, percentage points), with shading encoding

\Delta

. Mean

\Delta

averages the gain across the three models.

	(a) Source of intermediate context					(b) RotationCue content quality
Model	Raw	ZS-CoT	5-shot	Self-gen.	RotationCue	Shuffled	Raw	RotationCue	Oracle
Qwen3-VL-8B	$46.8$	$47.1$	$52.3$	$48.6$	58.0	$44.4$	$46.8$	58.0	65.3
Qwen2.5-VL-7B	$42.3$	$43.1$	$44.4$	$47.1$	54.9	$38.7$	$42.3$	54.9	63.6
InternVL3.5-8B	$43.0$	$39.7$	$44.4$	$47.4$	50.9	$42.9$	$43.0$	50.9	59.6

Table 3: Macro-8 (%) with (a) different intermediate-context strategies and (b) RotationCue content quality. Self-gen uses the same three-field format as RotationCue; Shuffled uses a cue from another example; Oracle uses ground-truth rotation information.

	OR-Bench		External benchmarks
Method	Macro-8	Micro-8	VSR-ZS	CV 2D	CV 3D	CV Avg.	SpinBench
Qwen3-VL-8B (Raw)	46.8	53.0	83.6	80.1	93.3	86.7	67.0
Rotation-SFT	51.2 (+4.4)	56.3 (+3.3)	75.6 ( $-8.0$ )	75.4 ( $-4.7$ )	89.0 ( $-4.3$ )	82.2 ( $-4.5$ )	61.8 ( $-5.2$ )
RotationCue	58.0 (+11.2)	62.9 (+9.9)	84.0 (+0.4)	80.1 (—)	92.5 ( $-0.8$ )	86.3 ( $-0.4$ )	69.2 (+2.2)

Table 4: Comparison between RotationCue and Rotation-SFT on Qwen3-VL-8B (%). RotationCue achieves larger gains on OR-Bench while largely preserving performance on external benchmarks. Parenthesized values denote changes from the raw model, computed from displayed scores.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Illustrative examples of object screening. Objects whose orientation is visually ambiguous due to symmetry, such as the vase (left), are excluded. Objects with distinctive orientation cues, such as the sofa (right), are retained.

Task Group	Task	Examples	Imgs./Ex.	Uniform reference
Two-view	Rotation Detection	486	2	50.0%
	Coarse Angle	243	2	33.3%
	$90^{\circ}$ Threshold	243	2	33.3%
	Exact Angle (MCQ)	243	2	11.1%
	Exact Angle (Open)	243	2	11.1%
Multi-view	View Matching	216	4	33.3%

Appendix

Table 5: Composition of OR-Bench. Uniform reference denotes uniform guessing over the answer choices for closed-set tasks and over the nine reference angles for Exact Angle (Open).

Figure 8: Human evaluation interface for OR-Bench. The interface supports task-wise navigation and progress tracking. A Rotation Detection example is shown.

Model	$\{0^{\circ},90^{\circ},180^{\circ}\}$	$90^{\circ}$
GPT-4o	82.0	62.3
GPT-4o-mini	98.4	70.5
GPT-5.4	68.9	28.7

Appendix

Table 6: Share of predictions assigned to canonical angles and to

90^{\circ}

by proprietary models (%).

	VE probes			Output-head readout
Model	L8	L16	Final	$h_{\mathrm{pre}}$	LM Head	$\Delta$
Qwen3-VL-8B	38.4	22.2	13.4	27.2	5.6	$+21.6$
Qwen2.5-VL-7B	29.2	31.0	13.9	9.9	6.2	$+3.7$
InternVL3.5-8B	31.0	20.8	24.1	24.1	6.2	$+17.9$

Appendix

Table 7: Probe and native performance on

90^{\circ}

Threshold (AdjBA, %; chance-adjusted, chance

=0

). VE probes use absdiff on the two frozen single-image representations; the

h_{\mathrm{pre}}

probe and the LM head read the same representation. Native is the model’s own answer and

\Delta

is the probe–native difference at

h_{\mathrm{pre}}

. Evaluated on 243 image pairs.

	Coarse Angle			$90^{\circ}$ Threshold
Model	Probe	Native	$\Delta$	Probe	Native	$\Delta$
Qwen3-VL-8B	22.2	11.1	$+11.1$	32.1	5.6	$+26.5$
Qwen2.5-VL-7B	16.0	12.3	$+3.7$	21.0	6.2	$+14.8$
InternVL3.5-8B	21.0	-1.9	$+22.8$	14.2	6.2	$+8.0$

Appendix

Table 8: Probes trained on external ALOI data and evaluated on OR-Bench (AdjBA, %; chance-adjusted, chance

=0

). Probes are fit on 972 ALOI pairs with object-disjoint splits and never see the test set.

\Delta

is the probe–native difference. Each test set contains 243 image pairs.

Model	True labels	Random labels
Qwen3-VL-8B	35.8	$-6.2$
Qwen2.5-VL-7B	28.4	$-4.3$
InternVL3.5-8B	38.9	$-3.8$

Appendix

Table 9: Random-label control for Coarse Angle readout from VE L8 representations (AdjBA, %).

Base VLM	Decoder input	Image preprocessing
Qwen3-VL-8B	Vision L8 mean	Qwen AutoProcessor, dynamic resolution
Qwen2.5-VL-7B	Vision L8 mean	Qwen AutoProcessor, dynamic resolution
InternVL3.5-8B	Vision L8 mean	$448\times 448$ , at most one tile

Appendix

Table 10: Frozen internal representations used as RotationCue inputs.

Configuration	Value
Training pairs	1,200,000
Validation pairs	50,000
Epochs	6
Batch size	1024
Optimizer	AdamW
Learning rate	$10^{-4}$

Appendix

Table 11: Shared RotationCue decoder training configuration.

Model	Epoch	Same-object	Rotation-present	Angle-range	Joint
Qwen3-VL-8B	5	98.70	98.68	67.44	77.75
Qwen2.5-VL-7B	6	98.03	98.01	63.85	74.95
InternVL3.5-8B	5	97.28	97.26	67.65	76.47

Appendix

Table 12: Internal validation performance (%) of the selected RotationCue decoder checkpoints.

Task	Constructed pairs	# Cards
View Matching	Ref–A, Ref–B, Ref–C	3
Post-Rotation View	Ref–A, Ref–B, Ref–C, Ref–D	4
Rotation Comparison	A1–A2, B1–B2	2

Appendix

Table 13: Pair construction for multi-image OR-Bench tasks.

Base VLM	Raw	Same + Rotation	Angle Range	Full RotationCue
Qwen3-VL-8B	46.8	48.5 $(+1.7)$	56.4 $(+9.6)$	58.0 $(+11.2)$
Qwen2.5-VL-7B	42.3	43.4 $(+1.1)$	50.7 $(+8.4)$	54.9 $(+12.6)$
InternVL3.5-8B	43.0	45.6 $(+2.6)$	50.3 $(+7.3)$	50.9 $(+7.9)$

Appendix

Table 14: Ablation of the fields included in the RotationCue context. Values are Macro-8 scores (%); values in parentheses are gains over Raw (

\Delta

, percentage points).

Method	Acc. Mean	Acc. Std	Exact Cons.	Flip Rate	Entropy	Correct-Stable
Raw	56.6	0.7	84.0	16.0	0.131	50.5
Zero-shot CoT	58.2	1.2	71.3	28.8	0.263	47.3
RotationCue	64.8	0.5	86.0	14.0	0.111	59.0

Appendix

Table 15: Accuracy and prediction stability under stochastic decoding on Qwen3-VL-8B. Each of 400 examples is decoded nine times with temperature

0.5

and top-

p=0.95

Benchmark	# Examples	RotationCue construction
VSR-ZS	1,222	Duplicate single image: $I$ – $I$
CV-Bench	2,638	Duplicate single image: $I$ – $I$
SpinBench	2,739	Task-aware pair construction with Raw fallback

Appendix

Table 16: External spatial benchmarks and RotationCue input construction.

Configuration	Value
Base model	Qwen3-VL-8B
Training examples	900,000
Validation examples	20,000
Epochs	1
LoRA rank	8
LoRA alpha	16

Appendix

Table 17: Rotation-SFT training configuration.

Task	N	Raw	Rotation-SFT	$\bm{\Delta}$
Rotation Detection	486	99.6	89.9	$-9.7$
Coarse Angle	243	40.7	85.2	$+44.5$
$90^{\circ}$ Threshold	243	37.0	41.6	$+4.6$
Exact Angle (MCQ)	243	24.7	40.7	$+16.0$
Exact Angle (Open)	243	25.5	27.6	$+2.1$
View Matching	216	65.3	54.2	$-11.1$

Appendix

Table 18: Task-level accuracy (%) of Qwen3-VL-8B before and after Rotation-SFT. Macro-8 gives equal weight to the eight tasks, whereas Micro-8 aggregates all 2,109 examples. Deltas are computed from displayed scores.

Explore similar work

Readable Yet Unpredictable: Rotated-Outcome Prediction in Vision-Language Models
Jun 1, 2026Lexin Wang, Shenghua Liu, Yiwei Wang +2
Institute of Computing Technology, Chinese Academy of Sciences · University of California, Merced
Fine-Tuning VLM for Enhancing AI's Spatial Intelligence: Understanding 3D and 2D Rotations
Oct 3, 2026Uttamasha Monjoree, Wei Yan3D Spatial Reasoning
Department of Architecture Texas A&M University College Station, USA
How Do LLMs and VLMs Understand Viewpoint Rotation Without Vision? An Interpretability Study
Apr 16, 2026Zhen Yang, Ping Jian, Zhongbin Guo +5Interpretability Linguistics
School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China · Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences), Jinan, China

Jun 1, 2026, cs.CVJ/K move · Enter open · S save

Readable Yet Unpredictable: Rotated-Outcome Prediction in Vision-Language Models

Lexin Wang, Shenghua Liu, Yiwei Wang, Jiafeng Guo+1

Institute of Computing Technology, Chinese Academy of Sciences · University of California, Merced

Open paper AskPDF arXiv

Can vision-language models predict what a 180° rotation would reveal from the original image alone? We study this ability through Rotated-Outcome Prediction: given an original image, a model must answer what would be seen or read after a 180° in-plane rotation, without directly observing the rotated target. To isolate this gap, we introduce RotOutBench, a paired diagnostic benchmark spanning open visual cases and controlled text-image rotations. A sharp pattern emerges: many VLMs can recognize the relevant content when directly given either the original or rotated image, yet fail to infer the rotated result from the original image alone. On controlled text-image rotations, predicted-rotation accuracy collapses to near zero even for models with high direct-reading accuracy. A model-level case study further shows that the prediction state can approach a rotated-image reading state, while the final readout still shifts toward the original string. Current VLMs can recognize a transformed visual state when it is shown, but often fail to predict that state from the original view.

Abstract

Explore similar work

Readable Yet Unpredictable: Rotated-Outcome Prediction in Vision-Language Models

Fine-Tuning VLM for Enhancing AI's Spatial Intelligence: Understanding 3D and 2D Rotations

How Do LLMs and VLMs Understand Viewpoint Rotation Without Vision? An Interpretability Study