cs.AIOct 3, 2026

Agentic AI with Structured CoT for Enhancing AI's Spatial Intelligence: Visualization and Reasoning of Rotation

Authors: Uttamasha Monjoree, Wei Yan

Organizations: Department of Architecture Texas A&M University College Station, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Recent studies show that artificial intelligence (AI) with language and vision capabilities still experiences limitations in spatial reasoning. In this paper, we have studied the spatial capabilities of advanced generative AI to understand the rotations of objects in 3D space, utilizing AI's image processing and language processing features. We trained and examined the spatial intelligence of a generative Agentic AI model (GPT-5.6) to understand the spatial rotation process with rotation diagrams based on the revised Purdue Spatial Visualization Test: Visualization of Rotations (Revised PSVT:R). We improvised the Revised PSVT:R by superimposing additional graphical and contextual features to evaluate how different Chain-of-Thought (CoT) reasoning strategies influence model performance. The results indicate that structured CoT reasoning improves the spatial reasoning performance of the base GPT-5.6 model in both datasets (PSVT:R and PSVT:R with coordinate system). We used three CoT approaches - (1) Structured CoT, (2) few-shot Structured CoT, and Structured CoT with Self-optimized Prompt. The three CoT approaches evaluated in this study showed no significant performance difference. Results showed that combining structured CoT reasoning with relevant contextual information leads to considerable improvements in VLM performance on 3D rotation tasks, demonstrating the potential of agentic AI for more effective spatial reasoning. However, when contextual information is removed, structured CoT reasoning alone provides limited improvement, and the models continue to exhibit notable difficulties in understanding spatial transformations. These findings suggest that effective spatial reasoning in VLMs relies on the integration of visual, textual, and reasoning-based information in future agentic AI systems for spatial intelligence.

Figures & tables

Fig. 1 : (a) An example question from Revised PSVT:R, (b) the example question from Revised PSVT:R w/ Axes. In both questions, D is the correct answer.
Fig. 2 : Framework of Baseline Test and Three CoT Experiments
Fig. 3 : Structured CoT
Fig. 4 : Structured CoT with Self-optimized Prompt
Fig. 5 : Three Strategies of Structured CoT with Self-optimized Prompting
 Performance of Human Test-TakersPerformance of AI
Test ConditionStudy 1 [ 47 ]Study 2 [ 14 ]BaselineExp 1Exp 2Exp 3
Original or Revised PSVT:R18% participants scored 60% or less; 82% scored above 60%Participants’ mean accuracy was 63.60%, participants’ accuracy ranged from 10% to 100%13.33%40%48%43.33%
Revised PSVT:R w/ AxesN/AN/A26.67%100%96.70%96.70%
TABLE I : Performance comparison based on PSVT:R with percentage of correct answers.
Fig. A1 : Prompt for Structured CoT used in Experiment 1 & Experiment 2
Fig. A2 : Sample answer of the Structured CoT method used in Experiment 1 & Experiment 2. This is the answer to question 1 of Revised PSVT:R (Correct answer B)
Fig. A3 : Structured CoT Self-optimized prompting - Strategy A
Fig. A4 : Structured CoT Self-optimized prompting - Strategy B
Fig. A5 : Structured CoT Self-optimized prompting - Strategy C
Fig. A6 : Sample answer self-optimized Structured CoT method for Question 6 of Revised PSVT:R (Correct answer C)

Explore similar work

CardsList
Oct 3, 2026cs.AI

Fine-Tuning VLM for Enhancing AI's Spatial Intelligence: Understanding 3D and 2D Rotations

Spatial intelligence is a fundamental skill in multiple domains, such as Science, Technology, Engineering, and Mathematics (STEM), Medicine, Architecture, and Construction. Recent studies indicate that Vision-Language Models (VLMs) still face limitations in spatial reasoning, which inhibits artificial intelligence (AI) from performing practical spatial tasks. Using multiple object-rotation datasets developed for training and evaluation, our experiments demonstrated promising improvements in both 2D and 3D rotation detection. Fine-tuned Google DeepMind-built Gemma-4 mixture-of-experts (MoE) models significantly outperformed fine-tuned Gemma-4 generalist models in predicting rotations defined by both their axes and angles. Fine-tuning also substantially improved angle estimation for 2D representation without requiring an explicit coordinate system. Furthermore, identifiable objects did not improve angle-detection accuracy; instead, objects with prominent linear features showed improved performance.

Uttamasha Monjoree, Wei Yan

Department of Architecture Texas A&M University College Station, USA

3D Spatial Reasoning
AskPDF
Aug 12, 2026cs.CV

SCOUT: Unlocking Enhanced Spatial Reasoning via Structured Chain-of-Thought and Multi-Objective Process Reward

Existing Vision-Language Models (VLMs) exhibits a critical bottleneck in robust spatial reasoning. Recent reinforcement learning (RL) methods aim to close this gap with verifiable outcomes, yet they suffer from poor credit assignment across intermediate reasoning steps. Concurrently, structured reasoning approaches overlook the critical depth perception necessary for comprehensive 3D understanding. To address these challenges, we propose SCOUT (Structured Chain-Of-Thought Utilizing Process-Supervised RL Training). Specifically, we design a structured Chain-of-Thought (CoT) framework that explicitly models 3D environmental perception to ensure robust spatial understanding and reasoning. Furthermore, we introduce a novel RL algorithm featuring multi-objective process rewards and a tailored advantage estimation method, facilitating fine-grained credit assignment across distinct segments of the reasoning trajectory. To support our framework, we develop SCOUT-24k, a structured spatial reasoning CoT dataset synthesized through a customized pipeline. Extensive evaluations demonstrate that SCOUT-3B improves upon baseline models by 16.85% and 6.3% on general spatial benchmarks and complex spatial reasoning tasks respectively. Notably, our larger SCOUT-7B even outperforms GPT-4o by a margin of 4.28%. Moreover, despite being trained exclusively on single image, SCOUT-7B exhibits robust out-of-domain generalization to multi-image and video scenarios. These empirical results render SCOUT as a critical step towards next generation of spatially-aware VLMs.

Zile Zhou, Huining Yuan, Weichen Zhang +2

Shenzhen International Graduate School, Tsinghua University, Shenzhen, China

3D Spatial ReasoningSpatial ReasoningRecent Vision-Language ModelsChain-of-Thought ReasoningOffline Reinforcement LearningProcess Reward Model
AskPDF
Apr 16, 2026cs.AI

How Do LLMs and VLMs Understand Viewpoint Rotation Without Vision? An Interpretability Study

Over the past year, spatial intelligence has drawn increasing attention. Many prior works study it from the perspective of visual-spatial intelligence, where models have access to visuospatial information from visual inputs. However, in the absence of visual information, whether linguistic intelligence alone is sufficient to endow models with spatial intelligence, and how models perform relevant tasks with text-only inputs still remain unexplored. Therefore, in this paper, we focus on a fundamental and critical capability in spatial intelligence from a linguistic perspective: viewpoint rotation understanding (VRU). Specifically, LLMs and VLMs are asked to infer their final viewpoint and predict the corresponding observation in an environment given textual description of viewpoint rotation and observation over multiple steps. We find that both LLMs and VLMs perform poorly on our proposed dataset while human can easily achieve 100% accuracy, indicating a substantial gap between current model capabilities and the requirements of spatial intelligence. To uncover the underlying mechanisms, we conduct a layer-wise probing analysis and head-wise causal intervention. Our findings reveal that although models encode viewpoint information in the hidden states, they appear to struggle to bind the viewpoint position with corresponding observation, resulting in a hallucination in final layers. Finally, we selectively fine-tune the key attention heads identified by causal intervention to improve VRU performance. Experimental results demonstrate that such selective fine-tuning achieves improved VRU performance while avoiding catastrophic forgetting of generic abilities. Our dataset and code will be released at https://github.com/Young-Zhen/VRU_Interpret .

Zhen Yang, Ping Jian, Zhongbin Guo +5

School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China · Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences), Jinan, China

InterpretabilityLinguisticsCausal Intervention
AskPDF