Large Language Models (LLMs) and Vision Language Models (VLMs) have shown impressive reasoning abilities, yet they struggle with spatial understanding and layout consistency when performing fine-grained visual editing. We introduce a Structured Reasoning framework that performs text-conditioned spatial layout editing via scene-graph reasoning. Given an input scene graph and a natural-language instruction, the model reasons over the graph to generate an updated scene graph that satisfies the text condition while maintaining spatial coherence. By explicitly guiding the reasoning process through structured relational representations, our approach improves both interpretability and control over spatial relationships. We evaluate our method on a new text-guided layout editing benchmark encompassing sorting, spatial alignment, and room-editing tasks. Our training paradigm yields an average 15% improvement in IoU and 25% reduction in center-distance error compared to Chain of Thought Fine-tuning (CoT-SFT) and vanilla GRPO baselines. Compared to SOTA zero-shot LLMs, our best models achieve up to 20% higher mIoU, demonstrating markedly improved spatial precision.
Figures & tables
Figure 2: Example from the synthetic 3D sorting benchmark. Given a instruction to group and sort objects by shape and height, our model generates a concise structured reasoning trace with JSON scene-graph updates that transforms the initial scene into the final layout, in contrast to a long, ambiguous free-form thinking path.
Figure 3: Overview of our training pipeline. The vision-language model predicts step-by-step layout edits from the instruction and initial scene graph, and rollouts are optimized using a combination of format, IoU, and collision-free rewards.
Model
Task 1: Caption-Free
Task 2: Noisy-Input
IoU
Collision Free
IoU
Collision Free
Qwen3-VL-235B-Instruct
0.408
0.631
0.428
0.679
Qwen3-VL-235B-Thinking
0.472
0.744
0.378
0.701
Qwen3-8B (Ours, for reference)
0.209
0.984
0.664
0.939
Qwen2.5-VL-7B (Ans-SFT)
0.521
0.700
0.619
0.667
Qwen2.5-VL-7B (CoT-SFT)
0.641
0.829
0.773
0.844
Table 1: Results on caption-free and noisy-input scene-graph grounding and editing.
Table 2: Quantitative results on the spatial alignment benchmark
Model
Mean IoU ↑
Ctr. Dist. ↓
Col. Free ↑
Edit Dist. ↓
Large Language Models (Zero-shot)
Qwen3-235B-Instruct
0.122
2.702
0.548
6.285
Qwen3-235B-Thinking
0.708
0.856
0.957
2.208
Qwen3-VL-235B-Instruct
0.145
2.323
0.421
8.120
Qwen3-VL-235B-Thinking
0.706
0.568
0.918
1.900
Deepseek-V3
0.236
1.242
0.620
2.010
Table 3: Quantitative results on the 3D Sorting benchmark. Models equipped with our structured scene-graph reasoning (“Ours”) consistently outperform both zero-shot LLM/VLM baselines and standard CoT-SFT fine-tuning.
Figure 4: Example of text-guided 3D room layout reasoning , showing how the model interprets constraints to update object poses step by step and validate distances.
Figure 5: Out-of-domain warehouse simulation results showing our model correctly following user instructions with 3D boxes.
Table 4: Quantitative results on the RoomEditing benchmark
Figure 6: Real-world tabletop rearrangement and pick-and-place task.
Figure 7: Template of the prompts used to generate reasoning traces over scene graphs for all spatial tasks.
Figure 8: Scaling behavior on the Sorting benchmark. Reasoning-based training improves performance consistently across all model scales (1.7B–8B).
Format
IoU
CF
IoU
Collision Free
Center Distance
✗
✓
✓
0.819
1.000
0.274
✓
✗
✓
0.415
1.000
0.998
✓
✓
✗
0.919
0.981
0.153
✓
✓
✓
0.924
1.000
0.145
Table 5: Ablation on reward components. Format and IoU rewards are essential, while CF brings an additional but smaller improvement.
Figure 9: Qualitative comparison of sorting and spatial alignment tasks: Qwen3-235B and DeepSeek often fail to follow the instructions and produce collisions/overlaps, while our method matches the ground truth.