Recent multimodal language models can reason directly about robot scenes, but it is unclear whether they need to repeatedly reinterpret raw visual observations at every step of an interaction. We propose shifting routine spatial reasoning from images to text and symbols through a 3D geometric abstraction that is refreshed locally from observations and read directly by the language models. Specifically, our interface represents scene geometry via typed, evidence-aware symbolic references to object parts, grounds them from calibrated RGB-D, and returns compact textual checkpoints with metric residuals after each action. Symbolic task structure can thus be reused across decisions while metric values are rebound to the current scene, with direct visual reasoning invoked only on demand. Across offline policy synthesis and continuous online execution, reasoning over grounded text and symbols improves multi-step manipulation completion while substantially reducing repeated multimodal queries, positioning explicit geometric state as an effective substrate for agentic robot policies.
Figures & tables
Figure 1: Geometric abstraction between observation, reasoning, and action. Visual observations are grounded locally into metric 3D geometry and exposed to the language model through symbolic references and compact textual checkpoints. Policies reason in text and symbols over stable geometric roles whose metric values are rebound to the current scene during execution.
Figure 2: Geometric abstraction between perception and reasoning. RGB-D observations are grounded into object geometry and exposed to the LLM through typed symbolic references and compact textual checkpoints. In policy synthesis, the LLM revises executable policies between rollouts, while perception and control run locally. In online execution, the same interface supplies updated geometry for successive decisions within a continuing episode.
Operation
Inputs
Output
reference
object and role
geometric entity with evidence
relation
predicate and objects
residuals and true / false / unknown
align
source and target geometry
desired rigid transform
checkpoint
active object and goal
compact textual state
Table 1: Operators on the geometric state.
Figure 3: Single-policy acquisition. Policies are revised between reset rollouts and evaluated after training freezes. Geometric Abstraction additionally uses supplied geometric references. Right: illustrative start and goal configurations, constructed in simulation rather than recorded policy trajectories.
Figure 4: Test success and acquisition cost. Lines show mean held-out test SR of training-retained policies; bars show cumulative input-plus-output tokens through each iteration, including rejected proposals. Both average over five chains. Ring-on-peg Pixel Reasoning excludes unknown usage from two interrupted requests.
Figure 5: Online execution and tasks. The agent acts in a single continuing scene until completion or the 200 s timeout; Geometric Abstraction receives grounded symbolic references and textual checkpoints. Right: illustrative start and goal states.
Figure 6: Online outcomes and cost. Completion rate within 200 s (left), mean capped completion time (middle; timeouts assigned 200 s), and mean recorded tokens (right) over 50 paired layouts per task ( † : excludes unknown interrupted tokens).
Figure 7: Model interaction cost. Mean (bars) and per-episode (markers) request wall-clock time (left) and recorded tokens (right) over 50 paired episodes per task ( △ : excludes unknown interrupted tokens).
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Retained test SR (%)
Cumulative tokens (k)
Task
Condition
R1
R2
R3
R1
R1–R2
R1–R3
Bridge + cylinder
Pixel Reasoning
21.2±22.0
53.0±11.7
56.0±9.1
24.6
66.2
125.7
Geometric Abstraction (ours)
45.2±25.4
63.4±5.9
65.4±3.5
25.8
67.3
127.0
Three-cube stacking
Pixel Reasoning
17.2±29.4
53.6±26.2
70.8±26.5
28.0
74.1
138.1
Geometric Abstraction (ours)
57.2±32.6
72.2±7.2
81.4±12.3
25.7
64.7
123.4
Ring-on-peg
Pixel Reasoning
0.0±0.0
0.0±0.0
0.0±0.0
37.8
93.5
166.8
Appendix
Table 2: Single-policy acquisition outcomes and token usage. Held-out test SR (mean ± sample SD) across five training chains on 100 shared held-out layouts per task, using training-only policy retention. Tokens report cumulative completed-request input plus output per chain (in thousands), including rejected proposals and counting cached input once. Ring-on-peg Pixel Reasoning excludes unknown token usage from two interrupted requests. Bold highlights where Geometric Abstraction (ours) improves over Pixel Reasoning.
Task
Condition
Success
Time (s)
Model (s)
Tokens (k)
Requests
Images
Bridge-tower
Pixel Reasoning
12/50
198.0
55.5
112.5
7.88
7.50
Geometric Abstraction (ours)
28/50
185.3
51.8
129.5
7.32
4.28
Ring-on-peg
Pixel Reasoning
49/50
115.8
90.0
214.5
11.88
10.94
Geometric Abstraction (ours)
50/50
77.2
45.6
101.3
6.90
2.72
Tower relocation
Pixel Reasoning
15/50
186.4
80.3
185.9
11.58
11.34
Geometric Abstraction (ours)
42/50
139.2
31.7
88.3
5.36
3.00
Appendix
Table 3: Online outcomes and interaction cost. Each task evaluates 50 paired layouts. Success reports completed episodes out of 50; all other columns report per-episode means across all 50 episodes, including timeouts. Time is capped completion time, assigning 200 seconds to timeouts. Model is cumulative request wall-clock time, including interrupted waits. Tokens are recorded input plus output tokens (in thousands), counting cached input once and excluding unknown interrupted usage. Requests counts completed model calls; Images counts newly attached camera images across all request attempts (excluding local sensing frames not sent to the LLM). Both conditions use audited repeats for nine runtime-flagged pairs. Bold highlights where Geometric Abstraction (ours) improves over Pixel Reasoning.
Figure 8: Cumulative success over elapsed wall-clock time. Each curve summarizes the 50 paired 200-second episodes for Pixel Reasoning and Geometric Abstraction (ours), stepping at the first verified completion time. Timeouts do not convert to successes at the 200 s deadline. Intermediate values are read directly from the 200-second episodes rather than separate evaluations under shorter budgets.
Designing or learning robot policies that generalize zero-shot across a range of language instructions and objects is a core problem in robotics. Vision-Language-Action models (VLAs) learn such policies end-to-end by repurposing existing Vision-Language Models (VLMs), but generalization to new instructions and objects remains challenging. An alternative is to implement a modular policy by leveraging an explicit VLM-based 3D scene representation and motion planning. While modular policies show strong zero-shot potential, they typically retrieve objects based on semantics without explicit spatial reasoning, severely restricting their overall grounding capabilities. They also interact with objects using basic grasping and navigation skills. In this work, we address these limitations by unifying grounding capabilities and robot skills in a single agentic action space through a scene-agent tool interface. By leveraging part-level affordances, our skills generalize across diverse objects and enable zero-shot interactions such as unplugging chargers and opening drawers. We name the resulting framework Agentic Scene Policies (ASP). Through extensive real-world experiments, we show how ASP consistently outperforms leading VLAs in the zero-shot setting. We also demonstrate the extensibility of our framework by introducing a mobile version of ASP to tackle room-level queries. See our project page (https://montrealrobotics.ca/agentic-scene-policies.github.io/) for more results.
Sacha Morin, Kumaraditya Gupta, Mahtab Sandhu +4
Université de Montréal · Mila - Quebec AI Institute · Sapienza University of Rome
Recent advancements in Multimodal Large Language Models (MLLMs) have incentivized models to ``think with images'' by actively invoking visual tools during multi-turn reasoning. The common Reinforcement Learning (RL) practice of relying on outcome-based rewards ignores the fact that textual plausibility often masks executive failure, meaning that models may exhibit intuitive textual reasoning while executing imprecise or irrelevant visual actions within their agentic reasoning trajectories. This reasoning-action discrepancy introduces noise that accumulates throughout the multi-turn reasoning process, severely degrading the model's multimodal reasoning capabilities and potentially leading to training collapse. In this paper, we introduce Multimodal Agentic Policy Optimization (MAPO), bridging the gap between textual reasoning and visual actions generated by models within their Multimodal Chain-of-Thought (MCoT). Specifically, MAPO mandates the model to generate explicit textual descriptions for the visual content obtained via tool usage. We then employ a novel advantage estimation that couples the semantic alignment between these descriptions and the actual observations with the task reward. Theoretical findings are provided to justify the rationale behind MAPO, which inherently reduces the variance of gradients, and extensive experiments demonstrate that our method achieves superior performance across multiple visual reasoning benchmarks.
Wenhao Yang, Yu Xia, Jinlong Huang +10
National Key Laboratory for Novel Software Technology, Nanjing University, China · School of Artificial Intelligence, Nanjing University, China · AI Business, Alibaba Group +5
Vision-language models with extended reasoning succeed on complex problems, but many real-world problems require external tools that internal reasoning alone often cannot resolve. Agentic reasoning therefore interleaves two behaviors with a structural asymmetry: thinking (the self-contained default) and tool use (a high-variance auxiliary acting). We refer to this asymmetry as the Thinking-Acting Gap. Under standard RL recipes like GRPO, the gap manifests as two diagnostic symptoms during training: tool use is attempted on only ~30% of rollouts, and when attempted, the tool-using rollouts within a group are all-wrong on ~40% of questions, suppressing the learning signal at the tool calls that needed it. We propose AXPO (Agent eXplorative Policy Optimization): for each all-wrong tool-using subgroup, AXPO fixes the thinking prefix and resamples the tool call and its continuation, paired with uncertainty-based prefix selection. Across nine multimodal benchmarks and three scales of Qwen3-VL-Thinking, SFT+AXPO outperforms SFT+GRPO at average (+1.8pp Pass@1 and +1.8pp Pass@4 at 8B on average) and 8B with SFT+AXPO surpasses the 32B Base on Pass@4 with 4 times fewer parameters.