Multimodal reasoning models (MRMs) trained with reinforcement learning with verifiable rewards (RLVR) show improved accuracy on visual reasoning benchmarks. However, we observe that accuracy gains often come at the cost of reasoning quality: generated Chain-of-Thought (CoT) traces are frequently inconsistent with the final answer and poorly grounded in the visual evidence. We systematically study this phenomenon across seven challenging real-world spatial reasoning benchmarks and find that it affects contemporary MRMs such as ViGoRL-Spatial, TreeVGR as well as our own models trained with standard Group Relative Policy Optimization (GRPO). We characterize CoT reasoning quality along two complementary axes: "logical consistency" (does the CoT entail the final answer?) and "visual grounding" (does each reasoning step accurately describe objects, attributes, and spatial relationships in the image?). To address this, we propose Faithful GRPO (FGRPO), a variant of GRPO that enforces consistency and grounding as constraints via Lagrangian dual ascent. FGRPO incorporates batch-level consistency and grounding constraints into the advantage computation within a group, adaptively adjusting the relative importance of constraints during optimization. We evaluate FGRPO on Qwen2.5-VL-7B and 3B backbones across seven spatial datasets. Our results show that FGRPO substantially improves reasoning quality, reducing the inconsistency rate from 24.5% to 1.7% and improving visual grounding scores by +13%. It also improves final answer accuracy over simple GRPO, demonstrating that faithful reasoning enables better answers.
Figures & tables
Figure 1: Unfaithful reasoning masked by correct answers. Both models answer correctly, but only FGRPO reasons faithfully. Left: The GRPO-Task model incorrectly claims there are no visible paths, contradicting its own answer of “1.0” (50% faithfulness, inconsistent). Right: The GRPO-Task model reasons toward “lamp” but answers “box” (33% faithfulness, inconsistent). FGRPO produces visually grounded reasoning in both cases (100% faithfulness, consistent). Sentences are color-coded: blue are grounded and orange are ungrounded.
Figure 2: Overview of the FGRPO training pipeline. We show advantage computation for a training batch with 3 samples, each with 2 rollouts. For each prompt-image pair si , the policy samples G=2 rollouts and we compute the task reward RT , the consistency reward RC , and grounding rewards RS and RG (only RG is shown for clarity). The consistency and semantic-grounding rewards are provided by an online VLM judge. Each signal is independently normalized within the rollout group to obtain decoupled advantages, which are then combined into the final FGRPO advantage using Lagrange multipliers. Batch-level constraint statistics are used to update the multipliers via dual ascent, increasing the weight of violated constraints and decreasing the weight of satisfied ones.
Method
CVB 2D
CVB 3D
Mind Cube
MMVP
Omni Spatial
Real WorldQA
SAT Real
Avg.
Base model (7B): Qwen2.5-VL-7B-Instruct
Non-reasoning
77.17
83.78
35.11
75.78
45.23
69.02
63.11
64.17
CoT prompting
75.92
76.09
30.83
72.44
40.40
63.05
59.22
59.71
Base model (3B): Qwen2.5-VL-3B-Instruct
Non-reasoning
70.58
74.00
43.71
64.67
45.92
65.88
59.00
60.54
CoT prompting
71.21
68.00
40.38
63.67
40.77
62.35
55.00
57.34
Table 1: Main results. Pass@1 accuracy on seven spatial reasoning benchmarks. All 7B models use Qwen2.5-VL-7B-Instruct as backbone; 3B models use Qwen2.5-VL-3B-Instruct. Bold = best, underline = second best among open-weight 7B models. FGRPO achieves the highest average accuracy at both scales, outperforming all MRM baselines and the unconstrained GRPO-T baseline while simultaneously improving reasoning quality.
Figure 3: Per-dataset reasoning quality breakdown . (a) Semantic grounding ( S ): FGRPO achieves uniformly higher semantic grounding than GRPO-T across all seven benchmarks (86.0% vs. 72.7%), with the largest gains on MindCube (+22.8 pp) and OmniSpatial (+21.1 pp). TreeVGR also outperforms GRPO-T (81.9%). (b) Inconsistency rate: FGRPO reduces inconsistency to 1.7% on average, compared to 26.1% (GRPO-T), 26.0% (TreeVGR), and 16.2% (ViGoRL). MindCube and OmniSpatial are the most challenging for all baselines; FGRPO virtually eliminates inconsistency on every benchmark.
Figure 5: An overview of the two-stage training pipeline. We curate CoT training data using MCTS with a strong visual teacher, covering bounding-box grounded visual reasoning. We perform SFT on the curated CoT data using data from SAT ( Ray et al., 2025 ) , VGR ( Wang et al., 2025c ) and VisCoT ( Shao et al., 2024a ) , followed by RL training. We add a subset of TreeVGR-RL-37K ( Wang et al., 2025a ) to construct diverse RL data with bounding-box supervision.
Table 9
Hyperparameter
Value
Advantage estimator
CGRPO
Constraint keys
[consistency, bbox_reward, faithfulness]
Consistency constraint
Threshold ( τC )
0.95
Initial λC
1.0
Masking
Only correct predictions ( Racc>0 )
Appendix
Table 6: FGRPO Constraint Hyperparameters
Hyperparameter
Value
Teacher Model
Qwen2.5-VL-72B-Instruct
Nsim
8
cpuct
2.0
Branching Factor ( k )
3
Rollouts per Node ( n )
2
Max Depth
10
Appendix
Table 7: Hyperparameters for MCTS-based data synthesis.
Dataset
#Questions
Subtasks
CV-Bench2D ( Tong et al., 2024a )
1.4K
2D, spatial relations
CV-Bench3D ( Tong et al., 2024a )
1.2K
3D, depth
MindCube ( Yin et al., 2025 )
1K
3D, multi-image
MMVP ( Tong et al., 2024b )
300
2D, spatial relations
OmniSpatial ( Jia et al., 2025 )
1.5K
3D, dynamics
RealWorldQA ( xAI, 2025 )
765
2D, real-world
Appendix
Table 8: Summary of evaluation datasets. Star (*) indicates circular evaluation is employed.
Figure 6: Prompt used for the consistency constraint evaluation. The judge receives only the question, reasoning trace, and final answer—no image is provided. It outputs YES (consistent) or NO (inconsistent).
Figure 7: Prompt used for the semantic grounding (faithfulness) constraint evaluation. The judge receives the image, question, accumulated reasoning context, and the latest sentence to evaluate. It outputs CORRECT , INCORRECT , or SKIP . The per-sample semantic grounding score S is the fraction of visual sentences (non- SKIP ) that are CORRECT .
Figure 8: Training dynamics (7B FGRPO). Top: Lagrange multiplier trajectories. Bottom: constraint satisfaction over training steps.
Figure 9: Perspective estimation. GRPO-Task claims the photo was taken from above despite the upward-looking perspective of the wine glass, contradicting its own correct answer. FGRPO correctly identifies the low vantage point.
Figure 10: Navigation and signage. GRPO-Task misreads the “Entering” sign as an “Exiting” sign and contradicts its own answer. FGRPO correctly interprets the signage and produces consistent reasoning.
Figure 11: Relative distance. GRPO-T incorrectly concludes the stop sign is closer, contradicting its answer of “school zone sign.” FGRPO correctly identifies relative depth from the scene layout.
Figure 12: Navigation and signage. GRPO-Task claims that the “Entering” sign does not lead to terminal B, yet selects option A (yes) as the final answer contradicting its reasoning. Whereas, FGRPO correctly identifies the entry for Terminal B and answers yes.
Figure 13: Object counting. GRPO-Task claims two giraffes are present despite the image showing three, contradicting its correct answer. FGRPO accurately counts three giraffes with consistent reasoning.
Figure 14: Depth with bounding boxes. GRPO-Task claims the lamp (red box) is closer, contradicting its answer of “pillow” (blue box). FGRPO correctly reasons about relative depth using the bounding-box annotations.
Figure 15: Directional reasoning. GRPO-Task concludes the left lane is correct for reaching Porte de Vaise, contradicting its answer of “Yes” (right lane). FGRPO correctly reads the road sign and produces consistent reasoning.
Figure 16: Egocentric spatial reasoning. GRPO-Task reasons that turning right would move parallel to the bus rather than toward the door, contradicting its answer. FGRPO correctly reasons about the egocentric perspective change.
Figure 17: FGRPO responses on the eval set. We observe that FGRPO responds faithfully and exhibits both spatial groundedness and consistency.
Reinforcement learning with verifiable rewards (RLVR) has emerged as a promising paradigm for advancing complex reasoning in large language models, and recent work extends RLVR to multimodal large language models (MLLMs). This transfer, however, surfaces a faithfulness challenge: faithful perception of task-relevant visual evidence and faithful use of that evidence during reasoning, leading to unsatisfactory gains on multimodal benchmarks. Specifically, existing perception supervision often operates on textual descriptions rather than natively on image regions, and faithful use is largely overlooked, exposing the perception-reasoning disconnect where correctly perceived evidence is dropped or contradicted during reasoning. To close these gaps, we propose Faithful-MR1, a training framework that anchors and reinforces visual attention to address both halves of faithful multimodal reasoning. The Anchoring stage turns perception into an explicit pre-reasoning subtask, supervising a dedicated <Focus> token's attention directly against image regions rather than through textual descriptions. The Reinforcing stage exposes faithful use through counterfactual image intervention, rewarding answer-correct trajectories that concentrate visual attention where vision causally matters. Extensive experiments demonstrate that Faithful-MR1 outperforms recent multimodal reasoning baselines on both Qwen2.5-VL-Instruct 3B and 7B backbones while using substantially less training data.
Changyuan Tian, Zhicong Lu, Huaxing Liu +7
AMAP, Alibaba Group · University of Chinese Academy of Sciences · Tsinghua University +1
Multimodal Reasoning Models (MRMs) leveraging Chain-of-Thought (CoT) based thinking have revolutionized mathematical and logical problem-solving. However, we show that this paradigm struggles with generalized spatial intelligence. We perform a comprehensive evaluation of seventeen models across thirteen spatial benchmarks and identify a critical gap: CoT prompting consistently degrades performance in visual spatial reasoning. Furthermore, through a novel No-Image++ ablation, we demonstrate that MRMs and CoT prompted MLMs suffer from severe shortcut learning, and hallucinate visual details from textual priors even when the image is absent. These findings challenge the efficacy of text-only CoT for spatial tasks and underscore the need for vision-centric reasoning paradigms.
Sai Srinivas Kancheti, Aditya Sanjiv Kanade, Vineeth N. Balasubramanian +1
Current Large Reasoning Models (LRMs) exhibit remarkable general capabilities but significantly underperform in spatial reasoning tasks. Existing approaches treat this gap as a knowledge deficit, relying on supervised fine-tuning (SFT) to ingest labeled spatial data from external vision sources or synthetic engines. In contrast, we argue that for many tasks, spatial reasoning capabilities are already present in pre-trained LRMs but require alignment through logical coherence under geometric 2D and 3D constraints. In this work, we propose a self-supervised reinforcement learning (RL) framework that targets the internal reasoning process without requiring ground-truth annotations. By formalizing the notion of consistency verifiers -- reward functions that check for geometric and semantic consistency under transformations -- we demonstrate that models can improve their spatial reasoning abilities. We use both image transformations, like flipping, and textual transformations, like swapping the order of objects in the question, and propose a new optimal transport-based RL strategy, OT-GRPO, which is a minimal-matching variant of group relative policy optimization tailored to pairwise verifiers. We show that this label-free consistency training approaches the accuracy of models trained with ground-truth supervision and achieves similar generalization across diverse tasks and data domains.
Theo Uscidda, Marta Tintore Gazulla, Maks Ovsjanikov +2
CREST, ENSAE, Institut Polytechnique de Paris · Google Zurich · Google DeepMind