Thinking with Constructions: A Benchmark and Policy Optimization for Visual-Text Interleaved Geometric Reasoning

Haokun Zhao; Wanshi Xu; Haidong Yuan; Songjun Cao; Long Ma; Yanghua Xiao

cs.AIMar 19, 2026

Thinking with Constructions: A Benchmark and Policy Optimization for Visual-Text Interleaved Geometric Reasoning

Authors: Haokun Zhao, Wanshi Xu, Haidong Yuan, Songjun Cao, Long Ma, Yanghua Xiao

Organizations: College of Computer Science and Artificial Intelligence, Fudan University · School of ECE, Peking University · School of Software and Microelectronics, Peking University · Tencent Youtu Lab

Abstract

Geometric reasoning inherently requires "thinking with constructions" -- the dynamic manipulation of visual aids to bridge the gap between problem conditions and solutions. However, existing Multimodal Large Language Models (MLLMs) are largely confined to passive inference with static diagrams, lacking the strategic knowledge of when and how to construct effective visual aids. To address this, we present a framework for Visual-Text Interleaved Chain-of-Thought. We first introduce GeoAux-Bench, the first benchmark comprising 4,334 geometry problems that aligns textual construction steps with ground-truth visual updates. Our pilot study reveals two critical insights: (1) interleaved visual-textual aids outperform single-modality counterparts, which cannot losslessly capture geometric synergy; and (2) valid constructions act as entropy reducers, strongly correlating with reduced reasoning perplexity. Building on these findings, we propose Action Applicability Policy Optimization (A2PO), a reinforcement learning paradigm for mastering strategic construction. A2PO employs Adaptive Reward Shaping to regulate the timing and quality of visual aids via counterfactual sampling to distinguish necessary from redundant constructions. Experiments demonstrate our approach enables MLLMs to leverage selective auxiliary constructions, yielding a 3.51% gain over strong baselines. Code and data are available on GitHub.

Figures & tables

Figure 1: A representative sample from GeoAux-Bench. The solution trajectory is structured in a visual-text interleaved format : the textual auxiliary construction step (e.g., <aux>... </aux> ) is explicitly paired with a corresponding auxiliary diagram ( $I_{aux}$ ).

Panel of Figure 2 — Figure 2: Overview of GeoAux-Bench.

Figure 3: Visual Enhancement Protocol. Bold red lines are used to highlight auxiliary elements ( $I_{\text{aux}}^{\text{{\color[rgb]{0.5469,0,0}red}}}$ ).

Setting	Input Modality	Acc% ( $\Delta$ )	Tokens	PPL
G1: Text-Only Settings
Standard	$\mathcal{Q},I_{orig}$	23.68	1336.2	1.1250
Text-Only	$+T_{aux}$	24.24 (+0.56)	1241.3	1.1185
G2: Visual-Only Settings
Visual-Only	$+I_{aux}$	23.34 (-0.34)	1258.0	1.1340
Enhanced Visual	$+I_{\text{aux}}^{\text{{\color[rgb]{0.5469,0,0}red}}}$	24.44 (+0.76)	1303.6	1.1310

Table 1: Reasoning Performance under Different Modal Input Combinations. Experiments are grouped by context structure for fair intra-group comparisons.

Figure 4: The framework of Action Applicability Policy Optimization (A 2 PO). The upper panel shows standard GRPO, while the lower panel illustrates our tri-partition sampling and adaptive reward shaping mechanism.

Closed-source MLMMs
Model	Think	Curriculum		Olympiad		Total
Model	Think	Senior	Junior	Senior	Junior	Total
Gemini-2.5-Pro	✓	83.91	84.75	82.28	88.24	83.16
Gemini-2.5-Flash	✓	80.10	81.68	64.56	82.13	79.53
GPT-5	✓	71.48	84.02	79.32	88.41	80.62
GPT-4o	✗	19.57	24.29	57.38	42.51	28.13

Table 2: Comparison on GeoAux-Bench-Core . Best closed and open scores are highlighted. “Ins.” and “Thk.” denote Instruct and Thinking models.

Method	GeoAux	Geomverse	Geometry3k	Avg.	GeoAux
Method	Acc%	Acc%	Acc%	Acc%	PPL $\downarrow$
Qwen2.5-VL-3B-Instruct
SFT	23.09	56.20	39.40	39.56	1.1389
GRPO	31.22	59.10	50.72	47.01	1.1550
ToRL	28.68	58.40	47.06	44.71	1.1558
GeometryZero	29.33	57.00	52.72	46.35	1.1535

Table 3: Main results on GeoAux and external benchmarks. Best and Second Best results are highlighted.

Base Model	Method	LR	TR	QR	Vis	Acc%
Qwen2.5-7B-Instruct	GRPO	✓	✗	✗	✗	39.28
	GRPO(w/o LR)	✗	✗	✗	✗	39.52
	A 2 PO (w/o TR, w/o Vis)	✗	✓	✗	✗	40.18
	A 2 PO (w/o Vis)	✗	✓	✓	✗	41.17
	A 2 PO	✗	✓	✓	✓	42.97

Table 4: Ablation study of A 2 PO components, including Length Reward (LR), Timing Reward (TR), Quality Reward (QR), and Visual Re-prompting (Vis). Accuracy results are based on GeoAux-Bench.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5: Attentional heatmaps of Qwen2.5-VL-7B-Instruct. Warmer colors indicate higher attention levels.

Statistic	GeoAux-Core					GeoAux-Canvas	Total
	Curriculum		Olympiad		Subtotal
	Senior	Junior	Senior	Junior	Subtotal
Problems	516	722	234	207	1,679	2,655	4,334
Questions	647	1,288	237	207	2,379	4,144	6,523
Problem Diagrams	422	1,097	207	206	1,932	1,693	3,625
Solution Diagrams	505	892	205	199	1,801	3,044	4,845

Appendix

Table 5: Detailed composition of GeoAux-Bench . The table breaks down the dataset by subset source, difficulty level, and data modality counts.

Closed-source MLLMs
Model	Think	Alg.	Ana.	Calc.	Plane	Solid	Stats.	Trans.	Trig.	Total
Model	Think	(Algebra)	(Geom.)	(Vector)	(Geom.)	(Geom.)		(Geom.)		Total
Gemini-2.5-Pro	✓	68.0	59.2	60.2	54.8	48.7	64.5	58.5	69.9	47.9
Gemini-2.5-Flash	✓	63.2	56.5	54.6	40.7	40.7	61.1	46.8	64.6	39.3
Gemini-2.0-Flash	✗	39.1	32.6	38.9	31.1	25.6	51.4	28.1	38.0	21.2
GPT-4.1	✗	40.4	30.7	37.1	24.1	25.1	54.0	21.5	42.5	19.0

Appendix

Table 6: Detailed performance breakdown on the GeoAux-Bench-Canvas subset. Results are cited from Shi et al. (2025) . Best scores in closed and open categories are highlighted.

Figure 6: Failure Cases of MathCanvas-7B with Visual Hallucinations.

Figure 7: An example where the model forces a coordinate system on a geometry problem, leading to unnecessary complexity.

Figure 8: Standard prompt template (used for $O$ , $O^{+}$ subsets and evaluation).

Figure 9: Prompt template for the Prohibited Subset ( $O^{-}$ ).

Figure 10: Prompt template for visual-injected re-prompting.

Figure 11: Prompt template for correctness judgment (used in evaluation and reward calculation).

Figure 12: Prompt template for auxiliary construction verification.

Hyperparameter	SFT (Warm-up)	A 2 PO (RL)
Base Model	Qwen2.5-VL-7B-Instruct
Precision	bfloat16
Optimizer	AdamW
Learning Rate	5e-5	1e-6
LR Scheduler	Cosine	Constant
Warm-up Ratio	0.1	0.0

Appendix

Table 7: Training Hyperparameters. Comparison between the SFT warm-up stage and the A 2 PO RL stage.

Reward Weights
Component	Symbol	Value
Accuracy Weight	$w_{acc}$	0.70
Format Weight	$w_{fmt}$	0.00
Timing Weight	$w_{time}$	0.15
Quality Weight	$w_{qual}$	0.15
Thresholds & Margins

Appendix

Table 8: A 2 PO Reward Coefficients.

Accuracy Reward Model
Parameter	Training Rollout	Evaluation
Temperature	1.0	0.0 (Greedy)
Top-p	1.0	0.0
Repetition Penalty	1.05	1.08
Max New Tokens	8,192	8,192
Verifier Model	Qwen3-30B-A3B-Thinking-2507

Appendix

Table 9: Generation & Verifier Configurations. Settings for training rollouts, the auxiliary verifier, and final evaluation.

Stage	Data Source / Subset	Count
Phase 1: SFT Warm-up (Mixed-Prompt)
	GeoAux-Bench (Train Split)	1,000
	GeomVerse (Train Subset)	300
	Geometry3k (Train Subset)	300
Phase 2: A 2 PO Training (Solvability Filtered)
	GeoAux-Bench (Train Split)	1,500

Appendix

Table 10: Data Statistics and Splits. Detailed breakdown of sample counts. Note that for SFT and RL phases, we utilize subsets from the training splits of external datasets to avoid contamination.

Explore similar work

LatentGeo: Learnable Auxiliary Constructions in Latent Space for Multimodal Geometric Reasoning
Mar 12, 2026Haiying Xu, Zihan Wang, Song Dai +3Multimodal Large Language Models Multimodal Reasoning
The Hong Kong University of Science and Technology (Guangzhou) · Nankai University · Communication University of China
How RL Unlocks the Aha Moment in Geometric Interleaved Reasoning
Mar 1, 2026Xiangxiang Zhang, Caijun Jia, Siyuan Li +9Visually Grounded Reasoning RL for Language Model Reasoning
ByteDance · Shenyang Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Science +2
MetaReason: Precise Interleaved Multimodal Reasoning via Editing Meta Information for Solving Geometry Problems
Aug 15, 2026Penghao Yin, Haomin Wang, Qihong Tang +3Multimodal Reasoning Mathematical Reasoning
Tsinghua University · Shanghai AI Laboratory · Shanghai Jiao Tong University +1

Mar 12, 2026J/K move · Enter open · S save

LatentGeo: Learnable Auxiliary Constructions in Latent Space for Multimodal Geometric Reasoning

Haiying Xu, Zihan Wang, Song Dai, Zhengxuan Zhang+2

The Hong Kong University of Science and Technology (Guangzhou) · Nankai University · Communication University of China

Multimodal Large Language Models Multimodal Reasoning Latent Visual Reasoning Geometric Representation Learning Mathematical Reasoning Benchmarks RL for Language Model Reasoning

Open paper AskPDF arXiv

Despite recent advances in multimodal reasoning, representing auxiliary geometric constructions remains a fundamental challenge for multimodal large language models (MLLMs). Such constructions are absent from the original diagram and must be introduced before theorems apply. Existing approaches predominantly rely on explicit construction paradigms, including text-based geometric specification, visual-token interleaving during reasoning, and tool-augmented geometric execution. However, these methods either fail to faithfully represent complex spatial relationships, incur representation mismatch between discrete symbols and continuous geometric structures, or rely on external capabilities that hinder end-to-end optimization. To address these limitations, we propose LatentGeo, a framework that learns continuous latent visual representations to internalize auxiliary geometric constructions without pixel-level rendering or external executors. We design a three-stage curriculum that progressively aligns and internalizes these latent representations through auxiliary visual supervision, followed by LaGDPO, a latent-aware reinforcement learning procedure that stabilizes latent representations during policy optimization while improving end-task correctness. To systematically evaluate construction-centric representation quality, we introduce GeoAux, a new benchmark targeting visually dependent geometry problems, and conduct experiments on GeoAux and MathVerse. Results show that LatentGeo achieves substantial gains on geometric reasoning tasks, particularly those requiring auxiliary constructions. Extensive analyses and ablation studies further validate the effectiveness of each component in our framework.

Abstract

Explore similar work

LatentGeo: Learnable Auxiliary Constructions in Latent Space for Multimodal Geometric Reasoning

How RL Unlocks the Aha Moment in Geometric Interleaved Reasoning

MetaReason: Precise Interleaved Multimodal Reasoning via Editing Meta Information for Solving Geometry Problems