Despite recent advances in multimodal reasoning, representing auxiliary geometric constructions remains a fundamental challenge for multimodal large language models (MLLMs). Such constructions are absent from the original diagram and must be introduced before theorems apply. Existing approaches predominantly rely on explicit construction paradigms, including text-based geometric specification, visual-token interleaving during reasoning, and tool-augmented geometric execution. However, these methods either fail to faithfully represent complex spatial relationships, incur representation mismatch between discrete symbols and continuous geometric structures, or rely on external capabilities that hinder end-to-end optimization. To address these limitations, we propose LatentGeo, a framework that learns continuous latent visual representations to internalize auxiliary geometric constructions without pixel-level rendering or external executors. We design a three-stage curriculum that progressively aligns and internalizes these latent representations through auxiliary visual supervision, followed by LaGDPO, a latent-aware reinforcement learning procedure that stabilizes latent representations during policy optimization while improving end-task correctness. To systematically evaluate construction-centric representation quality, we introduce GeoAux, a new benchmark targeting visually dependent geometry problems, and conduct experiments on GeoAux and MathVerse. Results show that LatentGeo achieves substantial gains on geometric reasoning tasks, particularly those requiring auxiliary constructions. Extensive analyses and ablation studies further validate the effectiveness of each component in our framework.
Figures & tables
Figure 1: Motivating comparison of auxiliary-construction paradigms for geometric reasoning.
Figure 2: Overview of the LatentGeo framework.
Model
Size
Overall
Knowledge Categories (Acc %)
GeoAux Categories (Acc %)
ALL
Anal.
Eucl.
Func.
Spat.
Analy.
Angle
Circ.
Conn.
Trans.
Align.
Proj.
Part.
Closed-source MLLMs
GPT-4o-mini [ 24 ]
–
12.2
14.3
13.9
12.4
6.2
5.3
8.3
7.3
13.1
11.1
13.3
5.5
10.8
GPT-4o [ 25 ]
–
30.3
34.3
31.4
33.2
23.4
30.9
25.6
31.8
32.3
26.1
29.6
27.4
25.5
Qwen3-VL-Flash [ 2 ]
–
33.4
43.8
34.1
34.3
25.2
27.7
29.3
30.9
32.8
30.5
36.5
21.2
33.1
Open-source MLLMs
Table 1: Main Results on GeoAux Benchmark. Performance is measured by Accuracy (Acc %) on 2,228 instances. We contrast our LatentGeo with proprietary models and state-of-the-art open-source baselines. Knowledge Domains : Analytic (Anal.), Euclidean (Eucl.), Functional (Func.), and Spatial (Spat.). Visual Operations : Analytic Overlay (Analy.), Angular Construction (Angle), Circular Augmentation (Circ.), Connectivity (Conn.), Transformation (Trans.), Alignment (Align.), Projection (Proj.), and Partitioning (Part.).
Model
Size
MathVerse ( testmini )
ALL
TD
VI
VD
VO
Baselines
Random
–
12.4
12.4
12.4
12.4
12.4
Human
–
66.9
71.2
61.4
68.3
66.7
Closed-source MLLMs
GPT-4o [ 25 ]
–
50.5
59.8
48.0
46.5
47.6
Table 2: Comparison on MATHVERSE ( testmini ). We report category-wise results: Text-Dominant (TD), Vision-Intensive (VI, requiring complex visual reasoning), Vision-Dominant (VD, where visual information prevails), and Vision-Only (VO, without text prompts). Results are averaged in the ALL column. We contrast these visually grounded categories (VI, VD, VO) against the TD reference.
Model Variant
Overall
GeoAux Categories (Acc %)
ALL
Analy.
Angle
Circ.
Conn.
Trans.
Align.
Proj.
Part.
LatentGeo (TextOnly)
26.7
28.7
27.1
30.0
25.3
22.1
24.3
30.8
24.3
w/o Curric. Step 2
13.1
12.8
19.6
6.4
12.8
7.5
12.2
13.7
14.1
w/o Curric. Step 3
23.1
24.5
28.6
20.9
22.4
22.6
21.5
21.2
20.5
w/o RL Phase
32.1
22.3
34.6
27.3
32.2
22.1
29.3
37.0
23.5
w/ GRPO [ 29 ]
26.7
20.2
34.6
27.3
27.0
23.9
24.2
24.7
20.9
Table 3: Ablation Study Results. Impact of different training stages and components on overall performance (ALL) and category-wise accuracy on GeoAux visual operations. The full model demonstrates superior performance across most categories, particularly in complex constructions like Angular Construction and Spatial Projection.
Figure 3: Perturbation-based visual attribution on four geometry problems. Submodular insertion and deletion attribution maps are shown for Qwen2.5-VL-7B (top, green) and LatentGeo-7B (bottom, blue).
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Statistic
Number
Total instances
2,228
Question type
– Free-form
1,313
– Multiple-choice
915
Knowledge domain
– Analytic coordinate geometry
265
Appendix
Table 4: Distribution and taxonomy of the GeoAux benchmark. Left: taxonomy counts. Right: pie chart. Visual-operation labels are non-exclusive, so their counts may exceed the total number of instances.
Figure 4: Radar visualization on GeoAux . Left: accuracy across four knowledge domains . Right: accuracy across eight visual operations . Curves are drawn from Table 1 for representative baselines and LatentGeo .
Figure 5: Benchmark example from Euclidean Plane Geometry.
Figure 6: Benchmark example from Spatial & Projective Geometry.
Figure 7: Benchmark example from Analytic Coordinate Geometry.
Figure 8: Benchmark example from Functional Graph Geometry.
Hyperparameter
Value
Optimizer
AdamW
Learning rate
1×10−5
Weight decay
0.01
Max sequence length
1,024
Latent size
10
SFT Stage 1 epochs
5
Appendix
Table 5: SFT hyperparameters.
Hyperparameter
Value
KL coefficient β
0.03
PPO epochs
1
Clip ratio (lower) ϵl
0.2
Clip ratio (upper) ϵh
0.3
Dual clip c
3.0
Optimizer
AdamW
Appendix
Table 6: RL hyperparameters.
Figure 9: Unified instruction prompt used for SFT, RL, and inference.
Figure 10: Answer-extraction template used before final matching.
Figure 11: Judge prompt used for final correctness decision.
Figure 12: Representative SFT training example.
Figure 13: RL training trends. We report four perspectives of RL optimization: overall behavior, latent-token dynamics, length control, and repetition control.
College of Computer Science and Artificial Intelligence, Fudan University · School of ECE, Peking University · School of Software and Microelectronics, Peking University +1