Organizations: School of Computing and Artificial Intelligence, Shanghai University of Finance and Economics, Shanghai, China · School of Information Management and Engineering, Shanghai University of Finance and Economics, Shanghai, China
Multimodal Large Language Models show strong vision-language capabilities, yet their geometry-centric reasoning remains insufficiently understood. Existing mathematical benchmarks typically rely on rasterized figures and cover limited geometric knowledge areas and response formats, making it difficult to assess whether models reason over geometric structures or exploit superficial visual patterns. We introduce TriGeoBench, a geometry-centric multimodal benchmark constructed from Chinese middle- and high-school mathematics examinations. TriGeoBench contains 7,000 question-answer pairs and 12,235 geometric figures with TikZ code, spanning plane geometry, solid geometry, analytic geometry, function, and vector geometry across diverse response formats. Each instance forms a {question-answer pair, image, TikZ} triplet: question-answer pairs are stored in LaTeX, images as RGB figures, and TikZ preserves geometric topology, symbolic annotations, and relational structure. We evaluate representative open- and closed-source MLLMs through image-to-TikZ and multimodal reasoning probes. Results reveal persistent fine-grained structure and image-text alignment challenges. Oracle experiments with human-curated reference TikZ suggest that accurate executable representations provide complementary geometric cues; they do not evaluate an end-to-end conversion pipeline. TriGeoBench offers a testbed for structure-aware mathematical reasoning.
Figures & tables
Figure 1: TriGeoBench examples at Easy, Medium, and Hard difficulty levels, with aligned problem text, figures, editable TikZ code, and solutions. Appendix G provides English translations and romanizations.
Table 1: Comparison with mathematical-reasoning and image-to- TikZ datasets. DaTikZ v1 and v3 are shown because they are the releases used by the AutomaTikZ and TikZero baselines, respectively; v2 is an intermediate release. Ann. : A denotes automatic and M manual annotation. Knowledge : Fn., function; Vec, vector geometry; PG, plane geometry; SG, solid geometry; AG, analytic geometry.
Figure 2: Detailed statistics of the TriGeoBench dataset. In (c) and (d), Text denotes the question–answer pair, while TikZ denotes the code corresponding to the image.
Figure 3: Human accuracy of different models on Image-to- TikZ tasks across difficulty levels.
Model
Gen Acc ↑
Image Metrics
TikZ Metrics
Human Acc ↑
SSIM ×102↑
KID ×103↓
CLIP ×102↑
DINO ×102↑
cBLEU ↑
MTE ×102↑
TED ↓
Open-Source Models
Llama-3.2-90B-Vision-Instruct
80.5%
77.54
30.9
77.86
54.61
35.27
64.22
74.67
1.7%
GLM-4.5V-108B
76.4%
75.45
4.0
94.54
87.17
45.91
76.37
6.55
7.5%
Qwen2.5-VL-72B-Instruct
93.5%
80.66
67.8
70.47
41.60
50.39
76.52
7.26
12.8%
LLaVA-OneVision-72B
68.8%
89.03
96.3
81.56
66.62
22.39
50.61
56.60
4.1%
Table 2: TriGeoBench results. Green shading marks the best in each column; bold marks the second-best.
Model
Stage
Gen Acc ↑
Image Metrics
TikZ Metrics
Human Acc ↑
SSIM ×102↑
KID ×103↓
CLIP ×102↑
DINO ×102↑
cBLEU ↑
MTE ×102↑
TED ↓
Qwen3-VL-2B
Base model
77.7%
86.95
64.0
85.82
73.26
38.34
61.99
228.24
8.8%
LoRA-SFT
72.7%
77.35
2.9
95.53
88.01
69.39
85.67
4.26
12.0%
Qwen3-VL-4B
Base model
80.1%
80.63
4.4
94.32
86.53
42.58
65.88
58.52
5.4%
LoRA-SFT
71.6%
82.49
2.5
95.69
89.25
69.71
85.88
4.11
13.4%
Qwen3-VL-8B
Base model
60.1%
81.26
3.8
95.38
88.27
40.91
64.12
75.16
0.6%
Table 3: Results for six small-scale MLLMs before and after fine-tuning. Bold marks the better result within a model pair; green shading marks the best result in a column.
Model
Text-only
Text+Figure
Text+ TikZ
Text+Figure+ TikZ
Easy
Med.
Hard
Avg.
Easy
Med.
Hard
Avg.
Easy
Med.
Hard
Avg.
Easy
Med.
Hard
Avg.
Open-Source Models
Qwen2.5-VL-72B
33.29
28.05
28.81
30.67
37.39
28.88
28.02
32.79
38.57
29.73
28.98
33.90
41.10
30.98
22.99
34.94
GLM-4.5V-108B
49.31
38.17
34.81
43.13
48.96
38.37
39.30
43.47
51.05
39.51
29.67
43.79
52.14
40.06
33.15
44.98
Llama-3.2-90B-Vision
13.04
14.57
10.38
13.33
12.78
14.67
10.64
13.26
14.57
13.42
16.39
14.32
16.88
14.24
12.97
15.34
LLaVA-OneVision-72B
21.88
17.11
16.09
18.95
22.25
19.19
17.61
20.03
21.37
22.46
16.58
19.60
24.29
14.77
17.87
20.59
Table 4: Reasoning accuracy (%) across input settings. Green marks the best result per column; bold marks the second-best. Instruct suffixes are omitted; full names appear in Table 2 .
Figure 4: Oracle effect of human-curated reference TikZ : (a) Text-only versus Text+ TikZ ; (b) point differences over Text-only and Text+Figure; and (c–d) differences by knowledge point and question type.
Benchmark
Text+Fig.
Text+ TikZ
Gain
MathVerse
57
70
+13
MATH-Vision
69
78
+9
VisioMath
81
84
+3
Table 5: Exploratory Seed-2.0-Pro accuracy (%) on 100 samples per benchmark. TikZ is human-authored and verified; Gain is Text+ TikZ minus Text+Figure.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Annotation and review interfaces for side-by-side comparison, L a T e X / TikZ editing, and independent verification. English translations and Pinyin of interface labels appear in Appendix G.4 .
Figure 6: Average generated token count across difficulty levels and input formats.
Figure 7: Agreement between the LLM-based evaluator and human audit results. Left: normalized score-level correlation between human scores and LLM-judge scores. Right: exact agreement and within-one-point agreement across question types, with 95% bootstrap confidence intervals.
We introduce GeoBuildBench, a benchmark designed to evaluate whether large language models and multimodal agents can ground informal natural-language plane geometry problems into executable geometric constructions. Unlike existing geometry benchmarks that focus on answer correctness or static diagram interpretation, GeoBuildBench treats geometry diagram as an interactive construction task: given a textual problem, an agent must generate a domain-specific language (DSL) program to produce a diagram satisfying explicitly specified geometric objects and verifiable constraints. The benchmark features 489 Chinese textbook-style problems, curated through automated filtering and human validation to ensure text-complete, constructible problem specifications. We evaluate several state-of-the-art multimodal models in a bounded iterative setting and show that, despite reasonable success rates, models frequently exhibit structural hallucinations, missing objects, and failures to satisfy geometric constraints, with limited ability to exploit visual and constraint-based feedback for self-correction. These results highlight geometry construction as a rigorous testbed for grounded, executable reasoning beyond textual or visual plausibility. Our benchmark and code are publicly available.
Jinwoong Kim, Rui Yang, Huishuai Zhang
Peking University · Wangxuan Institute of Computer Technology
Multimodal geometry reasoning requires models to jointly understand visual diagrams and perform structured symbolic inference, yet current vision--language models struggle with complex geometric constructions due to limited training data and weak visual--symbolic alignment. We propose a pipeline for synthesizing complex multimodal geometry problems from scratch and construct a dataset named GeoCode, which decouples problem generation into symbolic seed construction, grounded instantiation with verification, and code-based diagram rendering, ensuring consistency across structure, text, reasoning, and images. Leveraging the plotting code provided in GeoCode, we further introduce code prediction as an explicit alignment objective, transforming visual understanding into a supervised structured prediction task. GeoCode exhibits substantially higher structural complexity and reasoning difficulty than existing benchmarks, while maintaining mathematical correctness through multi-stage validation. Extensive experiments show that models trained on GeoCode achieve consistent improvements on multiple geometry benchmarks, demonstrating both the effectiveness of the dataset and the proposed alignment strategy. The code will be available at https://github.com/would1920/GeoCode.
Evaluating the symbolic reasoning of large language models (LLMs) calls for geometry benchmarks that require multi-step proofs grounded in both text and diagrams. However, existing benchmarks are often limited in scale and rarely provide visually grounded multiple-choice questions, limiting reliable evaluation of complex reasoning. We introduce GeoChallenge, a dataset of 90K automatically generated multiple-choice geometry proof problems, each requiring multi-step reasoning over aligned textual descriptions and diagrams. GeoChallenge provides fine-grained complexity ratings and formal language annotations to enable controlled evaluation. Experiments on multiple advanced LLMs show a clear performance gap between models and humans (the best-performing model, GPT-5-nano, achieves 75.89 exact match vs. 94.74 for humans). Further analysis also reveals three common failure patterns of LLMs: (1) exact match failures under the multiple-choice setting; (2) weak visual reliance; and (3) overextended reasoning without convergence.
Yushun Zhang, Weiping Fu, Zesheng Yang +6
School of Computer Science and Technology, Xi’an Jiaotong University, Xi’an, China · Shaanxi Province Key Laboratory of Big Data Knowledge Engineering, Xi’an, China