TriGeoBench: A Multimodal Mathematical Geometry Benchmark with Figures and TikZ

Jing Wang; Yitong Yang; Hao Dou; Yaomin Wu; Shuting He; Tingfei Shen; Jia Wu; Jiaqi Li

cs.AIOct 20, 2026

TriGeoBench: A Multimodal Mathematical Geometry Benchmark with Figures and TikZ

Authors: Jing Wang, Yitong Yang, Hao Dou, Yaomin Wu, Shuting He, Tingfei Shen, Jia Wu, Jiaqi Li,

Organizations: School of Computing and Artificial Intelligence, Shanghai University of Finance and Economics, Shanghai, China · School of Information Management and Engineering, Shanghai University of Finance and Economics, Shanghai, China

Abstract

Multimodal Large Language Models show strong vision-language capabilities, yet their geometry-centric reasoning remains insufficiently understood. Existing mathematical benchmarks typically rely on rasterized figures and cover limited geometric knowledge areas and response formats, making it difficult to assess whether models reason over geometric structures or exploit superficial visual patterns. We introduce TriGeoBench, a geometry-centric multimodal benchmark constructed from Chinese middle- and high-school mathematics examinations. TriGeoBench contains 7,000 question-answer pairs and 12,235 geometric figures with TikZ code, spanning plane geometry, solid geometry, analytic geometry, function, and vector geometry across diverse response formats. Each instance forms a {question-answer pair, image, TikZ} triplet: question-answer pairs are stored in LaTeX, images as RGB figures, and TikZ preserves geometric topology, symbolic annotations, and relational structure. We evaluate representative open- and closed-source MLLMs through image-to-TikZ and multimodal reasoning probes. Results reveal persistent fine-grained structure and image-text alignment challenges. Oracle experiments with human-curated reference TikZ suggest that accurate executable representations provide complementary geometric cues; they do not evaluate an end-to-end conversion pipeline. TriGeoBench offers a testbed for structure-aware mathematical reasoning.

Figures & tables

Figure 1: TriGeoBench examples at Easy, Medium, and Hard difficulty levels, with aligned problem text, figures, editable TikZ code, and solutions. Appendix G provides English translations and romanizations.

Panel of Table 1 — Table 1: Comparison with mathematical-reasoning and image-to- TikZ datasets. DaTikZ v1 and v3 are shown because they are the releases used by the AutomaTikZ and TikZero baselines, respectively; v2 is an intermediate release. Ann. : A denotes automatic and M manual annotation. Knowledge : Fn., function; Vec, vector geometry; PG, plane geometry; SG, solid geometry; AG, analytic geometry.

Figure 2: Detailed statistics of the TriGeoBench dataset. In (c) and (d), Text denotes the question–answer pair, while TikZ denotes the code corresponding to the image.

Panel of Figure 3 — Figure 3: Human accuracy of different models on Image-to- TikZ tasks across difficulty levels.

Model	Gen Acc $\uparrow$	Image Metrics				TikZ Metrics			Human Acc $\uparrow$
Model	Gen Acc $\uparrow$	SSIM $\times 10^{2}\!\uparrow$	KID $\times 10^{3}\!\downarrow$	CLIP $\times 10^{2}\!\uparrow$	DINO $\times 10^{2}\!\uparrow$	cBLEU $\uparrow$	MTE $\times 10^{2}\!\uparrow$	TED $\downarrow$	Human Acc $\uparrow$
Open-Source Models
Llama-3.2-90B-Vision-Instruct	80.5%	77.54	30.9	77.86	54.61	35.27	64.22	74.67	1.7%
GLM-4.5V-108B	76.4%	75.45	4.0	94.54	87.17	45.91	76.37	6.55	7.5%
Qwen2.5-VL-72B-Instruct	93.5%	80.66	67.8	70.47	41.60	50.39	76.52	7.26	12.8%
LLaVA-OneVision-72B	68.8%	89.03	96.3	81.56	66.62	22.39	50.61	56.60	4.1%

Table 2: TriGeoBench results. Green shading marks the best in each column; bold marks the second-best.

Model	Stage	Gen Acc $\uparrow$	Image Metrics				TikZ Metrics			Human Acc $\uparrow$
			SSIM $\times 10^{2}\!\uparrow$	KID $\times 10^{3}\!\downarrow$	CLIP $\times 10^{2}\!\uparrow$	DINO $\times 10^{2}\!\uparrow$	cBLEU $\uparrow$	MTE $\times 10^{2}\!\uparrow$	TED $\downarrow$
Qwen3-VL-2B	Base model	77.7%	86.95	64.0	85.82	73.26	38.34	61.99	228.24	8.8%
	LoRA-SFT	72.7%	77.35	2.9	95.53	88.01	69.39	85.67	4.26	12.0%
Qwen3-VL-4B	Base model	80.1%	80.63	4.4	94.32	86.53	42.58	65.88	58.52	5.4%
	LoRA-SFT	71.6%	82.49	2.5	95.69	89.25	69.71	85.88	4.11	13.4%
Qwen3-VL-8B	Base model	60.1%	81.26	3.8	95.38	88.27	40.91	64.12	75.16	0.6%

Table 3: Results for six small-scale MLLMs before and after fine-tuning. Bold marks the better result within a model pair; green shading marks the best result in a column.

Model	Text-only				Text+Figure				Text+ TikZ				Text+Figure+ TikZ
Model	Easy	Med.	Hard	Avg.	Easy	Med.	Hard	Avg.	Easy	Med.	Hard	Avg.	Easy	Med.	Hard	Avg.
Open-Source Models
Qwen2.5-VL-72B	33.29	28.05	28.81	30.67	37.39	28.88	28.02	32.79	38.57	29.73	28.98	33.90	41.10	30.98	22.99	34.94
GLM-4.5V-108B	49.31	38.17	34.81	43.13	48.96	38.37	39.30	43.47	51.05	39.51	29.67	43.79	52.14	40.06	33.15	44.98
Llama-3.2-90B-Vision	13.04	14.57	10.38	13.33	12.78	14.67	10.64	13.26	14.57	13.42	16.39	14.32	16.88	14.24	12.97	15.34
LLaVA-OneVision-72B	21.88	17.11	16.09	18.95	22.25	19.19	17.61	20.03	21.37	22.46	16.58	19.60	24.29	14.77	17.87	20.59

Table 4: Reasoning accuracy (%) across input settings. Green marks the best result per column; bold marks the second-best. Instruct suffixes are omitted; full names appear in Table 2 .

Panel of Figure 4 — Figure 4: Oracle effect of human-curated reference TikZ : (a) Text-only versus Text+ TikZ ; (b) point differences over Text-only and Text+Figure; and (c–d) differences by knowledge point and question type.

Benchmark	Text+Fig.	Text+ TikZ	Gain
MathVerse	57	70	+13
MATH-Vision	69	78	+9
VisioMath	81	84	+3

Table 5: Exploratory Seed-2.0-Pro accuracy (%) on 100 samples per benchmark. TikZ is human-authored and verified; Gain is Text+ TikZ minus Text+Figure.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5: Annotation and review interfaces for side-by-side comparison, L a T e X / TikZ editing, and independent verification. English translations and Pinyin of interface labels appear in Appendix G.4 .

Panel of Figure 6 — Figure 6: Average generated token count across difficulty levels and input formats.

Panel of Figure 7 — Figure 7: Agreement between the LLM-based evaluator and human audit results. Left: normalized score-level correlation between human scores and LLM-judge scores. Right: exact agreement and within-one-point agreement across question types, with 95% bootstrap confidence intervals.

Mathematical Reasoning Benchmarks

Mathematical reasoning benchmarks test whether AI systems can solve, explain, and verify problems across mathematical levels, formats, and languages.

See all 114 papers

GeoBuildBench: A Benchmark for Interactive and Executable Geometry Construction from Natural Language
May 13, 2026Jinwoong Kim, Rui Yang, Huishuai ZhangMathematical Reasoning Benchmarks LLM Grounding
Peking University · Wangxuan Institute of Computer Technology
Synthesizing Multimodal Geometry Datasets from Scratch and Enabling Visual Alignment via Plotting Code
Feb 21, 2026Haobo Lin, Tianyi Bai, Chen Chen +4Multimodal Reasoning Synthetic Data Generation
HKUST · JLU · USTC +1
GeoChallenge: A Multi-Answer Multiple-Choice Benchmark for Geometric Reasoning with Diagrams
Feb 25, 2026Yushun Zhang, Weiping Fu, Zesheng Yang +6Mathematical Reasoning Benchmarks LLM Mathematical Reasoning
School of Computer Science and Technology, Xi’an Jiaotong University, Xi’an, China · Shaanxi Province Key Laboratory of Big Data Knowledge Engineering, Xi’an, China

May 13, 2026J/K move · Enter open · S save

GeoBuildBench: A Benchmark for Interactive and Executable Geometry Construction from Natural Language

Jinwoong Kim, Rui Yang, Huishuai Zhang

Peking University · Wangxuan Institute of Computer Technology

Mathematical Reasoning Benchmarks LLM Grounding Benchmark Construction LLM-Based Program Synthesis

Open paper AskPDF arXiv

We introduce GeoBuildBench, a benchmark designed to evaluate whether large language models and multimodal agents can ground informal natural-language plane geometry problems into executable geometric constructions. Unlike existing geometry benchmarks that focus on answer correctness or static diagram interpretation, GeoBuildBench treats geometry diagram as an interactive construction task: given a textual problem, an agent must generate a domain-specific language (DSL) program to produce a diagram satisfying explicitly specified geometric objects and verifiable constraints. The benchmark features 489 Chinese textbook-style problems, curated through automated filtering and human validation to ensure text-complete, constructible problem specifications. We evaluate several state-of-the-art multimodal models in a bounded iterative setting and show that, despite reasonable success rates, models frequently exhibit structural hallucinations, missing objects, and failures to satisfy geometric constraints, with limited ability to exploit visual and constraint-based feedback for self-correction. These results highlight geometry construction as a rigorous testbed for grounded, executable reasoning beyond textual or visual plausibility. Our benchmark and code are publicly available.

Abstract

Mathematical Reasoning Benchmarks

Explore similar work

GeoBuildBench: A Benchmark for Interactive and Executable Geometry Construction from Natural Language

Synthesizing Multimodal Geometry Datasets from Scratch and Enabling Visual Alignment via Plotting Code

GeoChallenge: A Multi-Answer Multiple-Choice Benchmark for Geometric Reasoning with Diagrams