Understanding how (multimodal) large language models perform on physics problems requires benchmarks that reflect the difficulty and breadth of expert-level physical reasoning. Existing physics benchmarks remain limited in the following two important ways: (1) short of high-difficulty datasets, and (2) lack of comprehensive coverage of visual forms, knowledge points, and step-by-step solution processes. As a result, model performance on current datasets may not be fully representative of their ability to solve complex physics problems. To address these issues, we present PhysElite, a large-scale bilingual multimodal benchmark for Olympiad-level physics reasoning. PhysElite contains 11,586 Olympiad-tier problems. For each problem, we provide corresponding visual diagrams, step-by-step bilingual Chinese-English solution derivations, and the final answer. We benchmark 18 open-source and closed-source MLLMs, and find that even the strongest model reaches only 33.7% answer accuracy. We additionally conduct step-level process evaluation to diagnose where models fail in the reasoning chain. Our datasets are released at https://huggingface.co/datasets/physelite/PhysElite.
Figures & tables
Figure 1: Comparision of two existing K-12 level samples and one illustrative example of olympiad-level physics problem with detailed multimodal annotation of solution steps in our PhysElite (The cooresponding completed example can be found in Figure 13 )
Figure 2: Performance of four state-of-the-art MLLMs on our PhysElite and three representative existing physics benchmarks: while prior benchmarks are nearing saturation, PhysElite leaves a substantial gap between the best model and the human reference.
Dataset
Year
Number
Diagrams
Solution Step
Analysis Process
Language
K-12 level
SciBench [ 13 ]
2024
295
✓
✗
✗
EN
MMMU [ 12 ]
2024
983
✓
✗
✗
EN
PhysicsArena [ 14 ]
2025
5,103
✓
✗
✗
EN
PhysUniBench [ 2 ]
2025
3,304
✓
✓ (text only)
✗
EN/CN
PHYSICS [ 15 ]
2025
5,334
✗
✗
✗
EN/CN
Table 1: Comparison of PhysElite with existing physics reasoning benchmarks. PhysElite is the largest Olympiad-level physics benchmark with bilingual support, detailed process-level annotations.
Figure 4
Figure 4: Pipeline of PhysElite data annotation and a sample problem
Model
Mech.
E&M
Mod.
Therm.
Opt.
Ans. ↑
Proc. ↑
Closed-source
Grok-4.2 [ 23 ] †
34.0
26.6
33.0
35.4
25.4
33.7
47.6
Claude-Opus-4.6 [ 20 ]
27.2
20.9
24.7
33.1
21.3
28.1
49.6
o3-mini [ 24 ] † ⋆
25.3
20.4
26.8
28.4
19.6
26.2
43.9
GPT-5.2 [ 22 ] †
22.1
18.6
23.9
28.4
17.4
24.0
42.6
Gemini-3-Pro [ 21 ] †
21.9
15.5
20.0
23.7
16.1
21.8
41.4
Table 3: Main results on PhysElite , averaged across Chinese and English evaluations. Ans. : answer-level accuracy (%). Mech. : Mechanics. E&M : Electromagnetism. Mod. : Modern Physics. Therm. : Thermodynamics. Opt. : Optics. Proc. : process score of step-level evaluation. Best per column in bold , second best underlined .
Metric
GPT-5.2
Opus-4.6
Sonnet-4.5
Gemini-2.5
Qwen3-VL-235B
Qwen3-VL-32B
Process Diagram
Answer
26.7 (+2.7)
28.4 (+0.3)
19.1 (+0.6)
19.8 (+1.9)
22.5 (+2.5)
18.5 (+2.6)
Process
46.0 (+3.4)
51.3 (+1.7)
35.9 (+2.3)
40.6 (+3.8)
40.9 (+4.1)
37.0 (+5.5)
Solution Hint
Answer
23.5 (-0.5)
28.4 (+0.3)
18.5 (+0.0)
17.9 (+0.0)
19.9 (-0.1)
15.7 (-0.2)
Process
42.8 (+0.2)
50.0 (+0.4)
37.0 (+2.3)
36.7 (-0.1)
36.8 (+0.0)
32.4 (+0.9)
Table 4: Effect of extra information on selected models.
Figure 5: Error distribution of Claude Opus 4.6 and an example of physical law error.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: The distribution of the length per problem in PhysElite .
Type
Count / Share
Description
Easy (1–3)
28%
Shorter or more direct derivations
Medium (4–5)
55%
Multi-step Olympiad-style reasoning
Hard (6–7)
17%
Long derivations or advanced modeling
Symbolic expressions
3,700
Closed-form symbolic answers
Numerical values
2,810
Numeric answers with units or constants
Qualitative conclusions
978
Physical judgments or comparisons
Appendix
Table 5: Difficulty and answer-type distribution of PhysElite .
Figure 7: The distribution of the number of images per question in PhysElite .
Figure 8: A sample of Mechanics Problem
Figure 9: A sample of Electromagnetism Problem
Figure 10: A sample of Optics Problem
Figure 11: A sample of Thermodynamics Problem
Figure 12: A sample of Modern Physics Problem
Figure 13: A complete sample problem of our Dataset
Group
Model
Reasoning Mode
Input Modality
Closed-source
Grok-4.2
System-2
Text+image
o3-mini
System-2
Text-only
GPT-5.2
System-2
Text+image
Gemini-3-Pro
System-2
Text+image
Kimi-K2-Thinking
System-2
Text+image
Gemini-2.5
System-2
Text+image
Appendix
Table 6: Model grouping used in the evaluation.
Error Type
Explanation
Physical-perception error
The model misreads the physical setup, diagram, variables, constraints, or stated conditions.
Physical-law error
The model selects an inappropriate physical principle or applies a law outside its valid conditions.
Algebraic/symbolic error
The physical setup is mostly correct, but the derivation contains symbolic manipulation, sign, or formula errors.
Incomplete derivation
The model starts correctly but stops before all required quantities or sub-cases are solved.
Geometric/visual error
The model makes an incorrect geometric inference from a diagram, configuration, angle, path, or spatial relation.
Numerical mistake
The model makes an arithmetic, substitution, or unit-conversion error after the correct setup is available.