LLMs Can Tackle High Energy Theoretical Physics, yet Gaps Remain: Part I, Quantum Field Theory

Tong Liu; Bohan Li; Gongrui Cheng; Zixiong Yu; Yulan He; Volker Tresp

hep-phOct 23, 2026

LLMs Can Tackle High Energy Theoretical Physics, yet Gaps Remain: Part I, Quantum Field Theory

Authors: Tong Liu, Bohan Li, Gongrui Cheng, Zixiong Yu, Yulan He, Volker Tresp

Organizations: Ludwig-Maximilians-Universität München, München, Germany · Munich Center for Machine Learning · Okinawa Institute of Science and Technology Graduate University, Okinawa, Japan · Instituut voor Theoretische Fysica, K.U. Leuven, Leuven, Belgium ·

Abstract

Beyond math and code, Large language models (LLMs) are beginning to tackle problems that have traditionally required substantial training for theoretical physicists, but it remains unclear how far this capability extends and where it might break. As a first step toward pushing LLMs in high-energy theoretical physics, we introduce QFT-Bench, a benchmark of 75 graduate- to research-level problems requiring extensive reasoning and derivation across core Quantum Field Theory (QFT) subareas. Each question has a well-defined target and closed-form reference answer. Evaluation combines symbolic verification with LLM-based judge. Across 39 proprietary and open-source models, frontier models already demonstrate substantial capabilities on nontrivial QFT calculations, with the strongest models exceeding 95% accuracy. At the same time, clear gaps remain. Even strong models still make errors in calculations, conventions, and instruction following, while weaker models often fail at the level of physical concepts or derivations. We further monitor models' CoTs and show that current models exhibit nontrivial reasoning unfaithfulness, i.e., they could reach correct answers but the reasoning itself does not justify and logically support them. These results provide a quantitative assessment of current LLM capabilities in QFT and a first step toward broader evaluation in high-energy theoretical physics.

Figures & tables

Benchmarks	# of HEP questions	Level	Physics- specialized	HEP- specialized	Evaluation
GPQA-diamond ( Rein et al., 2023 )	-	undergraduate, graduate	no	no	rule
HLE ( Phan et al., 2025 )	-	graduate, research	no	no	LLM
SciBench ( Wang et al., 2023 )	0	undergraduate	no	no	rule
PHYSICS ( Feng et al., 2025 )	0	undergraduate	yes	no	rule + LLM
TPBench ( Chung et al., 2025 )	18	undergraduate, graduate, research	yes	no	rule + LLM
CritPt ( Zhu et al., 2025 )	10	research	yes	no	rule

Table 1: Comparison of representative benchmarks across High-Energy Physics (HEP) scale, difficulty level, and physics specialization.

Areas	Problems	Percentage
Classical Field Theory (Classical)	7	9.3%
Quantization of Scalar, Spinor and Vector Fields (Quant.)	8	10.7%
Quantum Electrodynamics (QED)	7	9.3%
Regularization and Renormalization (Renorm.)	6	8.0%
Non-Abelian Gauge Theory and QCD (Non-Ab.)	6	8.0%
Electroweak and Higgs Mechanism (EW.)	8	10.7%

Table 2: Distribution of questions in QFT-Bench across eight topic categories.

	Models	API Names	Release Date
Claude	Opus 5.5	claude-opus-5-5	2026-09
	Opus 5	claude-opus-5	2026-07
	Opus 4.6	claude-opus-4-6	2026-02
	Sonnet 5.5	claude-sonnet-5-5	2026-09
	Sonnet 5	claude-sonnet-5	2026-06
	Haiku 4.5	claude-haiku-4-5	2025-10

Table 3: Evaluated models, their API names, and release dates.


	Symbolic	Clas.	EW.	Non-Ab.	QED	Quant.	Renorm.	SUSY	CFT	Avg.
Opus-5.5	72.0	100.0	100.0	100.0	85.7	100.0	100.0	100.0	100.0	98.7
Sonnet-5.5	74.7	100.0	100.0	100.0	100.0	100.0	83.3	100.0	93.3	97.3
Opus-5	64.0	100.0	100.0	100.0	85.7	100.0	83.3	100.0	100.0	97.3
Opus-4.6	73.3	85.7	75.0	100.0	85.7	75.0	66.7	88.9	93.3	85.3
Sonnet-5	66.7	85.7	87.5	83.3	85.7	87.5	66.7	72.2	93.3	82.7

Table 4: Accuracy (%) on QFT-Bench. “Symbolic” reports symbolic verification scores. “Avg.” report final accuracy. LLMs are evaluated with 64k max tokens.

Figure 2: Reasoning faithfulness vs. model capability for LLM QFT reasoning. Faithfulness is approximately linearly scaled with model capability within model families.

Failure categories	Top half	Others
Conceptual error	$23.4\%\pm 2.0\%$	$47.2\%\pm 0.9\%$
Calculation error	$21.6\%\pm 0.0\%$	$18.6\%\pm 0.7\%$
Strong-prior error	$23.4\%\pm 0.8\%$	$13.4\%\pm 0.9\%$
Instruction following	$31.5\%\pm 2.0\%$	$20.8\%\pm 0.4\%$

Table 5: Distribution of failure categories (% of failures within each group). Top half denotes the half highest-scoring models in Table 4 ; Others denotes the remaining models.

Scientific Reasoning in Language Models

Scientific reasoning in language models studies how AI systems use evidence, domain knowledge, and tools to solve problems and support discovery.

See all 144 papers

Grading the Unspoken: Evaluating Tacit Reasoning in Quantum Field Theory and String Theory with LLMs
Apr 1, 2026Xingyang Yu, Yinghuan Zhang, Yufei Zhang +1Scientific Reasoning in Language Models LLM Evaluation
Department of physics, Virginia Tech · Independent Researcher · Department of Computer Science and Engineering, Michigan State University
Fine-Tuning Small Reasoning Models for Quantum Field Theory
Apr 21, 2026Nathaniel S. Woodward, Zhiqi Gao, Yurii Kvasiuk +3Physical Reasoning LLM Fine-Tuning
Department of Physics, University of Wisconsin-Madison · Department of Computer Science, University of Wisconsin-Madison · Perimeter Institute for Theoretical Physics
Testing Frontier Large Language Models' Physics Literacy in Parallel Physical Worlds
Jun 30, 2026Dong ZhangLLM Evaluation Physical Reasoning

Apr 1, 2026J/K move · Enter open · S save

Grading the Unspoken: Evaluating Tacit Reasoning in Quantum Field Theory and String Theory with LLMs

Xingyang Yu, Yinghuan Zhang, Yufei Zhang, Zijun Cui

Department of physics, Virginia Tech · Independent Researcher · Department of Computer Science and Engineering, Michigan State University

Scientific Reasoning in Language Models LLM Evaluation Implicit Reasoning in Language Models Reasoning Consistency in Language Models

Open paper AskPDF arXiv

Large language models have demonstrated impressive performance across many domains of mathematics and physics. One natural question is whether such models can support research in highly abstract theoretical fields such as quantum field theory and string theory. Evaluating this possibility faces an immediate challenge: correctness in these domains is layered, tacit, and fundamentally non-binary. Standard answer-matching metrics fail to capture whether intermediate conceptual steps are properly reconstructed or whether implicit structural constraints are respected. We construct a compact expert-curated dataset of twelve questions spanning core areas of quantum field theory and string theory, and introduce a five-level grading rubric separating statement correctness, key concept awareness, reasoning chain presence, tacit step reconstruction, and enrichment. Evaluating multiple contemporary LLMs, we observe near-ceiling performance on explicit derivations within stable conceptual frames, but systematic degradation when tasks require reconstruction of omitted reasoning steps or reorganization of representations under global consistency constraints. These failures are driven not only by missing intermediate steps, but by an instability in representation selection: models often fail to identify the correct conceptual framing required to resolve implicit tensions. We argue that highly abstract theoretical physics provides a uniquely sensitive lens on the epistemic limits of current evaluation paradigms.

Abstract

Scientific Reasoning in Language Models

Explore similar work

Grading the Unspoken: Evaluating Tacit Reasoning in Quantum Field Theory and String Theory with LLMs

Fine-Tuning Small Reasoning Models for Quantum Field Theory

Testing Frontier Large Language Models' Physics Literacy in Parallel Physical Worlds