We present StringQFT as a textbook collection for string constructions of quantum field theory and as infrastructure for AI-assisted theoretical physics. The collection comprises 18 volumes, 190 chapters and 3,795 pages, covering brane probes, brane webs, geometric engineering, F-theory constructions, wrapped M5-branes and exact worldsheet constructions. Its open-book benchmark inventory contains 1,073 items, including 499 with automatic finite-answer checking. For retrieval-augmented generation, it provides 1,371 section records linked to textbook sources and PDF locations, together with a local lexical retriever. Training-data preparation is illustrated by 21 private, source-linked examples. For agents, it exposes 19 calculation tools with specified inputs and finite domains, connecting tool use to textbook derivations. We illustrate these uses through specific questions, retrieved passages and finite calculations. The textbook collection and accompanying resources are available at https://github.com/xingyang-yu/StringQFT.
Figures & tables
Figure 1: The same textbook material supports four uses. Source identifiers and conventions connect questions, retrieved passages, prepared training examples and calculation tools. The amber training block denotes private prepared material; the diagram describes resources, not measured model performance.
Figure 2: Relations among string constructions. Dark links select concrete connections: T-dual probe and suspended-brane descriptions, the D4–NS5 lift to M5-branes, the relation between circle-compactified F-theory and M-theory on an elliptic geometry, and topological-string computations of protected quantities in geometric engineering. These relations require the appropriate backgrounds and limits; they do not identify entire construction families. Pale spokes indicate shared types of physical objects, not a claim that all six descriptions realize the same theory. The foundations supply the common language.
Volume
Questions
Automatic
Manual
Demo
I
28
0
28
0
II
54
43
11
0
III
54
46
8
0
IV
62
51
11
0
V
55
43
11
1
VI
38
0
38
0
Table 1: Questions and scoring modes by volume. Chapter batches aggregate these questions and do not add scoring units.
Figure 3: Question selection, submission and assessment. Blue blocks show question, submission and report objects; amber identifies private evaluation material, and dashed blocks denote participants or manual assessment. The one public worked demonstration is excluded from scoring. An automatically accepted answer does not grade its explanation.
Figure 4: Retrieval supplies a passage together with its source location. The demonstrated query searches the Volume V corpus; the 1,371-record total covers all 18 volumes. The dashed answering component is an external application; the supplied retriever itself does not generate an answer.
Figure 5: Training-example preparation and possible subsequent use. The private records pair tasks with authored explanations and source information; numerical examples have finite checks. Dashed blocks identify subsequent reuse, split design, training and evaluation that are not completed experiments in this work.
Figure 6: The deterministic Volume V client retrieves a convention, inverts exact coefficients and reconstructs the input. The dashed connection supplies the original table for comparison. This is an available sequence of operations for an external agent, not a measured autonomous reasoning loop or a test of whether the coefficients arise from a physical BPS spectrum.
We present a suite of 50 unpublished research-level theoretical physics problems with verifiable answers. Without AI assistance, a typical problem in our collection could be solved by a world expert in about a day or less but would take considerably longer for experts in adjacent specialties. We test the ability of off-the-shelf AI agents to tackle these problems autonomously, without human hints. When given access to tools available in a realistic setting, including the ability to search the literature and write scripts for numerical and symbolic work, the best-performing agent, GPT-6-Astra harnessed in Codex, obtains the final answers with a near-perfect accuracy of 98% at an average cost under USD 3 per problem. On average, agents show only small changes in accuracy when denied web search and access to background research papers, suggesting significant memorization of the literature. These results suggest that state-of-the-art agents can accurately perform concrete calculations required across theoretical physics research. We comment on the broader implications of these findings and discuss possible explanations for the current lack of agent-led breakthroughs in theoretical physics despite their outstanding problem-solving abilities.
A. Ramesh Chandra, Aaghaz Mahajan, Ainesh Sanyal +32
International Centre for Theoretical Sciences · Tata Institute of Fundamental Research · University of Texas at Austin +16
Large language models have demonstrated impressive performance across many domains of mathematics and physics. One natural question is whether such models can support research in highly abstract theoretical fields such as quantum field theory and string theory. Evaluating this possibility faces an immediate challenge: correctness in these domains is layered, tacit, and fundamentally non-binary. Standard answer-matching metrics fail to capture whether intermediate conceptual steps are properly reconstructed or whether implicit structural constraints are respected. We construct a compact expert-curated dataset of twelve questions spanning core areas of quantum field theory and string theory, and introduce a five-level grading rubric separating statement correctness, key concept awareness, reasoning chain presence, tacit step reconstruction, and enrichment. Evaluating multiple contemporary LLMs, we observe near-ceiling performance on explicit derivations within stable conceptual frames, but systematic degradation when tasks require reconstruction of omitted reasoning steps or reorganization of representations under global consistency constraints. These failures are driven not only by missing intermediate steps, but by an instability in representation selection: models often fail to identify the correct conceptual framing required to resolve implicit tensions. We argue that highly abstract theoretical physics provides a uniquely sensitive lens on the epistemic limits of current evaluation paradigms.
Xingyang Yu, Yinghuan Zhang, Yufei Zhang +1
Department of physics, Virginia Tech · Independent Researcher · Department of Computer Science and Engineering, Michigan State University
Despite the growing application of large language models (LLMs) to theoretical physics, there has been little academic exploration of how domain-specific physics reasoning ability develops during training. To investigate this, we perform the first academic comparison of reinforcement learning and supervised fine-tuning for LLMs on verifiable problem-solving tasks in theoretical physics. Because the open-source verifiable training data required to develop such capabilities is scarce, we built a robust data generation pipeline that can both create synthetic problems and make existing human-authored problems suitable for model training. Selecting quantum field theory (QFT) as our primary domain, we generated over 2,500 synthetic problems alongside a curated collection of human-adapted problems sourced from arXiv and standard pedagogical resources. We conduct both reinforcement learning (RL) and supervised fine-tuning (SFT) experiments, benchmarking performance gains as well as generalization to other physics domains. Although both methods improve over the base model on every dataset, SFT gains are concentrated on in-distribution synthetic tasks, while RL generalizes better, outperforming SFT on the human-adapted datasets and TPBench. We perform an extensive analysis of model chain-of-thought (CoT) reasoning before and after fine-tuning to understand how reasoning errors evolve during RL and SFT. Finally, we publicly release our data pipeline, verifiable QFT training data, and ~200M tokens of QFT reasoning traces.
Nathaniel S. Woodward, Zhiqi Gao, Yurii Kvasiuk +3
Department of Physics, University of Wisconsin-Madison · Department of Computer Science, University of Wisconsin-Madison · Perimeter Institute for Theoretical Physics