Reconstructing conformal field theoretical compositions with Transformers

Haotian Cao; Garrett Merz; Kyle Cranmer; Gary Shiu

hep-thMay 1, 2026

Reconstructing conformal field theoretical compositions with Transformers

Authors: Haotian Cao, Garrett Merz, Kyle Cranmer, Gary Shiu

Organizations: a Department of Physics, University of Wisconsin-Madison · b Data Science Institute, University of Wisconsin-Madison

Abstract

We study the use of transformers to reconstruct the compositions of tensor products of two-dimensional rational conformal field theories (RCFTs) based on their low-energy spectra. The task is challenging due to its combinatorial nature. The constituent theories are characterized by their central charges and affine Lie algebra labels. We achieve 98% accuracy in recovering the constituents of tensor products theories constructed from Wess-Zumino-Witten models. We further demonstrate that our method generalizes to CFTs with larger central charge and unseen classes of RCFTs by adding a small number of out-of-domain examples. Our results show that transformers are effective at this task and point towards a new tool for bulk reconstruction in AdS/CFT.

Explore similar work

Length Generalization for Transformers via Compression
Sep 8, 2026Georg Zetzsche, Hongjian Jiang, Andy Yang +4Transformer Architectures Generalization Bounds
Fixed Universal Transformers
May 29, 2026Jingwen Liu, Alexandr Andoni, Daniel HsuTransformer Architectures Unified Transformer
Generating Special Triangulations with Transformers
Jun 25, 2026Charles Arnal, Jacky H. T. Yip, François Charton +1Triangulations Polytopes

Sep 8, 2026 · cs.LGJ/K move · Enter open · S save

Length Generalization for Transformers via Compression

Georg Zetzsche, Hongjian Jiang, Andy Yang, Pascal Bergsträßer+3

Max Planck Institute for Software Systems (MPI-SWS) · RPTU Kaiserslautern-Landau · University of Notre Dame

Recent advancements in transformer length generalization theory enable us to reliably predict when a transformer can learn to solve a task. In particular, the C-RASP hypothesis (a formalized version of the so-called RASP-l conjecture) posits that transformers length-generalize on a task if and only if a solution is expressible in the C-RASP language. While this hypothesis has strong empirical validation, theoretical problems arise from the fact that no computable length generalization bounds exist for C-RASP, alongside the discovery of seemingly contradictory experiments. To address these problems, we refine the C-RASP hypothesis utilizing the recently-proposed fragments C-RASP+ and C-RASP1. These fragments have computable length generalization bounds, though in the worst case requiring an extremely large (double exponential) sample size. It is an open question whether these sample size bounds are tight. In this paper, we resolve this open question by providing an exponentially tighter bound. In doing so, we show a polynomial length generalization bound for transformers if we adopt compressed strings, via a novel connection to power words. As an application, we show how this yields a fine-grained analysis of the C-RASP conjecture that resolves contradicting experimental evidence against it.

Transformer Architectures Generalization Bounds Structural Generalization Large Models Conjecture Sample Size

Open paper Ask about it PDF

Abstract

Explore similar work

Length Generalization for Transformers via Compression

Fixed Universal Transformers

Generating Special Triangulations with Transformers