math.GTNov 5, 2026

Using Transformers to Generate 3-Manifold Triangulations

Authors: Daniel Bruwel, Ashvni Narayanan, Jonathan Spreer

Organizations: School of Mathematics and Statistics F07, The University of Sydney, NSW 2006 Australia · Department of Computer Science, Technion, Haifa, 3200003, Israel

Abstract

Using dehydration signatures as an encoding, we train GPT-2-style transformers to generate triangulations of 3-dimensional manifolds. Our main runs each use approximately 3 exaFLOPs, spanning a 200 million parameter model trained on 60 million sphere triangulations to an 11 million parameter model trained on 1.1 billion sphere triangulations. The optimal model has 10 blocks, width 640 and generates dehydration signatures of closed manifolds of various topological types at a rate of 6.9%, 82% of which are spheres. To investigate how our method scales, we use the technique of IsoFLOP curves on 15-tetrahedron triangulations of the 3-sphere, over a compute range of 6 to 3000 petaFLOPs, and observe neural scaling behaviour. We extract a parameter scaling exponent of α=0.49α= 0.49 ([0.48,0.50][0.48, 0.50] stat, ±0.02\pm 0.02 syst), and an optimal ratio of 234 tokens per parameter for training. In addition, we present a number of smaller experiments, including evidence that transformers are able to learn to count degree-one edges near-perfectly, but get progressively worse when counting edges with higher degrees.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Jun 25, 2026

Generating Special Triangulations with Transformers

Triangulations, i.e., well-structured decompositions of geometric objects into triangle-like pieces, are central objects in many domains of mathematics and physics. In particular, fine, regular, and star triangulations (FRSTs) of 4D reflexive polytopes give rise to smooth Calabi-Yau threefolds, which are of significant interest in string theory. However, the high dimensionality and combinatorial complexity of triangulations make them particularly challenging to model with classical numerical methods or machine learning. In this work, we show that transformers, equipped with an appropriate encoding scheme, can be effectively trained to representatively generate new FRSTs across a range of polytope sizes. Moreover, these models can also self-improve through retraining on their own output. This opens the door to both concrete applications to the classification of Calabi-Yau manifolds and further research in physics, combinatorics and algebraic geometry.
May 7, 2026

No Triangulation Without Representation: Generalization in Topological Deep Learning

Despite an ever-increasing interest in topological deep learning models that target higher-order datasets, there is no consensus on how to evaluate such models. This is exacerbated by the fact that topological objects permit operations, such as structural refinements, that are not appropriate for graph data. In this work, we extend MANTRA, a benchmark dataset containing manifold triangulations, to a larger class of manifolds with more diverse homeomorphism types. We show that, contrary to prior claims, graph neural networks can saturate the benchmark, while higher-order message passing methods lag behind. However, we find that this is contingent on the right representation and feature assignment, emphasizing their importance in baseline models. Hence, we present a novel evaluation protocol based on representational diversity and triangulation refinement. Even when including refinement-like changes in the training data, existing models fail to generalize beyond the combinatorial structure of the data. This points towards a research gap in developing models that understand topological structure independent of scale. Our work thus provides the necessary scaffolding to evaluate future models and enable the development of topology-aware inductive biases.
Oct 26, 2026

Quantitative Generalization Bounds for Multi-Layer Transformers: A Few Long Strings can Beat Many Short Ones

Transformer-based language models exhibit an astonishing capacity to generalize to longer and more difficult problem instances than those seen in training. However, theoretical understanding of this phenomenon still lags behind. Unlike most machine learning models, the generalisation abilities of transformers inherently require considering two axes. On one hand, one must examine in-distribution statistical generalisation, which concerns the ability of a model to broaden a function from training samples to novel samples of the same distribution. On the other hand, given the sequential nature of language data, one must also consider length generalisation, which reflects whether a model trained on inputs of a given length can extrapolate to longer ones. These two notions have mostly been studied separately, leaving open how they interact. In this work, we give a joint treatment of both axes for fixed-precision transformers. We give generalisation guarantees in terms of the parameter sizes of the involved models, as well as the quantization space and the alphabet size that supply the sequences. Finally, we illustrate how the requirements in the sample count and training length trade off. In particular, we find that increasing training length can be exponentially more efficient than increasing the sample size. Experiments support predictions of the theory.