math.GTNov 5, 2026

Using Transformers to Generate 3-Manifold Triangulations

Authors: Daniel Bruwel, Ashvni Narayanan, Jonathan Spreer

Organizations: School of Mathematics and Statistics F07, The University of Sydney, NSW 2006 Australia · Department of Computer Science, Technion, Haifa, 3200003, Israel

Abstract

Using dehydration signatures as an encoding, we train GPT-2-style transformers to generate triangulations of 3-dimensional manifolds. Our main runs each use approximately 3 exaFLOPs, spanning a 200 million parameter model trained on 60 million sphere triangulations to an 11 million parameter model trained on 1.1 billion sphere triangulations. The optimal model has 10 blocks, width 640 and generates dehydration signatures of closed manifolds of various topological types at a rate of 6.9%, 82% of which are spheres. To investigate how our method scales, we use the technique of IsoFLOP curves on 15-tetrahedron triangulations of the 3-sphere, over a compute range of 6 to 3000 petaFLOPs, and observe neural scaling behaviour. We extract a parameter scaling exponent of α=0.49α= 0.49 ([0.48,0.50][0.48, 0.50] stat, ±0.02\pm 0.02 syst), and an optimal ratio of 234 tokens per parameter for training. In addition, we present a number of smaller experiments, including evidence that transformers are able to learn to count degree-one edges near-perfectly, but get progressively worse when counting edges with higher degrees.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Generating Special Triangulations with Transformers

    Jun 25, 2026Charles Arnal, Jacky H. T. Yip, François Charton +1Generative Modeling

  2. No Triangulation Without Representation: Generalization in Topological Deep Learning

    May 7, 2026Johannes S. Schmidt, Martin Carrasco, Ernst Röell +3Neural Network GeneralizationGraph Neural Networks

  3. Quantitative Generalization Bounds for Multi-Layer Transformers: A Few Long Strings can Beat Many Short Ones

    Oct 26, 2026Marius Henry, Michael Rizvi-Martel, Michael HahnLanguage Model GeneralizationGeneralization Bounds