cs.CLNov 7, 2026

Can small transformers learn the language of Coxeter groups?

Authors: J. Elisenda Grigsby, Yiyao Huang, Arushi Katyal, Ryan Lin, Pranav Ramakrishnan, Weiya Wang

Organizations: Department of Mathematics, Boston College, Chestnut Hill, MA 02467

Abstract

The field of mechanistic interpretability is well-suited to collaborative projects between mathematicians and AI researchers. This work describes one such project. We trained a small transformer to learn the language of the Coxeter group affine A2, best known as the symmetries of the tiling of the Euclidean plane by equilateral triangles. We then formulated and proved a word-theoretic conjecture about affine A2 after observing the attention patterns of the trained model.

Figures & tables

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Transformers Provably Learn to Internalize Chain-of-Thought

    May 27, 2026Yixiao Huang, Hanlin Zhu, Zixuan Wang +4TransformerCoT Reasoning

  2. Fully Interpretable Minimal Transformers: From Geometry to Algorithm

    Oct 7, 2026Raneem Mahajne, Toviah MoldwinTransformer InterpretabilityMechanistic Interpretability