cs.CLNov 17, 2026

What's in a Tree ? Hierarchical nn-gram models as controlled approximators of language

Authors: Aritra Das, Francesco Cagnetta, Darshil Doshi, Victor V. Albert, Maissam Barkeshli, Andrey Gromov

Organizations: Joint Center for Quantum Information and Computer Science, UMD & NIST. · Simplex. · Data Science and AI Institute, and Department of Physics and Astronomy, JHU. · Department of Physics, UMD. · 

Abstract

Hierarchical models have been proposed as synthetic settings for studying language. However, their relevance to realistic language corpora remains underexplored. In this work, we introduce tree-grams, autoregressive generative nn-gram models that build the conditional probability tensor p(xn∣x1:n−1)p(x_n \mid x_{1:n-1}) through a hierarchical composition of order-2 and order-3 tensors. nn controls the maximal context window, whereas a rank hyperparameter dd controls model capacity, yielding an interpretable and systematically scalable family of approximators of language. We show how tree-grams outperform existing nn-gram baselines, while remaining competitive with deep transformer-based language models. We demonstrate the utility of such controlled approximations by using tree-grams as generative models. In particular, we show that hyperparameter transfer under μμP can be made more robust when narrower models are tuned on simpler data, generated by tree-grams with moderate nn and dd. In addition, we show that entropy and effective range of correlations control performance gap after fine-tuning. This is because tree-grams allow us to explicitly control horizon and capacity. When allowed to route across candidate topologies, we observe emergent sparsity in tree-grams. Such concentrated routing permits pruning upto 95% of topologies without significant change in performance.

Figures & tables

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. A Hierarchical Language Model with Predictable Scaling Laws and Provable Benefits of Reasoning

    May 13, 2026Jason Gaitonde, Frederic Koehler, Elchanan Mossel +2Long-Context Language ModelingLanguage Model Scaling Laws

  2. Tensorizing Engram: Sharing Latents Across N-Gram Embeddings is Beneficial in LLMs

    Jun 6, 2026Wuyang Zhou, Yuxuan Gu, Giorgos Iacovides +3Text EmbeddingsLLM Compression

  3. Lngram: N-gram Conditional Memory in Latent Space

    May 24, 2026Yunao Zheng, Guoyang Xia, Xiaojie Wang +1Memory-Augmented Language ModelsLong-Context Language Modeling