stat.MLNov 16, 2026

Modeling Document and Section Hierarchies in Embedding Space

Authors: Kriste Krstovski, Sharyn O'Halloran, Kshitij Chandna

Organizations: Columbia University

Abstract

Embedded topic models integrate word embeddings into the topic generation process, producing coherent topics and robust performance on large vocabularies. However, each document is represented by a single topic distribution, ignoring its internal structure. Many real world collections, including legal documents, company and government reports, and online discussion forums, exhibit hierarchical organization in which documents consist of multiple sections that often address distinct subtopics. We introduce the Hierarchical Embedded Topic Model (hETM), a structured topic model that couples the topic distribution of a document and those of its sections through a hierarchical prior in the embedding space. We derive an amortized variational inference procedure for hETM and evaluate it on structured collections from each of these domains. Experimental results show that hETM achieves the lowest or near-lowest held-out perplexity among flat and hierarchical topic model baselines, along with comparable or higher topic quality. It also performs strongly on a downstream clustering task, achieving the best clustering performance on most of the collections.

Figures & tables

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Sentence-Level Topic Modeling with Lexical and Contextual Evidence

    Oct 16, 2026Federica Granese, Serena Villata, Charles BouveyronTopic Modeling

  2. From Documents to Segments: A Contextual Reformulation for Topic Assignment

    May 18, 2026Hoonsang Yoon, Takyoung Kim, Wonkee Lee +3Topic Modeling

  3. A Manifold-Aware Topic Modeling Approach via Rank-Based Prototypes

    Aug 29, 2026Thiago César Castilho Almeida, Daniel Carlos Guimarães PedronetteTopic Modeling