In this paper, we propose Precision-Informed Semantic Modeling (PRISM), a structured topic modeling framework combining the benefits of rich representations captured by LLMs with the low cost and interpretability of latent semantic clustering methods. PRISM fine-tunes a sentence encoding model using a sparse set of LLM- provided labels on samples drawn from some corpus of interest. We segment this embedding space with thresholded clustering, yielding clusters that separate closely related topics within some narrow domain. Across multiple corpora, PRISM improves topic separability over state-of-the-art local topic models and even over clustering on large, frontier embedding models while requiring only a small number of LLM queries to train. This work contributes to several research streams by providing (i) a student-teacher pipeline to distill sparse LLM supervision into a lightweight model for topic discovery; (ii) an analysis of the efficacy of sampling strategies to improve local geometry for cluster separability; and (iii) an effective approach for web-scale text analysis, enabling researchers and practitioners to track nuanced claims and subtopics online with an interpretable, locally deployable framework.
Figures & tables
Figure 1. Pareto curves depicting cluster purity against number of topics across three corpora.
mpnet + Training Approach
Model
CText
FR
RB
Emb
mpnet
Mini
LEmb
HU
.806
.838
.892
.812
.789
.891
LI
.745
.824
.792
.745
.750
.818
IM
.906
.912
.811
.584
.605
.789
Table 1. AUC of models predicting labels in DRB(test) ; mpnet corresponds to all-mpnet-base-v2 , Mini corresponds to all-MiniLM-L12-v2 .
Figure 2. AUC and AUPC across training data size on HumAID for comparison and embedding dataset.
PRISM + Training Approach
Topic Model
CText
No train
RB
Emb
Emb. & RB
T2V (mpnet)
T2V (d2v)
A∘LEmb
HU
.758
.782
.792
.807
.717
.679
.786
LI
.635
.641
.636
.638
.636
.631
.639
IM
.900
.948
. 948
.966
.913
.886
.957
Table 2. AUPC results on labeled holdout test set (higher is better). Top2Vec uses Doc2Vec (d2v) as the default model; this is tested against Top2Vec using all-mpnet-base-v2 (mpnet).
Topic modeling seeks to uncover latent semantic structure in text, with LDA providing a foundational probabilistic framework. While recent methods often incorporate external knowledge (e.g., pre-trained embeddings), such reliance limits applicability in emerging or underexplored domains. We introduce PRISM, a corpus-intrinsic method that derives a Dirichlet parameter from word co-occurrence statistics to initialize LDA without altering its generative process. Experiments on text and single cell RNA-seq data show that PRISM improves topic coherence and interpretability, rivaling models that rely on external knowledge. These results underscore the value of corpus-driven initialization for topic modeling in resource-constrained settings. Code is available at: https://github.com/shaham-lab/PRISM.
Tal Ishon, Yoav Goldberg, Uri Shaham
Department of Computer Science, Bar Ilan University
Traditional topic models are effective at uncovering latent themes in large text collections. However, due to their reliance on bag-of-words representations, they struggle to capture semantically abstract features. While some neural variants use richer representations, they are similarly constrained by expressing topics as word lists, which limits their ability to articulate complex topics. We introduce Mechanistic Topic Models (MTMs), a class of topic models that operate on interpretable features learned by sparse autoencoders (SAEs). By defining topics over this semantically rich space, MTMs can reveal deeper conceptual themes with expressive feature descriptions. Moreover, uniquely among topic models, MTMs enable controllable text generation using topic steering vectors. To properly evaluate MTM topics against word list approaches, we propose topic judge, an LLM-based pairwise comparison evaluation framework. Across eight datasets, MTMs match or exceed traditional and neural baselines on coherence metrics, are consistently preferred by topic judge, and enable effective LLM steering.
Carolina Zheng, Nicolas Beltran-Velez, Sweta Karlekar +5
Columbia University · Independent · Google Research
Recent advancements in topic modeling have split into two main paths: efficient embedding-based methods that offer scalability but are often outperformed on standard clustering metrics, and Large Language Model (LLM) approaches that achieve state-of-the-art clustering performance but at the cost of prohibitive computational expense and poor scalability. This paper introduces TopicMerge, a novel multi-stage pipeline that bridges these approaches to achieve both state-of-the-art clustering performance and computational efficiency. TopicMerge first employs an efficient clustering pipeline to deliberately over-segment a document corpus into a large set of fine-grained, semantically pure micro-clusters, which are then given initial semantic labels. It then introduces a novel, iterative merging algorithm where an LLM, guided by local neighborhood context, arbitrates on consolidating candidate pairs. These approved merges are then selectively executed and semantically relabeled by the LLM, hierarchically building broader, more coherent topics. Extensive experiments on the Bills and Wiki datasets show that TopicMerge achieves state-of-the-art clustering performance while reducing required LLM calls by over 92% compared to document-level methods, offering a practical and scalable solution.
Matthew Raffel, Lizhong Chen, Sai Munikoti +1
Oregon State University · Pacific Northwest National Laboratory