Recent advancements in topic modeling have split into two main paths: efficient embedding-based methods that offer scalability but are often outperformed on standard clustering metrics, and Large Language Model (LLM) approaches that achieve state-of-the-art clustering performance but at the cost of prohibitive computational expense and poor scalability. This paper introduces TopicMerge, a novel multi-stage pipeline that bridges these approaches to achieve both state-of-the-art clustering performance and computational efficiency. TopicMerge first employs an efficient clustering pipeline to deliberately over-segment a document corpus into a large set of fine-grained, semantically pure micro-clusters, which are then given initial semantic labels. It then introduces a novel, iterative merging algorithm where an LLM, guided by local neighborhood context, arbitrates on consolidating candidate pairs. These approved merges are then selectively executed and semantically relabeled by the LLM, hierarchically building broader, more coherent topics. Extensive experiments on the Bills and Wiki datasets show that TopicMerge achieves state-of-the-art clustering performance while reducing required LLM calls by over 92% compared to document-level methods, offering a practical and scalable solution.
Figures & tables
Figure 1: An overview of TopicMerge. The process begins with (1) initial clustering stage, where document embeddings are deliberately over-segmented into numerous, fine-grained micro-clusters. These are then given preliminary semantic representations in the (2) initial labeling stage. Then in the (3) a LLM-guided cluster merging stage, an LLM acts as a context-aware semantic arbiter to hierarchically consolidate the micro-clusters.
Bills
Wiki
P1
ARI
NMI
P1
ARI
NMI
LDA
0.52
0.31
0.42
0.72
0.63
0.71
NMF
0.24
0.04
0.15
0.58
0.38
0.55
CTM
0.37
0.19
0.32
0.61
0.46
0.62
FASTopic
0.40
0.30
0.33
0.65
0.56
0.60
EDTM
0.58
—
—
0.65
—
—
Table 1: Experimental results comparing TopicMerge against baselines. The best results are denoted in bold and the second-best results are marked by underline .
Models
PromptTopic
TopicGPT
LiSA
TopicMerge
Wiki
15,242+
15,242+
15,242+
562
Bills
8,024+
8,024+
8,024+
645
Table 2: A reporting of the number of prompts required for an LLM to classify each dataset. The number of prompts reported for PromptTopic, TopicGPT, and LiSA are lower-bounds based on their respective algorithms.
Bills
Wiki
P1
ARI
NMI
P1
ARI
NMI
TopicMerge
0.57
0.42
0.53
0.72
0.71
0.73
- Context
0.56
0.41
0.53
0.70
0.63
0.70
Table 3: Ablation results of our TopicMerge with and without context clusters.
Bills
Wiki
CC
P
P1
ARI
NMI
CC
P
P1
ARI
NMI
80
0.72
0.57
0.42
0.53
80
0.88
0.72
0.71
0.73
60
0.71
0.56
0.43
0.52
60
0.87
0.73
0.71
0.72
40
0.68
0.58
0.41
0.51
40
0.82
0.68
0.57
0.69
30
0.62
0.51
0.37
0.47
30
0.79
0.62
0.50
0.67
Table 4: A clustering quality comparison with the merging stage across different cluster counts from the initial clustering. The CC and P columns show the initial cluster count and corresponding purity without merging.
Figure 2: A visual ablation of the LLM-merging stage on the Bills dataset.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Prompt for Labeling
You are an expert content analyst and text categorizer.
Your goal is to propose a concise, descriptive topic label that accurately reflects the dominant and most central theme of the cluster. This label should capture the essence of the majority of the summaries, allowing for the possibility that a small percentage of summaries might be outliers that do not perfectly align. Granularity Target: The following are examples of the level of specificity your labels should aim for or exceed: ‘high_level_topic_examples‘. Your label should be at least as specific as these examples — never broader. These represent the coarsest acceptable level; prefer more specific labels when the texts support it. If the cluster’s dominant theme clearly falls under one of these examples, use that example as your label.
Each label must adhere to the following requirements:
1. Concise: Be 1-3 words long.
Appendix
Table 7
Prompt for Merging
You are deciding whether ‘Cluster_A‘ and ‘Cluster_B‘ should be merged into a single cluster.
Goal: Consolidate fragments of the same topic while keeping genuinely different topics apart.
Calibration:
Look at ‘high_level_topic_examples‘ — these are real topic labels from this dataset (though not exhaustive). They define the TARGET granularity level your clusters should reach. Your merged clusters should be roughly at this level of breadth. Use ‘local_context_clusters‘ and ‘global_context_clusters‘ to understand the broader landscape of topics in this dataset.
Appendix
Table 8
Prompt for Relabeling
You are an expert content analyst and text categorizer.
This cluster was formed by merging ‘cluster_A‘ and ‘cluster_B‘. These prior labels indicate the full scope of content in this cluster. Use them to understand the breadth of what this cluster contains, but do NOT simply concatenate them. Your job is to synthesize a single unified label that naturally encompasses both, guided by the representative ‘texts‘ provided. You may generalize, refine, or completely rename if the texts suggest a more accurate characterization.
Your Goal: Propose a concise, descriptive topic label that accurately reflects the dominant and most central theme of the combined cluster. This label should capture the essence of the majority of the texts, allowing for the possibility that a small percentage might be outliers.
Appendix
Table 9
Data
Topic Labels
Bills
Ground Truth
health: 2316
government operations: 1937
domestic commerce: 1323
Defense: 1297
Public Lands: 1232
Law and Crime: 967
Environment: 692
Transportation: 663
Energy: 661
Macroeconomics: 610
Appendix
Table 5: The topic labels for the Bills and Wiki datasets.
Bills
Wiki
P1
ARI
NMI
P1
ARI
NMI
LDA
0.52
0.31
0.42
0.72
0.63
0.71
NMF
0.24
0.04
0.15
0.58
0.38
0.55
CTM
0.37
0.19
0.32
0.61
0.46
0.62
FASTopic
0.40
0.30
0.33
0.65
0.56
0.60
EDTM
0.58
—
—
0.65
—
—
Appendix
Table 6: Experimental results comparing TopicMerge with greedy ground-truth guided merging against baselines. The best results among standard methods are marked in bold and second-best are underlined ; oracle results are shown for upper-bound comparison.