Similar but Not Redundant: Embedding-Gated Refinement for High-Granularity Topic Models
Organizations: Strategic Futures Lab, Heriot-Watt University, Edinburgh, United Kingdom · STMicroelectronics, Edinburgh, United Kingdom
Abstract
High-granularity topic modelling extracts hundreds of niche topics from a corpus, yet this regime is rarely studied and is more commonly framed as a failure mode: "over-clustering," in which coherent themes fragment into redundant subtopics. We argue that this framing conflates genuine redundancy with topics that are semantically similar but meaningfully distinct, and that the central challenge in high-granularity modelling is separating the two. Recent post-hoc refinement methods address topic redundancy through either embedding-based similarity thresholding or LLM-guided merging. We show that neither suffices in high-granularity settings and propose embedding-gated iterative merge (EGIM), a hybrid approach that iteratively generates context-rich topic representations, uses embedding similarity to propose candidate merges across the topic space, then applies LLM judgement to accept only genuinely redundant pairs while preserving niche distinctions. Across two patent corpora and one legal corpus, LLM judgement rejects 83% of embedding-identified candidates as non-redundant. EGIM improves discriminability on merged topics by 25-55% over a prior iterative LLM baseline while better preserving the original topic distribution.
Figures & tables
| K | UMass | Coh. | Intr. | Div. |
|---|---|---|---|---|
| 20 | 2.34 | .66 | .89 | |
| 40 | 2.36 | .67 | .85 | |
| 80 | 2.32 | .64 | .80 | |
| 160 | 2.26 | .62 | .76 | |
| 320 | 2.19 | .59 | .72 |
| LASER | LEGAL | SPAD | |
|---|---|---|---|
| Original | .533 | .355 | .369 |
| EGIM | .537 (101%) | .349 (98%) | .346 (94%) |
| LLM-Iter | .469 (88%) | .275 (78%) | .127 (34%) |
| Corpus | EGIM | LLM-Iter | |
|---|---|---|---|
| LASER | .130 | .103 | +25% |
| LEGAL | .104 | .079 | +32% |
| SPAD | .096 | .062 | +55% |
| LIDAR and LADAR Systems |
| + Weapon-Locating Ladar |
| + Raman/Fluorescence Explosive Detection |
| + Laser Range Finding |
| + Semi-Active Laser Guidance |
| + Laser Wavelength Selection |
| Result: Laser Targeting & Communication |
| Topic A | Topic B | Sim |
|---|---|---|
| Declined (distinct concepts) | ||
| Contract Interp. Principles | Statutory Interp. Principles | .893 |
| Misleading Conduct | Unconscionable Conduct | .854 |
| Merged (true duplicates) | ||
| Tax Assessment Objections | Taxation Objection Appeals | .861 |
| Extension of Time Applications | Extension of Time for Appeals | .875 |
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| Corpus | Context line |
|---|---|
| LASER | You are labeling topics from a corpus of laser technology patents filed by defense companies. |
| SPAD | You are labeling topics from a corpus of patents related to Single-Photon Avalanche Diode (SPAD) technology. |
| LEGAL | You are labeling topics from a corpus of Australian Federal Court legal cases. |
| Decision | Guidance (LASER corpus example) |
|---|---|
| MERGE | • Same core technology, even if applied differently (e.g., “WDM Networks” and “WDM for Data Transmission” both WDM) • One topic is a specific technique/variant of the other (e.g., “Grating-Based Beam Combining” is a type of “Coherent Beam Combining”) • Same physical principle with different implementations • A domain expert would file these under one category for competitive analysis |
| KEEP | • Fundamentally different physical principles (e.g., “Thermal Cooling” vs “Optical Sensing”) • Completely different application domains (e.g., “Medical Imaging” vs “Weapons Targeting”) • Topics serve genuinely different analytical purposes • A domain expert would want these tracked separately |
| Component | Parameter | Type | Range |
|---|---|---|---|
| UMAP | n_neighbors | int | |
| n_components | cat | ||
| min_dist | float | ||
| HDBSCAN | min_cluster_size | int | |
| min_samples | int | ||
| cluster_sel_epsilon | float |
| Var. | Embedding Model | Instruction |
|---|---|---|
| 4 | GTE-Multilingual-Base 6 6 6 Zhang et al. (2024) ; https://huggingface.co/Alibaba-NLP/gte-multilingual-base . | None |
| 5 | Qwen3-Embedding-0.6B | “…Emphasize identifying repetitive patterns, formulaic language, and structural boilerplate …” |
| 6 | Qwen3-Embedding-0.6B | “…Focus on novel substantive content, key concepts, and unique information …” |
| Corpus | Metric | Original | EGIM | LLM-Iter |
|---|---|---|---|---|
| laser | .710 | .710 | .710 | |
| NPMI | .154 | .156 | .155 | |
| UMass | ||||
| LLM Coherence | 2.38 | 2.40 | 2.38 | |
| LLM Intrusion | .701 | .716 | .703 | |
| Diversity | .724 | .731 | .730 |
| Topic A (label + summary) | Topic B (label + summary) | Sim. | Dec. |
|---|---|---|---|
| Beam Expansion and Collimation. Use of multi-lens arrangements and beam expanders to manipulate laser beam divergence and collimation. | Laser Beam Expanders with Liquid Lenses. Beam expanders utilising liquid lens elements, often made with Cargille liquids, to achieve specific optical properties. | .860 | merge |
| Optical Reflection and Refraction. Use of mirrors and prisms for manipulating laser beams through reflection and refraction in optical systems. | Turning Mirror in Laser Cladding. Use of a turning mirror in laser cladding systems to redirect beams and protect components during welding operations. | .838 | decline |
| Optical Beam Splitters. Use of beam splitters to divide laser beams into multiple segments for various applications, including measurement and signal processing. | Optical Signal Distribution in Sensor Arrays. Methods and systems for distributing optical signals to sensor arrays using couplers, distribution buses, and return buses. | .799 | decline |
| Narrow Bandpass Optical Filters. Narrow bandpass filters designed to selectively transmit specific wavelengths while blocking others, often used in laser applications. | Metamaterial Filters in IR Systems. Use of metamaterial filters in active infrared systems for spectral filtering and handling various laser wavelengths and incident angles. | .789 | decline |
| Optical Comb Filtering and Sampling. Use of optical comb filters and samplers to process and integrate modulated spectral copies for frequency analysis and signal processing. | Optical and Femtosecond Frequency Combs. Generation, stabilisation, and application of optical frequency combs, consisting of multiple phase-locked frequency tones. | .787 | decline |
| Diffraction Grating Systems. Use of diffraction gratings to manipulate and control beam dispersion and spatial chirping in optical systems. | In-Plane and Bevel Angles in Waveguides. Manipulation of in-plane and bevel angles in optical waveguides to control photon trajectories and enhance light guidance. | .779 | decline |
| Topic A (label + summary) | Topic B (label + summary) | Sim. | Dec. |
|---|---|---|---|
| Copyright Infringement Analysis. Legal analysis of copyright infringement, focusing on the concepts of substantial similarity, originality, and the reproduction of works. | Copyright Infringement Dispute. A legal dispute involving allegations of copyright infringement related to architectural plans created by Atrium and used by the Gaunt brothers and Beaumonde. | .805 | merge |
| Damages for IP Infringement : not selected for any merge by EGIM; preserved as a distinct topic. | |||
| Topic A (label + summary) | Topic B (label + summary) | Sim. | Dec. |
|---|---|---|---|
| Photodiode Array Structures. Design and arrangement of photodiode arrays, including avalanche photodiodes, within semiconductor substrates for light detection. | Photodiode Array Configurations. Various configurations and applications of photodiode arrays, including avalanche photodiodes, for precise photon detection. | .907 | merge |
| Photodetector–Substrate Integration and HA and VA Photodetector Arrays : not selected for any merge by EGIM; preserved as distinct topics. | |||
Topic Modeling
Topic modeling discovers recurring themes and document-topic structure in text, helping organize, compare, and track large collections.
See all 62 papers