Segment-Level Agentic Topic Modeling for Improved Data Exploration and Resource Efficiency

Myeongjun Erik Jang; Antonios Georgiadis; Sae Young Moon; Fran Silavong

cs.AISep 25, 2026

Segment-Level Agentic Topic Modeling for Improved Data Exploration and Resource Efficiency

Authors: Myeongjun Erik Jang, Antonios Georgiadis, Sae Young Moon, Fran Silavong

Organizations: J.P. Morgan Chase, London, United Kingdom.

Abstract

Topic modeling is an effective technique for discovering hidden themes within documents and is widely used in text mining and data analysis across a variety of industry sectors. Recently, large language model (LLM)-based topic models have been emerged that prompt LLMs to generate topics then assign the topics to documents, producing more natural and human-readable topics than conventional topic modeling algorithms. However, the nature of topic assignment process causes certain drawbacks, such as the incapability to produce topic distributions over a document, too broad or narrow topics, and high resource consumption, which increases with the number and length of of documents being assigned topics. These issues are particularly critical for industrial applications, which require high-quality, in-depth analysis and the processing of large volumes of documents. In this context, this paper introduces a framework called SeLATM, which addresses these concerns by employing segment-level topic generation and topic refinement through agentic feedback loops. Experimental results on various datasets demonstrate that SeLATM significantly reduces the LLM resources compared to methods based on topic assignment process, while maintaining superior performance.

Figures & tables

Data set	Data size	Token Len	# of Labels
Banking77	1,947	13	77
Bills	1,000	233	101
Wiki	1,100	3950	207
CCC	603	116	-
BCR	1,250	15	-
CI	3,000	17	-

Table 1: Basic statistics of datasets used for experiments.

$\mathcal{TA}$	$\mathcal{TC}$	Score
Incorrect	Not Covered	0
Partially Correct	Minorly Covered	1/3
Mostly Correct	Mostly Covered	2/3
Completely Correct	Complete	1

Table 2: Textual labels and corresponding scores used for LLM-as-a-Judge evaluation.

Models	Banking77					Bills					Wiki
Models	P1	ARI	NMI	$\mathcal{TA}$	$\mathcal{TC}$	P1	ARI	NMI	$\mathcal{TA}$	$\mathcal{TC}$	P1	ARI	NMI	$\mathcal{TA}$	$\mathcal{TC}$
BERTopic	.705	.468	.817	.634	.667	.359	.230	.558	.099	.066	.303	.144	.652	.137	.076
C-Top2vec	.557	.406	.788	.470	.704	.424	.230	.767	.350	.379	.490	.341	.685	.501	.574
TopicGPT	.172	.078	.532	.734	.683	.385	.244	.709	.748	.740	.392	.183	.779	.826	.627
LLooM	.111	.037	.322	.283	.259	.159	.048	.295	.501	.424	.110	.046	.424	.544	.353
TIDE	.624	.486	.800	.782	.867	.440	.285	.711	.759	.719	.389	.259	.816	.641	.621

Table 3: Experimental results on the public datasets. We report an average of 5 repetitions. The best performance for each evaluation metric is highlighted in bold; underlined values indicate statistically significant performance gap over the best-performing baselines (or over SeLATM when a baseline performs best), determined by t-test with a p-value

<

0.05. ‘ SeLATM Iter:0’ refers to SeLATM results without the refinement loop; italicized values indicate where ‘ SeLATM Iter:0’ outperforms the baselines.

Models	CCC		BCR		CI
Models	$\mathcal{TA}$	$\mathcal{TC}$	$\mathcal{TA}$	$\mathcal{TC}$	$\mathcal{TA}$	$\mathcal{TC}$
C-Top2vec	.723	.886	.050	.016	.607	.829
TIDE	.750	.773	.502	.555	.836	.842
LLM-ITL	.453	.409	.344	.369	.228	.189
SeLATM Iter:0	.825	.959	.832	.870	.845	.780
SeLATM Iter:1	.840	.957	.845	.880	.847	.782

Table 4: Experimental results on the business datasets. We report an average of 5 repetitions. The best performance for each evaluation metric is highlighted in bold; underlined values indicate statistically significant performance gap over the best-performing baselines (or over SeLATM when a baseline performs best), determined by a t-test with a p-value

<

0.05.

Panel of Figure 1 — Figure 1: An average LLM token consumption comparison between SeLATM , TIDE, and TopicGPT.

Panel of Figure 2 — Figure 2: The $\mathcal{TA}$ and $\mathcal{TC}$ results on the parent topics.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

$n_{S}$	Banking77		Bills		Wiki		CCC		BCR		CI
$n_{S}$	$\mathcal{TA}$	$\mathcal{TC}$	$\mathcal{TA}$	$\mathcal{TC}$	$\mathcal{TA}$	$\mathcal{TC}$	$\mathcal{TA}$	$\mathcal{TC}$	$\mathcal{TA}$	$\mathcal{TC}$	$\mathcal{TA}$	$\mathcal{TC}$
30	.932	.924	.775	.744	.814	.586	.791	.954	.831	.870	.753	.703
40	.935	.927	.795	.769	.832	.574	.810	.942	.809	.844	.807	.735
50	.947	.935	.774	.764	.801	.573	.825	.959	.825	.857	.796	.723
60	.955	.950	.787	.755	.803	.587	.812	.958	.806	.839	.832	.762
70	.955	.953	.777	.734	.805	.584	.821	.958	.805	.828	.845	.780

Appendix

Table 5: Results of

n_{S}

search. We report an average of 5 repetitions. For each dataset, the best performance achieved by SeLATM with varying

n_{S}

values is highlighted in bold. Regarding the Banking77 and BCR datasets,

n_{S}=80

was not evaluated because the largest cluster in each dataset contained fewer than 80 instances. The best baseline performance is underlined if it achieves a higher score than the lowest performance of SeLATM across the different

n_{S}

values.

	Models	Banking77			Bills			Wiki
	Models	P1	ARI	NMI	P1	ARI	NMI	P1	ARI	NMI
Avg.	K-Means	.664	.524	.819	.415	.247	.680 *	.500	.334	.788
Avg.	Agglomerative	. 684 *	.542 *	.832 *	.440 *	.255	.663	.522 *	.357 *	.805 *
Std.	K-Means	.014	.018	.007	.011	.026	.005	.012	.023	.004
Std.	Agglomerative	.002	.002	.000	.005	.002	.001	.004	.005	.001

Appendix

Table 6: Ablation study on the clustering methods. We report an average of 5 repetitions (Avg.) and their standard deviations (Std.). The best average performance for each evaluation metric is highlighted in bold. The best performance exhibits statistically significant gap (*) determined by t-test with a p-value

<

0.05.

	Iterations	Banking77		Bills		Wiki
	Iterations	$\mathcal{TA}$	$\mathcal{TC}$	$\mathcal{TA}$	$\mathcal{TC}$	$\mathcal{TA}$	$\mathcal{TC}$
no Iteration	0	.955	.953	.795	.769	.832	.574
split + merge	1	.962	.956	.803	.782	.807	.567
	3	.966	.960	.823	.792	.842	.587
	5	.962	.961	.815	.789	.841	.588
w/o split	1	.957	.955	.754	.770	.687	.525

Appendix

Table 7: Ablation study on the split operation. We report an average of 5 repetitions. The performance of ‘ split + merge ’ is underlined when it is statistically significantly higher than that of ‘w/o split ’ in the corresponding iteration under a t-test with a p-value

<

0.05.

Dataset	Input Text	Generated Topic
Banking77	The exchange rate applied is wrong for my international purchase.	{‘Incorrect Exchange Rates’: 1.0}
Bills	Restore American Dream Act of 2007 - Amends the Internal Revenue Code to establish tax-exempt homeownership plans. Allows a tax deduction from gross income for cash contributions to such plans. Defines ”homeownership plan” as a trust established for the exclusive purpose of paying the costs (e.g., downpayment, interest, mortgage insurance, closing costs, etc.) of acquiring a principal residence by an individual who has never owned a principal residence. Excludes from gross income distributions from a homeownership plan used to pay the costs of acquiring a principal residence. Sets forth rules governing: (1) transfers of an interest in a plan due to death or divorce; (2) penalties for making distributions from a plan for purposes other than to acquire a principal residence; and (3) employer homeownership plans established for the benefit of their employees.	{‘Housing Legislation’: 0.2, ‘Tax Legislation’: 0.8}
Wiki	Borodino-class battlecruiser = The Borodino-class battlecruisers were a group of four battlecruisers ordered by the Imperial Russian Navy before World War I. Also referred to as the Izmail class , they were laid down in December 1912 at Saint Petersburg for service with the Baltic Fleet. […] None of the proposals was accepted, and all three of the less complete ships were sold to a German company for scrap on 21 August 1923 to raise much-needed cash for the government . The Operational Administration of the Soviet Navy worked out the requirements in May 1925 for a conversion that would have made Izmail into an aircraft carrier with a top speed of 27 knots (50 km / h;31 mph) and a capacity of fifty aircraft. She would have been armed with eight 183-millimetre (7.2 in) guns and her armour reduced to a maximum of 76 millimetres (3.0in). This proposal was approved by Alexey Rykov , Chairman of the Council of the People ’s Commissars on 6 July 1925. The plan was cancelled on 16 March 1926 after the Red Army managed to gain control of a commission appointed to review the needs of the Navy, and the Army was strongly opposed to spending additional money on naval projects. She was scrapped beginning in 1931 in Leningrad . = = Ships = = The ships were named after battles fought by the Russian Empire:	{‘Naval Warfare’: 0.517, ‘Naval Armament’: 0.379, ‘Naval Propulsion’: 0.103}

Appendix

Table 8: Examples of public datasets and their corresponding topics generated by SeLATM .

Topic	Generated Topic Description
Incorrect Exchange Rates	This topic is about issues related to incorrect exchange rates applied during purchases or cash withdrawals in foreign currencies. Customers are expressing concerns about discrepancies between expected and actual exchange rates applied to their international transactions.
Housing Legislation	This topic is about various legislative acts and amendments related to housing policies, including public housing, tax exemptions for homeownership, mortgage regulations, and housing assistance programs. It encompasses issues such as rent adjustments, affordable housing eligibility, support for vulnerable populations, and regulatory simplifications to improve housing access and management.
Tax Legislation	This topic is about various legislative amendments to the Internal Revenue Code aimed at modifying tax rates, exclusions, and rules related to corporate, estate, and investment taxation. These amendments address issues such as estate and gift tax exclusions, corporate expatriations, timber gains, and regulated investment companies, reflecting ongoing efforts to refine tax policies and their economic implications.
Naval Warfare	This topic is about naval engagements and operations during various wars, focusing on the activities, missions, and modifications of naval ships and fleets. It includes minelaying, escort duties, battles, and ship modifications across different time periods and conflicts.
Naval Armament	This topic is about the various armaments and defensive features used in naval ships across different eras, including guns, torpedoes, and armor configurations.
Naval Propulsion	This topic is about the propulsion systems and specifications of various naval ships, detailing their power sources, engine types, speeds, fuel capacities, and ranges. It encompasses the technological aspects of how these ships were designed to achieve their operational capabilities and endurance at sea.

Appendix

Table 9: The descriptions of topics presented in Table 8 .

Dataset	Input Text	Generated Topic
CCC	VIRTUAL_AGENT : Hi [Customer_Name]. How can I help you? CUSTOMER : Hi, how to avoid overdrawn fee VIRTUAL_AGENT : Which account would you like to get info on fees for? We want to make sure you’re not surprised by a fee you didn’t expect. VIRTUAL_AGENT : Here’s where you can learn about the fees associated with your account, plus tips on avoiding some of them.	{‘Chatbot Greetings’: 0.25, ‘Overdraft Fees’: 0.25, ‘Fee Notifications,: 0.25, ‘Fee Information’: 0.25}
BCR	I kept getting the same response to an issue different than the one I’m asking about	{‘Repetitive Responses’: 1.0}
CI	Customer noticed a $109 charge on their Apple bill pay for an in-person transaction in California, which they did not recognize.	{‘Apple Pay Issues’: 1.0}

Appendix

Table 10: Examples of business datasets and their corresponding topics generated by SeLATM .

Topic	Generated Topic Description
Chatbot Greetings	This topic is about the variety of introductory messages used by the bank’s virtual agent to greet customers and initiate conversations, highlighting the personal touch in customer service interactions.
Overdraft Fees	This topic is about customers’ concerns and inquiries regarding overdraft fees, including requests for fee reversals, waivers, protection, and clarifications on charges. Customers often seek assistance to avoid or dispute these fees and express dissatisfaction with unexpected charges.
Fee Notifications	This topic is about the bank’s virtual agent ensuring that customers are informed about fees that might appear on their accounts, preventing any unexpected charges. It reflects the bank’s proactive approach in maintaining transparency with its clients regarding potential fees.
Fee Information	This topic is about providing information on bank account fees and offering advice on how to avoid them. It focuses on educating customers about the costs associated with their accounts and helping them manage or reduce potential charges.
Repetitive Responses	This topic is about users experiencing frustration with the chatbot providing repetitive and unhelpful responses, requiring them to rephrase questions multiple times without improvement in the outcome.
Apple Pay Issues	This topic is about various difficulties customers face with Apple Pay, including unauthorized charges, declined transactions, and problems with account activation or verification.

Appendix

Table 11: The descriptions of topics presented in Table 10 .

Topic Modeling

Topic modeling discovers recurring themes and document-topic structure in text, helping organize, compare, and track large collections.

See all 58 papers

From Documents to Segments: A Contextual Reformulation for Topic Assignment
May 18, 2026Hoonsang Yoon, Takyoung Kim, Wonkee Lee +3Topic Modeling
1LG AI Research · University of Illinois Urbana-Champaign
PRISM: LLM-Guided Semantic Clustering for High-Precision Topics
Apr 3, 2026Connor Douglas, Utkucan Balci, Joseph Aylett-BullockSemantic Clustering Topic Modeling
New York University New York, NY, USA · Binghamton University Binghamton, NY, USA · United Nations New York, NY, USA
A Neural Topic Method Using a Large-Language-Model-in-the-Loop for Business Research
Mar 4, 2026Stephan Ludwig, Peter J. Danaher, Xiaohao YangTopic Modeling
Monash University, Australia

May 18, 2026J/K move · Enter open · S save

From Documents to Segments: A Contextual Reformulation for Topic Assignment

Hoonsang Yoon, Takyoung Kim, Wonkee Lee, Ilmin Cho+2

1LG AI Research · University of Illinois Urbana-Champaign

Topic Modeling

Open paper AskPDF arXiv

Traditional topic modeling assigns a single topic to each document. In practice, however, many real-world documents, such as product reviews or open-ended survey responses, contain multiple distinct topics. This mismatch often leads to topic contamination, where unrelated themes are merged into a single topic, making it difficult to identify documents that truly focus on a specific subject. We address this issue by introducing segment-based topic allocation (SBTA), a reformulation of topic modeling that assigns topics not to entire documents, but to segments: short, coherent spans of text that each express a single theme. By modeling topical structure at the segment level, our approach yields cleaner and more interpretable topics and better supports analysis of multi-theme documents. To support systematic evaluation, we construct a SemEval-STM, a new dataset inspired by aspect-based sentiment analysis. Documents are first decomposed into topical segments using large language models (LLMs), followed by human refinement to ensure segment quality. We also propose a segment-level extension of the word intrusion task, enabling human evaluation of topical coherence at the granularity where topics are actually assigned. Across multiple models and evaluation metrics, we show that SBTA improves clustering quality and interpretability. Overall, this work provides a practical, scalable framework for fine-grained topic analysis in heterogeneous text corpora where documents naturally span multiple topics. URL: https://huggingface.co/datasets/LG-AI-Research/SemEval-STM

Abstract

Topic Modeling

Explore similar work

From Documents to Segments: A Contextual Reformulation for Topic Assignment

PRISM: LLM-Guided Semantic Clustering for High-Precision Topics

A Neural Topic Method Using a Large-Language-Model-in-the-Loop for Business Research