cs.CLAug 1, 2025

GHTM: A graph-based hybrid topic modeling approach for Bengali language

Authors: Farhana Haque, Md. Abdur Rahman, Sumon Ahmed

Organizations: Institute of Information Technology (IIT), University of Dhaka, Dhaka, Bangladesh · Centre for Advanced Research in Sciences (CARS), University of Dhaka, Dhaka, Bangladesh

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Topic modeling is a natural language processing technique used to discover latent themes and abstract topics from text corpora by grouping co-occurring keywords. Although widely researched in English, topic modeling remains understudied in Bengali due to a lack of adequate resources and initiatives. Existing Bengali topic modeling research lacks exploration of modern methodological approaches, reproducible implementations, diverse datasets, and systematic benchmarking. To bridge these gaps, this study presents Graph-based Hybrid Topic Model (GHTM), a novel architecture that synthesizes the distinct strengths of multiple paradigms by unifying statistical (TF-IDF), semantic (GloVe), and structural (GCN) modalities to construct semantically aligned document representations. GHTM later employs matrix factorization for topic extraction, offering a computationally lightweight alternative to resource-intensive neural and graph-based approaches. Through comprehensive benchmarking across three Bengali datasets of varying scales, GHTM consistently surpasses both traditional and contemporary topic modeling approaches, achieving improved topic coherence (NPMI: 0.27-0.28) and high topic diversity while maintaining computational efficiency. The model also outperformed established graph-based models on the English 20Newsgroups benchmark. Additionally, we introduce NCTBText, a diverse Bengali textbook-based dataset comprising 8,650 text documents, providing much-needed topical diversity beyond newspaper-centric Bengali corpora.

More in this topic

Topic Modeling

Topic modeling discovers recurring themes and document-topic structure in text, helping organize, compare, and track large collections.

See all 57 papers

Explore similar work

CardsList
Feb 22, 2026

TriTopic: Tri-Modal Graph-Based Topic Modeling with Iterative Refinement and Archetypes

Topic modeling extracts latent themes from large text collections, but leading approaches like BERTopic face critical limitations: stochastic instability, loss of lexical precision ("Embedding Blur"), and reliance on a single data perspective. We present TriTopic, a framework that addresses these weaknesses through a tri-modal graph fusing semantic embeddings, TF-IDF, and metadata. Three core innovations drive its performance: hybrid graph construction via Mutual kNN and Shared Nearest Neighbors to eliminate noise and combat the curse of dimensionality; Consensus Leiden Clustering for reproducible, stable partitions; and Iterative Refinement that sharpens embeddings through dynamic centroid-pulling. TriTopic also replaces the "average document" concept with archetype-based topic representations defined by boundary cases rather than centers alone. In benchmarks across 20 Newsgroups, BBC News, AG News, and Arxiv, TriTopic achieves the highest NMI on every dataset (mean NMI 0.575 vs. 0.513 for BERTopic, 0.416 for NMF, 0.299 for LDA), guarantees 100% corpus coverage with 0% outliers, and is available as an open-source PyPI library.

Roman Egger

Smartvisions.at & Modul University Vienna

Topic Modeling
AskPDF
Feb 20, 2026

Improving Topic Modeling by Distilling Soft Labels from Language Models

Traditional neural topic models are typically optimized by reconstructing the document's Bag-of-Words (BoW) representations, overlooking contextual information and struggling with data sparsity. In this work, we introduce a novel topic model training framework by Distilling Soft Labels (DSL) from Language Models (LMs). To construct the contextually enriched reconstruction signals, we project the next token probabilities, conditioned on a specialized prompt, onto a pre-defined vocabulary, and train the topic models to reconstruct the soft labels using the LM hidden states. This produces higher-quality topics that are more closely aligned with the underlying thematic structure of the corpus. Extensive experiments demonstrate that DSL achieves substantial improvements in topic coherence and assignment accuracy over existing baselines. Additionally, we also introduce a retrieval-based metric, which shows that our approach significantly outperforms existing methods in identifying semantically similar documents, highlighting its effectiveness for retrieval-oriented applications.

Raymond Li, Amirhossein Abaskohi, Chuyuan Li +2

University of British Columbia, Vancouver, Canada.

Topic Modeling
AskPDF
Feb 27, 2026

Benchmarking BERT-based Models for Sentence-level Topic Classification in Nepali Language

Transformer-based models such as BERT have significantly advanced Natural Language Processing (NLP) across many languages. However, Nepali, a low-resource language written in Devanagari script, remains relatively underexplored. This study benchmarks multilingual, Indic, Hindi, and Nepali BERT variants to evaluate their effectiveness in Nepali topic classification. Ten pre-trained models, including mBERT, XLM-R, MuRIL, DevBERT, HindiBERT, IndicBERT, and NepBERTa, were fine-tuned and tested on the balanced Nepali dataset containing 25,006 sentences across five conceptual domains and the performance was evaluated using accuracy, weighted precision, recall, F1-score, and AUROC metrics. The results reveal that Indic models, particularly MuRIL-large, achieved the highest F1-score of 90.60%, outperforming multilingual and monolingual models. NepBERTa also performed competitively with an F1-score of 88.26%. Overall, these findings establish a robust baseline for future document-level classification and broader Nepali NLP applications.

Nischal Karki, Bipesh Subedi, Prakash Poudyal +2

Information and Language Processing Research Lab, Kathmandu University, Kavre, Nepal

Bidirectional Encoder Representations from TransformersText ClassificationLow-Resource Language ProcessingMultilingual Language Model EvaluationTransformer
AskPDF