Dynamic Topic Modeling for Cross-Corpus Temporal Analysis

Ruoxuan Li; Bruce Kogut

cs.CLAug 24, 2026

Dynamic Topic Modeling for Cross-Corpus Temporal Analysis

Authors: Ruoxuan Li, Bruce Kogut

Organizations: Columbia University New York, NY, USA · Department of Sociology and Columbia Business School Columbia University New York, NY, USA

Abstract

Dynamic Embedded Topic Models (D-ETM) provide an interpretable framework for modeling temporal semantic evolution, but cross-corpus comparison remains difficult because topics are often learned independently and aligned only after training, a process that does not guarantee stable topic correspondence across corpora and time. To address this problem, we propose a D-ETM framework that first learns a common dynamic topic space over a merged multi-corpus collection, which we call the shared backbone, then introduces corpus-specific residual adaptation around the frozen backbone without creating separate latent topic spaces. This design preserves a shared topic index for cross-corpus comparison while allowing each corpus to specialize lexically. We evaluate the framework on three temporally structured corpora spanning 97 years: the Corpus of Historical American English, Harvard Business Review, and International Labour Review. Residual adaptation improves corpus-specific fit relative to the shared backbone while preserving the same-index cross-corpus topic trajectories, achieving substantially stronger alignment than full fine-tuning from the same backbone, with $97.5 \pm 0.7\%$ versus $17.9 \pm 1.1\%$ trajectory Retrieval@1, as well as stronger alignment than independent training with post-hoc Hungarian matching. These results suggest that incorporating topic alignment into the model can support more stable over-time cross-corpus comparisons while retaining corpus-specific lexical variation.

Figures & tables

Figure 1. Overview of the two-stage shared-backbone framework: merged-corpus backbone training followed by source-specific residual adaptation. Overview of the two-stage shared-backbone topic modeling framework with merged-corpus training followed by source-specific residual adaptation.

Corpus	Train	Valid	Test	Total
COHA (Mag. / News, $\rho{=}0.3$ )	32,460	1,999	6,042	40,501
HBR	33,513	1,777	6,358	41,648
ILR	32,615	2,169	5,821	40,605
Merged D	98,588	5,945	18,221	122,754

Table 1. Dataset statistics after preprocessing. Counts refer to 500-token chunks in the merged-vocabulary configuration.

Condition	Description
Ind-CS	Independently train one D-ETM per corpus using that corpus’s own active vocabulary. This represents the standard independent-training setting, where cross-corpus comparison requires post-hoc topic matching.
Ind-MV	Independently train one D-ETM per corpus using the shared merged vocabulary. This controls for vocabulary differences while keeping the training process corpus-specific.
SB-Joint	Train a single D-ETM on the merged multi-corpus collection. This model learns the shared dynamic topic backbone used to define common topic indices across corpora.
SB-RA	Initialize from SB-Joint , freeze the shared backbone, and learn source-specific residual topic offsets. This is the proposed method for corpus-local lexical specialization while preserving shared topic identities.
SB-FT	Initialize from SB-Joint and fully fine-tune all model parameters separately on each corpus. This provides a less constrained adaptation baseline that tests what happens when the shared backbone is not preserved.

Table 2. Training and comparison conditions.

Source	$n_{\mathrm{test}}$	Ind-CS	Ind-MV	SB-Joint	SB-RA	SB-FT
COHA	6,042	6,446.75	6,458.27	9,819.13	7,896.94	6,364.36
HBR	6,358	2,479.74	2,530.01	2,570.88	2,396.78	2,127.39
ILR	5,821	1,722.68	1,701.09	1,561.29	1,486.90	1,320.03

Table 3. Held-out perplexity across experimental conditions on the shared merged test split. Lower values are better. Each row is evaluated on the same held-out documents within the corresponding source.

Setup	TD	TC	$\boldsymbol{\mathrm{C}_{V}}$	TQ
Ind-CS-COHA	0.5843	0.0551	0.2300	0.0322
Ind-CS-HBR	0.5304	0.0460	0.1968	0.0244
Ind-CS-ILR	0.4494	0.0350	0.2172	0.0157
Ind-MV-COHA	0.5810	0.0558	0.2229	0.0324
Ind-MV-HBR	0.5015	0.0413	0.1975	0.0207
Ind-MV-ILR	0.4824	0.0377	0.2198	0.0182

Table 4. Full-reference topic quality across experimental conditions.

Method	$\boldsymbol{\lambda}_{\textbf{anchor}}$	$\boldsymbol{\lambda}_{\textbf{smooth}}$	Same JSD $\downarrow$	Hung. JSD $\downarrow$	Margin $\uparrow$	R@1 $\uparrow$
Ind-CS	–	–	–	0.594	–	–
Ind-MV	–	–	–	0.619	–	–
SB-RA	$10^{-3}$	$10^{-3}$	$\mathbf{0.169\pm 0.001}$	$\mathbf{0.169\pm 0.001}$	$\mathbf{+0.166\pm 0.002}$	$\mathbf{97.5\pm 0.7\%}$
SB-RA	$3{\times}10^{-4}$	$3{\times}10^{-4}$	$0.334\pm 0.004$	$0.334\pm 0.004$	$+0.029\pm 0.002$	$70.4\pm 0.8\%$
SB-RA	$10^{-4}$	$10^{-4}$	$0.428\pm 0.009$	$0.424\pm 0.008$	$-0.034\pm 0.005$	$47.1\pm 3.1\%$
SB-RA	$10^{-3}$	$0$	$0.176\pm 0.001$	$0.176\pm 0.001$	$+0.155\pm 0.001$	$96.9\pm 0.4\%$

Table 5. Trajectory-level cross-corpus alignment across experimental and ablation conditions. All results are averaged over the three corpus pairs: COHA–HBR, COHA–ILR, and HBR–ILR. Shared-backbone-initialized adaptation conditions (SB-RA and SB-FT) are reported as mean

\pm

standard deviation over four Stage-2 random seeds using the same pretrained SB-Joint backbone. Independently trained baselines are single runs and are evaluated using post-hoc Hungarian matching.

$K$	Method	Same JSD $\downarrow$	Hung. JSD $\downarrow$	Margin $\uparrow$	R@1 $\uparrow$
10	SB-RA	0.257	0.257	+0.106	85.0%
	SB-FT	0.533	0.526	-0.058	26.7%
20	SB-RA	0.170	0.170	+0.164	98.3%
	SB-FT	0.592	0.573	-0.074	17.5%
30	SB-RA	0.106	0.106	+0.196	100.0%
	SB-FT	0.570	0.553	-0.087	13.9%

Table 6. Sensitivity of trajectory alignment to the number of topics. All models use the same random seed within each topic-count setting. Results are averaged over the three corpus pairs.

$K$	SB-Joint time	SB-RA time	SB-FT time	Peak GPU memory
10	57m 26s	13m 11s	14m 53s	3.14 GB
30	82m 13s	18m 21s	20m 07s	7.51 GB

Table 7. Runtime and peak GPU memory for the

K

-sensitivity experiments on NVIDIA A100-SXM4-80GB GPUs. SB-FT runtime is summed across COHA, HBR, and ILR.

	Avg. Top-30 overlap	Avg. # outside shared Top-30
COHA	0.853 $\pm$ 0.106	4.42 $\pm$ 3.19
HBR	0.790 $\pm$ 0.103	6.31 $\pm$ 3.08
ILR	0.791 $\pm$ 0.131	6.26 $\pm$ 3.93

Table 8. Average Top-30 vocabulary overlap with the shared backbone across all topics and time bins.

	Industrial crisis	Corporate management	Workplace organization	Digital services
	1922–1942	1947–1977	1982–1992	1997–2017
COHA	man, credit, public, power, department, material, period, place	control, policy, operation, production, price, service, major, small	information, change, state, price, rate, power, group, small	people, world, group, price, return, board, industry, cost
HBR	bank, rate, export, cost, material, credit, income, investment	profit, sale, capital, investment, product, price, rate, tax	price, capital, competition, investment, product, company, profit, corporate	cost, product, customer, sale, consumer, manager, stock, revenue
ILR	wage, labor, worker, condition, factory, rate, income, country	worker, labor, country, wage, policy, income, estimate, level	policy, level, worker, labor, trade, economic, rate, price	social, worker, labor, job, pay, country, level, cost

Table 9. Corpus-local Top-30 words for Topic 0. Source-local words are obtained by removing the corresponding shared-backbone Top-30 words from each source-specific Top-30 list.

Industrial crisis 1922–1942

Corporate / welfare 1947–1977

Workplace reorganization 1982–1992

Digital / labor market 1997–2017

Table 10. Phase-level summaries for three topics selected from the trained shared-backbone model.

Figure 2. Raw topic-word probability trajectories for selected Topic 0 words across COHA, HBR, and ILR. The plotted values are source-specific topic-word probabilities over five-year bins. Line plots showing source-specific word-probability trajectories for selected words in aligned Topic 0 across COHA, HBR, and ILR over five-year time bins.

Topic Modeling

Topic modeling discovers recurring themes and document-topic structure in text, helping organize, compare, and track large collections.

See all 57 papers

When Lexical Change Misleads: Rethinking Dynamic Topic Model Evaluation with Traditional and LLM-Based Metrics
Aug 13, 2026Charu Karakkaparambil JamesTopic Modeling
RPTU University Kaiserslautern-Landau Kaiserslautern, Germany
LLM-XTM: Enhancing Cross-Lingual Topic Models with Large Language Models
May 5, 2026Minh Chu Xuan, Tien-Phat Nguyen, Linh Ngo Van +3Topic Modeling
1Hanoi University of Science and Technology · 2VNU University of Engineering and Technology · 3Monash University
Comment-level Topic Drift Analysis in the Reddit Corpus
Aug 19, 2026Steven Morse, Daniel Runfola, Trenton W. FordSemantic Change Detection Topic Modeling
William & Mary

Aug 13, 2026J/K move · Enter open · S save

When Lexical Change Misleads: Rethinking Dynamic Topic Model Evaluation with Traditional and LLM-Based Metrics

Charu Karakkaparambil James

RPTU University Kaiserslautern-Landau Kaiserslautern, Germany

Topic Modeling

Open paper AskPDF arXiv

Dynamic topic models capture evolving word distributions, but traditional coherence metrics may fail when vocabulary changes while semantic meaning persists. We evaluate 120 topics from CoNTM and DLDA across NYT, DBLP, and arXiv, using three human annotators and Low, Medium, and High lexical-change categories. Traditional temporal coherence shows highly variable agreement with human judgments (

ρ

=-0.256 to 0.614). In contrast, LLM-based semantic similarity agrees strongly with human semantic judgments for CoNTM on NYT (

ρ

=0.609), DBLP (

ρ

=0.721), and arXiv (

ρ

=0.502), but is less consistent for DLDA. Lexical-change stratification reveals variation hidden by aggregate evaluation. We therefore advocate lexical-change-aware evaluation, jointly reporting traditional coherence and LLM-based semantic measures as complementary rather than interchangeable signals.

Abstract

Topic Modeling

Explore similar work

When Lexical Change Misleads: Rethinking Dynamic Topic Model Evaluation with Traditional and LLM-Based Metrics

LLM-XTM: Enhancing Cross-Lingual Topic Models with Large Language Models

Comment-level Topic Drift Analysis in the Reddit Corpus