Discovering Topics from Linear Autoencoders

Jutiyakhun Srisoparp; Donlapark Ponnoprat

cs.IROct 26, 2026

Discovering Topics from Linear Autoencoders

Authors: Jutiyakhun Srisoparp, Donlapark Ponnoprat

Organizations: Chiang Mai University, Chiang Mai, Thailand

Abstract

Linear recommenders and topic models are similar in that they implicitly learn from co-occurrences of items. However, there has been little study of how these two model families are related. In this work, we show that, under a latent-factor model of the user-item interactions, the latent topics can be discovered from the weight matrix of a closed-form linear autoencoder. These topics provide a basis for interpreting the recommendations, and can be obtained directly from the trained recommender without additional training. Building on our analysis, we further propose novel item and user normalizations that prevent popular items and very active users from dominating the topics. Our experiments on six public datasets demonstrate that our method achieves significantly higher topic quality than every dedicated topic model and matches or exceeds the best linear autoencoder on all six datasets, while its recommendation performance is competitive with the best tuned recommender.

Figures & tables

Figure 1. Subspace recovery on synthetic data. Left: the largest eigenvalues of

\widehat{A}

n=150000

. Right: the relative error

\min_{C}\lVert S^{\star}-\widehat{V}C\rVert/\lVert S^{\star}\rVert

against

n

, with the mean and one standard deviation over four seeds (the error bars are too small to be visible). The dotted line is the limit of the relative error with

\widehat{G}=G^{\star}

n\to\infty

. Two panels on synthetic data with K=5 topics. The left panel shows the 10 largest eigenvalues of the matrix A-hat, divided by the largest, with a visible cut-off after the fifth. The right panel shows the relative error of the estimated eigenspace, which decreases from 0.30 to 0.010 as the number of users grows from 500 to ten million. A horizontal dotted line at 0.0083 shows the limit of the relative error of the eigenspace computed from the expected co-occurrence matrix.

	books			cds			gowalla			ml-1m			retail			steam
Model	NDCG	Rec	TQ	NDCG	Rec	TQ	NDCG	Rec	TQ	NDCG	Rec	TQ	NDCG	Rec	TQ	NDCG	Rec	TQ
WMF	0.040	0.077	$-$	0.046	0.085	$-$	0.083	0.151	$-$	0.029	0.051	$-$	0.043	0.049	$-$	0.018	0.033	$-$
LightGCN	0.021	0.039	$-$	0.017	0.032	$-$	0.052	0.091	$-$	0.025	0.042	$-$	0.027	0.030	$-$	0.027	0.054	$-$
SLIMElastic	0.047	0.082	$-$	0.051	0.092	$-$	0.094	0.164	$-$	0.042	0.072	$-$	0.058	0.064	$-$	0.028	0.057	$-$
ADMMSLIM	0.046	0.081	$-$	0.052	0.093	$-$	0.096	0.168	$-$	0.041	0.070	$-$	0.056	0.060	$-$	0.026	0.051	$-$
NCEPLRec	0.040	0.073	$-$	0.050	0.090	$-$	0.093	0.167	$-$	0.042	0.072	$-$	0.056	0.062	$-$	0.025	0.048	$-$

Table 1. NDCG@10, recall@10 (Rec), and topic quality (TQ). Bold NDCG@10, and recall@10 numbers are based on paired two-sided

t

-tests over the users at 5% level, and bold TQ numbers are based on bootstrap resampling tests at 5% level.

Topic	Top items	$n^{\mathrm{eff}}$
amazon-books : five of the $K=6$ topics. Items are books.
young adult fiction	Mockingjay (The Hunger Games), Divergent, Catching Fire (Hunger Games, Book Two), The Hunger Games (Book 1), City of Bones (The Mortal Instruments, Book 1)	$22.5$
Women’s Murder Club series	4th of July (Women’s Murder Club), The 6th Target (Women’s Murder Club), 2nd Chance (Women’s Murder Club), 7th Heaven (Women’s Murder Club), 10th Anniversary (Women’s Murder Club)	$8.2$
psychological thrillers	Don’t Look for Me: A Novel, The Maid: A Novel (Molly the Maid Book 1), The Golden Couple: A Novel, The Plot: A Novel, Dark Roads: A Novel	$7.7$
Nameless thriller series	In the Heart of the Fire (Nameless: Season One Book 1), The Mercy of Snakes (Nameless: Season One Book 5), The Praying Mantis Bride (Nameless: Season One Book 3), Red Rain (Nameless: Season One Book 4), Photographing the Dead (Nameless: Season One Book 2)	$16.7$
physics workbooks	Physics Concept Questions - Book 1 (Mechanics 1), Physics Concept Questions - Book 3 (Matter), Physics Concept Questions - Book 4 (Heat), Physics Concept Questions - Book 2 (Mechanics 2), Physics Concept Questions - Book 5 (Sound)	$13.4$

Table 2. Example topics recovered by TEA, each with its five highest-weight items.

n^{\mathrm{eff}}

is the effective number of users who support the topic (defined in Appendix I ). We omit gowalla , whose items are location identifiers without text.

Figure 2. Validation NDCG@10 of TEA against the item exponent $\alpha$ . For each $\alpha$ , the line is the largest value over the $20$ pairs of $\lambda$ and $p$ , and the dots are the individual pairs. Six panels in three rows of two, one per dataset, plotting the validation NDCG@10 of TEA against the item exponent alpha from -0.5 to 0.5. Each panel shows the largest value over the regularization parameter and the dropout probability as a line, the value of each of their 20 pairs as dots, and the selected alpha as a star, whose value is in the panel title. On every dataset, the line rises up to the selected alpha, which is between 0 and 0.25, and falls toward alpha equal to -0.5 and 0.5.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Model	Hyperparameter	books	cds	gowalla	ml-1m	retail	steam
WMF	number of factors	128	128	64	128	128	64
	regularization	0.01	0.01	0.01	0.01	0.01	0.01
LightGCN	learning rate	0.001	0.01	0.01	0.001	0.001	0.001
	number of layers	2	2	2	2	3	3
SLIMElastic	regularization	0.01	0.01	0.01	0.1	0.1	0.1
	L1 ratio	0.01	0.01	0.001	0.01	0.01	0.001

Appendix

Table 3. Selected hyperparameters of each method on each dataset.

Recommender Systems

Recommender systems predict and rank items for people using interaction histories, content, context, and feedback.

See all 516 papers

On the Regularization Landscape for the Linear Recommendation Models
Sep 10, 2026Dong Li, Zhenming Liu, Ruoming Jin +4Nuclear Norm Minimization Low-Rank Approximation
Kent State University, USA · College of William and Mary, USA · iLambda, USA
Generalizing Linear Autoencoder Recommenders with Decoupled Expected Quadratic Loss
Mar 8, 2026Ruixin Guo, Xinyu Li, Hao Zhou +2Recommender Systems Autoencoders
Kent State University · Auburn University
Enhancing High-order Interaction Awareness in LLM-based Recommender Model
Sep 30, 2024Xinfeng Wang, Jin Cui, Fumiyo Fukumoto +1Sequential Recommendation Feature Interaction Modeling
Graduate School of Engineering University of Yamanashi, Kofu, Japan · Interdisciplinary Graduate School University of Yamanashi, Kofu, Japan

Sep 10, 2026J/K move · Enter open · S save

On the Regularization Landscape for the Linear Recommendation Models

Dong Li, Zhenming Liu, Ruoming Jin, Hao Zhou+3

Kent State University, USA · College of William and Mary, USA · iLambda, USA

Nuclear Norm Minimization Low-Rank Approximation Recommender Systems

Open paper AskPDF arXiv

Recently, a wide range of recommendation algorithms inspired by deep learning techniques have emerged as the performance leaders on several standard recommendation benchmarks. While these algorithms were built on different DL techniques (e.g., dropouts, autoencoder), they have similar performance and even similar cost functions. This paper studies whether the models' comparable performance are sheer coincidence, or they can be unified under a single framework. We find that all linear performance leaders effectively add only a nuclear-norm based regularizer, or a Frobenius-norm based regularizer. The former ones possess a (surprising) rigid structure that limits the models' predictive power but their solutions are low rank and have closed form. The latter ones are more expressive and more efficient for recommendation but their solutions are either full-rank or require executing hard-to-tune numeric procedures such as ADMM. Along this line of finding, we further propose two low-rank, closed-form solutions, derived from carefully generalizing Frobenius-norm based regularizers. The new solutions get the best of both nuclear-norm and Frobenius-norm world.

Abstract

Recommender Systems

Explore similar work

On the Regularization Landscape for the Linear Recommendation Models

Generalizing Linear Autoencoder Recommenders with Decoupled Expected Quadratic Loss

Enhancing High-order Interaction Awareness in LLM-based Recommender Model