UniMixer: A Unified Architecture for Scaling Laws in Recommendation Systems

Mingming Ha; Guanchen Wang; Linxun Chen; Xuan Rao; Yuexin Shi; Tianbao Ma; Zhaojie Liu; Yunqian Fan

cs.IRApr 1, 2026

UniMixer: A Unified Architecture for Scaling Laws in Recommendation Systems

Authors: Mingming Ha, Guanchen Wang, Linxun Chen, Xuan Rao, Yuexin Shi, Tianbao Ma, Zhaojie Liu, Yunqian Fan,

Organizations: Kuaishou Technology, Beijing, China

Abstract

In recent years, the scaling laws of recommendation models have attracted increasing attention, which govern the relationship between performance and parameters/FLOPs of recommenders. Currently, there are three mainstream architectures for achieving scaling in recommendation models, namely attention-based, TokenMixer-based, and factorization-machine-based methods, which exhibit fundamental differences in both design philosophy and architectural structure. In this paper, we propose a unified scaling architecture for recommendation systems, namely UniMixer, to improve scaling efficiency and establish a unified theoretical framework that unifies the mainstream scaling blocks. By transforming the rule-based TokenMixer to an equivalent parameterized structure, we construct a generalized parameterized feature mixing module that allows the token mixing patterns to be optimized and learned during model training. Meanwhile, the generalized parameterized token mixing removes the constraint in TokenMixer that requires the number of heads to be equal to the number of tokens. Furthermore, we establish a unified scaling module design framework for recommender systems, which bridges the connections among attention-based, TokenMixer-based, and factorization-machine-based methods. To further boost scaling ROI, a lightweight UniMixing module is designed, UniMixing-Lite, which further compresses the model parameters and computational cost while significantly improve the model performance. The scaling curves are shown in the following figure. Extensive offline and online experiments are conducted to verify the superior scaling abilities of UniMixer.

Figures & tables

Figure 1: The scaling laws between AUC and Parameters for the present UniMixer/UniMixer-Lite and RankMixer architectures. The x-axis is presented on a logarithmic scale.

Figure 2: The UniMixer architecture for scaling laws in recommendation systems.

Figure 3: (a) The global mixing weights of different methods. (b) Equivalent parameterization of the rule-based TokenMixer.

Heterogeneous Feature Mixing	Local Mixing Pattern	Global Mixing Pattern $G(X,W_{G})$
Self-Attention	$XW_{V}$	$\text{softmax}(\frac{(XW_{Q})(XW_{K})^{\mathsf{T}}}{\sqrt{d}})$
Heterogeneous Attention	$X\tilde{W}_{V}$	$\text{softmax}(\frac{(X\tilde{W}_{Q})(X\tilde{W}_{K})^{\mathsf{T}}}{\sqrt{d}})$
TokenMixer	$X$	$G$
FM	$Y$	$XI(XI)^{\mathsf{T}}$

Table 1: The differences of attention-based, TokenMixer-based, FM-based methods under the unified theoretical framework.

Model	User Retention				Efficiency
Model	AUC $\uparrow$	$\Delta$ AUC $\uparrow$	UAUC $\uparrow$	$\Delta$ UAUC $\uparrow$	Params	FLOPs/Batch
Heterogeneous Attention Gui et al. (2023)	0.744577	–	0.733829	–	132.7M	1.68T
HiFormer Gui et al. (2023)	0.741685	-0.2892%	0.731086	-0.2743%	107.5M	1.37T
Wukong Zhang et al. (2024)	0.744477	-0.0100%	0.733849	0.0020%	107.1M	1.40T
FAT Yan et al. (2025)	0.744883	0.0306%	0.734280	0.0451%	138.4M	1.83T
RankMixer Zhu et al. (2025)	0.749329	0.4752%	0.738938	0.5109%	135.5M	1.68T

Table 2: Performance and efficiency of

\sim 100

M-parameter UniMixer and SOTA models in ad serving scenarios.

Figure 4: The scaling laws between AUC and Parameters/FLOPs for UniMixer-2-Blocks/UniMixer-Lite-2-Blocks and RankMixer architectures. The x-axis is presented on a logarithmic scale.

Setting	User Retention				Efficiency
Setting	AUC $\uparrow$	$\Delta$ AUC $\uparrow$	UAUC $\uparrow$	$\Delta$ UAUC $\uparrow$	Params	FLOPs/Batch
UniMixer	0.748464	–	0.738017	–	6.57M	0.16T
w/o Temperature Coefficient	0.746819	-0.1645%	0.736527	-0.1490%	6.57M	0.16T
w/o Symmetry Constraint	0.747891	-0.0573%	0.737447	-0.0570%	6.57M	0.16T
w/o Block-Specific Local Mixing Weight	0.748028	-0.0436%	0.737770	-0.0240%	6.53M	0.16T
w/o Model Warm-Up	0.747608	-0.0856%	0.737180	-0.0837%	6.57M	0.16T

Table 3: Ablation on components of UniMixer 6.57M.

Figure 5: $\bar{W}_{G}$ and $\bar{W}_{B}^{i}$ of UniMixer-Lite with different temperature coefficients. (a) $\bar{W}_{G}$ with $\tau=1$ ; (b) $\{\bar{W}_{B}^{i}\}_{i=1}^{6}$ with $\tau=1$ ; (c) $\bar{W}_{G}$ with $\tau=0.05$ ; (d) $\{\bar{W}_{B}^{i}\}_{i=1}^{6}$ with $\tau=0.05$ .

Basis Number	User Retention				Efficiency
Basis Number	AUC $\uparrow$	$\Delta$ AUC $\uparrow$	UAUC $\uparrow$	$\Delta$ UAUC $\uparrow$	Params	FLOPs/Batch
b=2	0.749228	–	0.738776	–	4.968M	0.161T
b=4	0.750230	0.1002%	0.739876	0.1100%	4.973M	0.161T
b=8	0.750283	0.1055%	0.739854	0.1078%	4.98M	0.161T
Rank	User Retention				Efficiency
Rank	AUC $\uparrow$	$\Delta$ AUC $\uparrow$	UAUC $\uparrow$	$\Delta$ UAUC $\uparrow$	Params	FLOPs/Batch

Table 4: Effects of the basis number, rank, and UniMixer block number in UniMixing-Lite.

Figure 6: The scaling curves between AUC and Parameters/FLOPs for RankMixer and UniMixer/UniMixer-Lite with 2 blocks and 4 blocks.

Recommender Systems

Recommender systems predict and rank items for people using interaction histories, content, context, and feedback.

See all 514 papers

Expand More, Shrink Less: Shaping Effective-Rank Dynamics for Dense Scaling in Recommendation
May 22, 2026Guoming Li, Shangyu Zhang, Junwei Pan +7Representation Collapse Recommender Systems
The Hong Kong University of Science and Technology (Guangzhou) Guangzhou, China · Tencent Inc. Shenzhen, China
UniScale: Synergistic Entire Space Data and Model Scaling for Search Ranking
Mar 25, 2026Liren Yu, Caiyuan Li, Feiyi Dong +5Learning to Rank E-Commerce Search
Taobao & Tmall Group of Alibaba Hangzhou, China · Taobao & Tmall Group of Alibaba Beijing, China
MuonRec: Shifting the Optimizer Paradigm Beyond Adam in Scalable Generative Recommendation
Feb 28, 2026Rong Shan, Aofan Yu, Bo Chen +7Recommender Systems Muon Optimizer
Shanghai Jiao Tong University, Shanghai, China · Kuaishou Technology, Beijing, China

May 22, 2026J/K move · Enter open · S save

Expand More, Shrink Less: Shaping Effective-Rank Dynamics for Dense Scaling in Recommendation

Guoming Li, Shangyu Zhang, Junwei Pan, Wentao Ning+6

The Hong Kong University of Science and Technology (Guangzhou) Guangzhou, China · Tencent Inc. Shenzhen, China

Representation Collapse Recommender Systems

Open paper AskPDF arXiv

Scaling recommendation models is a central challenge in recommender systems. Recently, RankMixer has emerged as an effective solution, operating on a unified token representation and alternating between token mixing and per-token feedforward networks (P-FFNs) to achieve scalable performance. However, RankMixer suffers from embedding collapse, where learned representations have low effective rank, limiting expressivity and underutilizing the expanded representation space. Through empirical analysis and theoretical insights, we identify rigid token mixing and P-FFN modules as the primary causes of this phenomenon, jointly inducing a damped oscillatory trajectory in effective-rank evolution across layers. To address it, we propose RankElastor, a novel architecture that produces spectrum-robust representations with provable collapse mitigation. RankElastor introduces two components: (i) parameterized full mixing, which enables expressive token mixing with improved spectral robustness; and (ii) GLU-improved P-FFNs, which stabilize representation spectra through GLU-style FFN modules. Extensive experiments on large-scale industrial datasets demonstrate that RankElastor consistently improves recommendation performance, mitigates embedding collapse, and exhibits robust scaling behavior. Code is available at this GitHub repository: https://github.com/vasile-paskardlgm/RankElastor

Abstract

Recommender Systems

Explore similar work

Expand More, Shrink Less: Shaping Effective-Rank Dynamics for Dense Scaling in Recommendation

UniScale: Synergistic Entire Space Data and Model Scaling for Search Ranking

MuonRec: Shifting the Optimizer Paradigm Beyond Adam in Scalable Generative Recommendation