Industrial recommenders rely on two model families that have evolved largely independently: feature-interaction models over multi-field user/item features, and sequential models over user-behavior histories. Production systems couple them only loosely. To unify the two, we present UniDot, a novel architecture for post-click conversion prediction built from the factorization-machine (FM) point of view: the embedding inner product---which powers collaborative filtering and lets a recommender generalize to unseen user--item pairs---is the same primitive as attention's query dot key scoring, so a single dot-product of tokens can underlie both feature interaction and sequence modeling. UniDot tokenizes non-sequential fields and multi-domain behavioral sequences into one shared token space and stacks a single macro-block in which a token-mixing bus and a sequence-retrieval bus (item tokens cross-attending the histories) run in parallel and exchange state each layer through an MLP-Mixer fusion, while an FM Highway carries explicit per-layer dot-product interactions around the residual stack directly to the classifier. The sequence side is embedded once per forward pass and shared by all consumers, bounding inference latency. Trained with a dual sparse/dense (Adagrad + Muon) optimizer, an auxiliary conversion-delay head, and multi-path mutual learning, UniDot finished as the runner-up on the Industrial track of the TAAC KDD Cup 2026.
Figures & tables
Figure 1. UniDot architecture (§ 4 ): two buses co-evolve through L macro-blocks, exchange state via the per-layer MLP-Mixer fusion, and the FM Highway routes explicit per-layer dot-products past the fuser to the classifier.
Rank
AUC ↑
Gap to #1
1
0.83254
—
2 (UniDot, ours)
0.83217
0.037%
(UniDot, single path) ⋆
0.83184
0.070%
3
0.83145
0.109%
4
0.83080
0.174%
5
0.83073
0.181%
Table 1. Final Industrial-track leaderboard (round 2, top 10). ⋆ Serving a single path of our all-data DML model (1 × inference cost) would still rank second—multi-path training pulls even a single model to a better minimum (§ 4.6 ).
Figure 2. One sequence trunk (§ 4.3 ).
Figure 3. MultiChannelSeqPool (§ 4.4 ): positions are gated into C channels by independent state-conditioned sigmoid weights.
Table 2. Training dynamics by epoch of the d=128 , N=2 multi-path run. Eval / Test (sel.) : the model-selection run ( remote , 10% held-out). Test (final) : the final submission, retrained on all data with EMA. LL = LogLoss (train over both paths, eval over their mean); “—” = not submitted.
#
Change
Test AUC
Δ
0
competition baseline
0.81398
—
1
+ UniDot
0.82500
+1.102
2
+ item-id hash embedding
0.82565
+0.064
3
tune token counts
0.82609
+0.044
4
tune batch size
0.82704
+0.095
5
+ more FM-Highway dot products
0.82722
+0.018
Table 3. Round-2 incremental improvements.
Variant
AUC ↑
LogLoss ↓
Δ AUC
Full UniDot (6 layers)
0.83657
0.2151
—
2 macro-layers
0.83600
0.2157
−0.057%
4 macro-layers
0.83603
0.2154
−0.054%
8 macro-layers
0.83624
0.2152
−0.033%
10 macro-layers
0.83615
0.2152
−0.042%
− FM Highway (all)
0.83530
0.2159
−0.127%
Table 4. Component ablations ( tiny mode, single-path, d=64 ; in-distribution held-out eval). Δ is AUC vs. the full model.
Configuration
Dense par.
GFLOP
Test AUC ↑
Δ
d=64 , sparse fold × 2
18.8M
5.9
—
no win
d=64 (baseline)
18.8M
5.9
0.82993
—
d=96
36.7M
12.4
0.83024
+0.031%
d=128
60.3M
21.2
0.83043
+0.050%
d=64 , 2-path
37.6M
11.7
0.83128
+0.135%
d=128 , 2-path
120.6M
42.5
0.83196
+0.203%
Table 5. Scaling study.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Quantity
Industrial track
Train / test samples
35M / 12M (2M, r1)
Columns (categories)
142 (7)
User Int / Dense fids
54 / 17
Item Int / Dense fids
17 / 4
Pre-trained emb fids (u/i)
7 / 4
Aligned per-position fids
10
Appendix
Table 6. Industrial-track dataset and schema (second round; first round in parentheses).
Mode
Data (train / eval)
GPUs
Role
tiny
4M / 1M, random hold-out
1
rapid design A/B
remote
35M / 10% random hold-out
4
full-scale validation
final
35M / no hold-out
4
submission (EMA weights)
Appendix
Table 7. Training/evaluation modes.
Group
Setting
Macro-layers L / mix-blocks-per-layer W
6 / 2
dmodel
128
Token-mix block
Wukong (parallel LCB + FMB, rank 32)
User NS / user emb tokens
8 / 2 ( × 7 fids) ⇒Tu=22
Item NS tokens (mix / retrieval bus)
4 / 16 (+ 4 emb fids × 2) ⇒Ti=12 , Tih=24
Fuse / classifier hidden mult
4 / 8
Appendix
Table 8. Submitted configuration behind the reported numbers.
Figure 6. Field-aware feature embedding (FAFE, § C.3 ): a DIN-style attention pool scores each value of a multi-value field against the ranking candidate, yielding a candidate-dependent field token; all other fields keep the static NCB pooling.
Figure 7. Data scaling ( tiny mode, single-path d=64 ): held-out AUC (left, blue ) and LogLoss (right, red ) vs. training examples ( log2 axis). Both improve log-linearly across the full 4M–32M range.