cs.IROct 16, 2026
SaveCandidate-dependent residual routing trades top-of-list accuracy for recall in sequential recommendation
Organizations: Department of Computer Science, The University of Texas at Austin, Austin, Texas, United States · Unaffiliated, Fairfax, Virginia, United States
Abstract
Candidate-dependent routing lets a sequential recommender weight a user's past behaviors differently for each item it scores. We test whether this flexibility helps by adding a small residual routing module to a self-attentive baseline, in two versions that differ only in what sets the routing weights: the user's history representation or the candidate item's embedding. Across five paired seeds on the REES46 cosmetics-shop dataset, with matched training for all three models, candidate-dependent routing leaves the primary metric, normalized discounted cumulative gain at cutoff ten (NDCG@10), essentially unchanged (mean relative change +0.003%). Secondary and exploratory analyses show that this flat average hides two effects that cancel. The candidate-dependent model places more relevant items in its top ten (recall +1.81%) but ranks the first one lower (mean reciprocal rank -1.09%), and most of its new hits land at positions six to ten. Its gains come from queries with at least one target among the user's last 50 events, while queries whose targets all lie outside that window get worse. In this form, candidate-dependent routing trades accuracy at the top of the list for recall further down.
Figures & tables
Figure 1: Scoring with the residual routing module. (A) The baseline (B) scores every candidate against the same history representation . (B) The two routed models use the same residual form and differ only in the routing vector that weights the behavior-stream means: the history representation for user-dependent routing (U) and the candidate embedding for candidate-dependent routing (C). In both, the residual term still depends on the candidate. The scalar starts at zero, and all parameters are trained.
| Seed | B | U | C | C/B (%) |
|---|---|---|---|---|
| 2026100601 | 0.343049 | 0.344798 | 0.343869 | +0.2391 |
| 2026100602 | 0.344962 | 0.343568 | 0.342639 | -0.6734 |
| 2026100603 | 0.346149 | 0.346769 | 0.346484 | +0.0968 |
| 2026100604 | 0.343179 | 0.345101 | 0.345436 | +0.6578 |
| 2026100605 | 0.345625 | 0.347483 | 0.344564 | -0.3068 |
Table 1: Normalized discounted cumulative gain at cutoff ten (NDCG@10) for each of the five paired seeds at epoch 12. B, baseline; U, user-dependent routing; C, candidate-dependent routing. Values are query means weighted to the stratum proportions of the test partition. Relative changes are computed before rounding.
Figure 2: Effect of candidate-dependent routing on NDCG@10, overall and by query type. (A) Relative change for the candidate-dependent model (C) against the baseline (B) in each of the five seeds, with their mean. The one-sided 95% confidence bound (+0.49%) lies below the +1% margin set in advance. (B) Mean paired relative change in each stratum (exploratory). A query is all-novel if none of its targets appears in its history (at most 50 earlier events), all-repeat if all of them do, and mixed otherwise. Percentages beside the stratum names are shares of the test population; each stratum contributes 1,536 evaluated queries. Lines run from zero to each estimate and are not confidence intervals.
| Metric @10 | B | U | C | C/B (%) | Seeds favoring C |
|---|---|---|---|---|---|
| NDCG | 0.344592 | 0.345544 | 0.344598 | +0.003 | 3/5 |
| Recall | 0.429810 | 0.434808 | 0.437542 | +1.806 | 5/5 |
| Hit rate | 0.449208 | 0.454164 | 0.456819 | +1.701 | 5/5 |
| MRR | 0.329954 | 0.328900 | 0.326372 | -1.085 | 0/5 |
Table 2: Ranking metrics at cutoff ten. B, baseline; U, user-dependent routing; C, candidate-dependent routing; NDCG, normalized discounted cumulative gain; MRR, mean reciprocal rank. Model means average the five seed-level weighted means, and relative changes average the five paired changes. NDCG@10 is the primary metric; the others are secondary. Supplemental Tables S3 and S4 give seed-level results at cutoffs 10 and 20.
| Position | Baseline B | Candidate C | C B |
|---|---|---|---|
| 1 | 0.266793 | 0.257033 | -0.009761 |
| 2 | 0.029430 | 0.034616 | +0.005186 |
| 3 | 0.014428 | 0.017917 | +0.003489 |
| 4–10 | 0.033942 | 0.035034 | +0.001092 |
| Total | 0.344592 | 0.344598 | +0.000006 |
Table 3: Contribution of each rank position to NDCG@10 (exploratory). Each entry is the weighted mean over queries of the discounted gain from a target at that position, divided by the query’s ideal discounted cumulative gain, then averaged over the five seeds (absolute NDCG scale). The columns sum to the model means, so the C B column sums to the difference in means, not to the mean paired relative change.
Figure 3: Candidate-dependent routing and cross-stream attention in the earlier Transformer models. Points are relative changes in NDCG@10 between seed-averaged model means; bars are exploratory 95% bootstrap intervals that resample users and paired seeds. Candidate-dependent routing starts the routing token from the candidate embedding instead of a history summary, with streams unable to attend to one another directly. Cross-stream attention lets the streams attend to one another in the candidate-dependent model. On REES46, both models in the routing comparison were retrained, and the cross-stream comparison pairs a retrained model with an original one.
| Dataset | Training | Validation | Test | Catalog items |
|---|---|---|---|---|
| REES46 | 1,761,398 | 423,081 | 507,957 | 40,749 |
| RetailRocket | 787,209 | 246,408 | 299,730 | 225,101 |
| KuaiSAR-small | 1,013,508 | 185,227 | 205,745 | 1,151,617 |
Table S1: Processed query counts and ranking catalog sizes. Counts precede selection of the evaluation cohorts. The catalog contains training target items and excludes reserved tokens.
| Seed | C B | C U | U B | C/U (%) | U/B (%) |
|---|---|---|---|---|---|
| 2026100601 | +0.00082011 | -0.00092968 | +0.00174979 | -0.2696 | +0.5101 |
| 2026100602 | -0.00232286 | -0.00092863 | -0.00139423 | -0.2703 | -0.4042 |
| 2026100603 | +0.00033500 | -0.00028591 | +0.00062090 | -0.0824 | +0.1794 |
| 2026100604 | +0.00225743 | +0.00033504 | +0.00192240 | +0.0971 | +0.5602 |
| 2026100605 | -0.00106033 | -0.00291821 | +0.00185788 | -0.8398 | +0.5375 |
Table S2: Paired NDCG@10 contrasts for all five seeds. Absolute differences use the NDCG scale; relative differences are percentages. C/B is the primary contrast; C/U and U/B are descriptive.
| Model / seed | NDCG@10 | Recall@10 | MRR@10 | Hit rate@10 |
|---|---|---|---|---|
| B / 2026100601 | 0.343049 | 0.425816 | 0.329085 | 0.445334 |
| B / 2026100602 | 0.344962 | 0.429973 | 0.330582 | 0.449318 |
| B / 2026100603 | 0.346149 | 0.434908 | 0.330473 | 0.454692 |
| B / 2026100604 | 0.343179 | 0.426366 | 0.328928 | 0.445636 |
| B / 2026100605 | 0.345625 | 0.431984 | 0.330703 | 0.451058 |
| U / 2026100601 | 0.344798 | 0.438126 | 0.327147 | 0.457117 |
Table S3: Weighted query metrics at cutoff 10 for every epoch-12 checkpoint in the paired experiment. B denotes the baseline, U user-dependent routing and C candidate-dependent routing.
| Model / seed | NDCG@20 | Recall@20 | MRR@20 | Hit rate@20 |
|---|---|---|---|---|
| B / 2026100601 | 0.359525 | 0.490730 | 0.333617 | 0.509691 |
| B / 2026100602 | 0.359072 | 0.486571 | 0.334577 | 0.507937 |
| B / 2026100603 | 0.360277 | 0.491965 | 0.334308 | 0.511388 |
| B / 2026100604 | 0.358354 | 0.487403 | 0.333085 | 0.506862 |
| B / 2026100605 | 0.359533 | 0.488003 | 0.334495 | 0.507147 |
| U / 2026100601 | 0.357951 | 0.490254 | 0.330795 | 0.509644 |
Table S4: Weighted query metrics at cutoff 20 for every epoch-12 checkpoint. These outcomes are descriptive secondary measures.
| Contrast | Metric | Mean difference | Relative (%) | Positive seeds |
|---|---|---|---|---|
| C/B | NDCG@10 | +0.000006 | +0.003 | 3/5 |
| C/B | Recall@10 | +0.007732 | +1.806 | 5/5 |
| C/B | Hit rate@10 | +0.007611 | +1.701 | 5/5 |
| C/B | MRR@10 | -0.003582 | -1.085 | 0/5 |
| C/U | NDCG@10 | -0.000945 | -0.273 | 1/5 |
| C/U | Recall@10 | +0.002734 | +0.632 | 5/5 |
Table S5: Overall descriptive contrasts at cutoff 10. Relative effects are the mean of five paired seed ratios, not ratios of the model means. A positive seed favors the numerator model.
| Stratum | Contrast | Mean difference | Relative (%) | Positive seeds |
|---|---|---|---|---|
| all-novel | C/B | -0.005092 | -7.186 | 0/5 |
| all-novel | C/U | -0.002650 | -3.792 | 0/5 |
| all-novel | U/B | -0.002442 | -3.430 | 1/5 |
| all-repeat | C/B | +0.004879 | +0.764 | 5/5 |
| all-repeat | C/U | +0.000517 | +0.082 | 3/5 |
| all-repeat | U/B | +0.004362 | +0.682 | 5/5 |
Table S6: Exploratory NDCG@10 contrasts within repeat strata. Novelty is defined against the retained history of at most 50 strictly prior events. Relative effects average the five paired seed ratios. Positive seeds favor the first model in each contrast.
| Model / seed | Train (min) | Train peak (MiB) | Evaluate (s) | Eval. peak (MiB) |
|---|---|---|---|---|
| B / 2026100601 | 55.37 | 540.4 | 31.74 | 130.1 |
| B / 2026100602 | 50.52 | 540.4 | 28.23 | 130.1 |
| B / 2026100603 | 67.86 | 540.4 | 33.44 | 130.1 |
| B / 2026100604 | 54.84 | 540.4 | 32.04 | 130.1 |
| B / 2026100605 | 47.05 | 540.4 | 29.22 | 130.1 |
| U / 2026100601 | 52.21 | 551.3 | 37.63 | 140.2 |
Table S7: Observed training and evaluation costs. Peak memory is allocated GPU tensor memory. Scheduling and device contention differed among runs, so timing differences cannot be attributed solely to the models.
| Model / seed | Epoch 1 | Epoch 2 | Epoch 3 | Epoch 4 | Epoch 5 | Epoch 6 |
|---|---|---|---|---|---|---|
| B / 2026100601 | 3.1383 | 2.2913 | 1.7979 | 1.5140 | 1.3450 | 1.2320 |
| B / 2026100602 | 3.1300 | 2.2575 | 1.7734 | 1.5014 | 1.3376 | 1.2266 |
| B / 2026100603 | 3.1225 | 2.2634 | 1.7902 | 1.5112 | 1.3412 | 1.2268 |
| B / 2026100604 | 3.1219 | 2.2698 | 1.7850 | 1.5087 | 1.3407 | 1.2276 |
| B / 2026100605 | 3.1250 | 2.2803 | 1.7856 | 1.5053 | 1.3380 | 1.2268 |
| U / 2026100601 | 3.1150 | 2.2797 | 1.7758 | 1.4922 | 1.3263 | 1.2168 |
Table S8: Mean training losses for epochs 1–6 of all fifteen fits. Each epoch contains 1,761,398 queries and 3,441 optimizer updates.
| Model / seed | Epoch 7 | Epoch 8 | Epoch 9 | Epoch 10 | Epoch 11 | Epoch 12 |
|---|---|---|---|---|---|---|
| B / 2026100601 | 1.1491 | 1.0839 | 1.0306 | 0.9834 | 0.9428 | 0.9080 |
| B / 2026100602 | 1.1458 | 1.0817 | 1.0281 | 0.9835 | 0.9428 | 0.9077 |
| B / 2026100603 | 1.1439 | 1.0777 | 1.0242 | 0.9789 | 0.9379 | 0.9039 |
| B / 2026100604 | 1.1455 | 1.0794 | 1.0270 | 0.9810 | 0.9402 | 0.9060 |
| B / 2026100605 | 1.1441 | 1.0794 | 1.0258 | 0.9802 | 0.9411 | 0.9062 |
| U / 2026100601 | 1.1361 | 1.0728 | 1.0208 | 0.9751 | 0.9355 | 0.9012 |
Table S9: Mean training losses for epochs 7–12 of all fifteen fits. The final checkpoint was specified before the experiment. Training loss alone does not show that ranking performance has stopped improving.
| Dataset | Family | Width | Heads | Seq. layers | Dropout | Learning rate |
|---|---|---|---|---|---|---|
| RetailRocket | Transformer | 64 | 2 | 1 | 0.0 | 0.001 |
| REES46 | Transformer | 64 | 2 | 1 | 0.0 | 0.001 |
| KuaiSAR-small | Transformer | 64 | 2 | 2 | 0.1 | 0.0003 |
Table S10: Training configurations used in the retrospective Transformer analysis, taken from the saved input summaries. Seq. layers denotes the number of causal sequence-encoder layers. Each model uses two query–stream attention blocks and weight decay 0.0001.
| Dataset | Model | Seeds | NDCG@10 |
|---|---|---|---|
| KuaiSAR-small | Candidate-dependent, full | 5 | 0.002266 |
| KuaiSAR-small | Candidate-dependent, local | 5 | 0.002188 |
| KuaiSAR-small | User-dependent, local | 5 | 0.001587 |
| REES46 | Candidate-dependent, full | 5 | 0.298720 |
| REES46 | Candidate-dependent, local | 5 | 0.312651 |
| REES46 | User-dependent, local | 5 | 0.247203 |
Table S11: Nine model means supporting the six retrospective Transformer contrasts. Each entry averages five seed-specific weighted query means. REES46 user-dependent and candidate-dependent local models use replacement checkpoints.
| Dataset | Contrast | Historical interval (%) | Revised interval (%) |
|---|---|---|---|
| KuaiSAR-small | Conditioning | [+8.99, +66.07] | [+1.78, +98.26] |
| KuaiSAR-small | Cross-stream attention | [-19.43, +27.64] | [-22.29, +38.93] |
| REES46 | Conditioning | [+22.63, +29.96] | [+21.48, +31.57] |
| REES46 | Cross-stream attention | [-7.83, -1.29] | [-7.98, -1.12] |
| RetailRocket | Conditioning | [-6.62, +16.49] | [-6.74, +17.46] |
| RetailRocket | Cross-stream attention | [-0.50, +20.98] | [-0.52, +23.74] |
Table S12: Historical and revised exploratory 95% intervals for all six complete Transformer contrasts. Historical intervals resampled users independently for each seed draw and fixed the ratio denominator; revised intervals use the same sampled users across seeds and recompute both model means.
| Dataset | Family | Contrast | Current evidence |
|---|---|---|---|
| KuaiSAR-small | Occurrence | Conditioning | Excluded; paired ranks incomplete |
| KuaiSAR-small | Occurrence | Cross-stream attention | Excluded; paired ranks incomplete |
| KuaiSAR-small | Transformer | Conditioning | Reported; paired ranks complete |
| KuaiSAR-small | Transformer | Cross-stream attention | Reported; paired ranks complete |
| REES46 | Occurrence | Conditioning | Excluded; paired ranks incomplete |
| REES46 | Occurrence | Cross-stream attention | Excluded; paired ranks incomplete |
Table S13: Status of all twelve comparisons in the internal analysis plan dated 11 September 2026. All reported Transformer comparisons have five paired seed labels. Comparisons without complete paired query ranks are excluded. Occurrence is the original plan’s name for the excluded model family.
| Recall | Hit rate | NDCG | MRR | Recall seeds | Hit seeds | |
|---|---|---|---|---|---|---|
| 1 | -3.6655 | -3.6603 | -3.6603 | -3.6603 | 0/5 | 0/5 |
| 2 | -0.5986 | -0.7513 | -1.4785 | -2.0956 | 1/5 | 0/5 |
| 3 | +1.4463 | +1.3299 | -0.3198 | -1.2681 | 5/5 | 5/5 |
| 5 | +1.3752 | +1.2705 | -0.2859 | -1.2214 | 5/5 | 5/5 |
| 10 | +1.8062 | +1.7015 | +0.0027 | -1.0851 | 5/5 | 5/5 |
| 20 | +1.5688 | +1.4953 | +0.0195 | -1.0615 | 5/5 | 5/5 |
Table S14: Exploratory cutoff sensitivity from original historical ranks. Entries are mean paired C/B relative changes (percent). The last columns count seeds favoring C for recall and hit rate. All cutoffs use the original candidate sets, targets and stratum-count weights. NDCG@10 remains the primary outcome; no new primary cutoff is selected.
| Stratum | Share (%) | Recall | Hit rate | NDCG | MRR |
|---|---|---|---|---|---|
| All-novel | 50.9433 | -4.5490 | -4.6252 | -7.1862 | -9.6983 |
| All-repeat | 47.9298 | +3.2887 | +3.0563 | +0.7636 | -0.5539 |
| Mixed | 1.1269 | +17.9248 | +7.5018 | +11.5140 | +2.8232 |
Table S15: Exploratory C/B changes by query type at cutoff ten, using original ranks. Shares refer to the complete test partition; each evaluated stratum contains 1,536 queries. Metric changes are means of five paired seed-relative effects (percent). Every seed favors C for recall, hit rate and NDCG in repeat and mixed queries; every seed favors B for those metrics in all-novel queries.
| Paired outcome | Share (%) | Recall | NDCG | MRR |
|---|---|---|---|---|
| B only | 2.9002 | -0.028362 | -0.010923 | -0.005929 |
| C only | 3.6613 | +0.034963 | +0.014213 | +0.008377 |
| Both hit | 42.0206 | +0.001131 | -0.003284 | -0.006029 |
| Neither hit | 51.4179 | +0.000000 | +0.000000 | +0.000000 |
| Within queries hit by both models: | ||||
| C first hit earlier | 5.1862 | +0.000252 | +0.008747 | +0.011180 |
Table S16: Exploratory paired-query decomposition at cutoff ten from original saved ranks. Shares and absolute metric contributions average over five seeds using original query weights. The first four rows partition all query–seed observations. The final three rows partition the both-hit row; their shares are also expressed relative to all query weight.
| Score source | Mean paired C/B | One-sided 95% upper |
|---|---|---|
| Original GPU evaluation | +0.002698 | +0.490404 |
| New CPU diagnostic | +0.008627 | +0.526662 |
Table S17: Original primary result and separate CPU score diagnostic for NDCG@10. Effects and bounds are percentages and retain the same five-seed calculation. All tested exact-tie policies give identical cutoff-ten metrics on the CPU vectors. The CPU result does not replace the historical estimate or reconstruct historical GPU tie behavior.
| Recall | Hit rate | NDCG | MRR | NDCG seeds | |
|---|---|---|---|---|---|
| 1 | -12.3233 | -12.3293 | -12.3293 | -12.3293 | 0/5 |
| 2 | -4.0882 | -3.9869 | -6.2650 | -7.1758 | 0/5 |
| 3 | -2.8691 | -2.6988 | -5.1816 | -6.2535 | 1/5 |
| 5 | -5.2148 | -5.1563 | -6.1217 | -6.7087 | 1/5 |
| 10 | -0.2634 | -0.2022 | -3.1374 | -5.0403 | 0/5 |
| 20 | -1.8052 | -1.7661 | -3.5464 | -5.1044 | 0/5 |
Table S18: Exploratory changed-task diagnostic on new CPU scores. Only targets absent from the recorded history are retained, all recorded-history competitors are excluded, and 3,072 original queries with at least one novel target are evaluated. Original weights are renormalized over this subset. Entries are mean paired C/B relative changes (percent). The last column counts seeds favoring C for NDCG. This is neither a cold-start experiment nor a replacement for the original evaluation.
More in this topic
Sequential Recommendation
Sequential recommendation predicts what a user will choose next from ordered interactions, adapting to changing interests and context.
See all 131 papers