Do Search-Augmented Recommenders Use the Latest Search Episode?
Organizations: Huawei Technologies Co., Ltd. Shanghai, China
Abstract
When a user returns to a recommendation feed after searching, does the latest search episode help rank the next item? Aggregate ranking gains leave open whether search-augmented recommenders use the latest search at prediction time or benefit from broader search state and representation learning. FocalSwap is a counterfactual audit that masks or replaces the latest episode or the full search history while holding the target, candidates, recommendation history, and trained scorer fixed. A planted-signal test and GenSAR's native search-item prediction task verify that FocalSwap detects known search dependence. In proxy analyses, KuaiSAR's routing gain falls below the shuffled threshold, and Qilin's clearest matched-search advantage appears at high taxonomy overlap. The primary comparisons for SESRec and UniSAR do not establish a positive ranking gain from the latest search episode alone. For the full search history, the estimated contribution depends on the control. On 66,139 shared instances across three UniSAR checkpoints, neutral masking consistently lowers mean reciprocal rank (RR). Replacing the history with observed searches matched on category and activity profile changes 44.1 to 44.8 percent of target ranks, yet the original history has little average advantage. In the primary assignment, matched replacements yield mean RR 0.002770 to 0.006604 higher than neutral masking across checkpoints. The adjusted intervals for this difference lie above zero under the primary assignment and all five alternative assignments per checkpoint. Search audits should specify which inputs changed and what replaced them, and report both how often ranks move and how much the original search helps.
Figures & tables
| Setting | Question | Finding |
|---|---|---|
| SESRec; UniSAR seeds 1–3 | Latest episode | The primary comparisons do not establish a positive ranking contribution. |
| UniSAR seeds 1–3 | Full history | Masking yields a positive contribution on 66,139 shared instances. Matched replacement gives smaller estimates. |
| Planted signal | Calibration | The audit detects routing opportunity, contribution, and specificity at full signal strength in all three runs. |
| GenSAR | Calibration | The original query raises mean RR@30 over matched replacement on 8,958 search-item instances. |
| KuaiSAR and Qilin | Auxiliary analyses | KuaiSAR falls below the shuffled threshold. Qilin clears the 95th-percentile threshold with BM25. Its strongest matched-search result is a preference lift on 6.1% of instances. |
| Dataset | Users | Valid episodes | Proxy episodes |
|---|---|---|---|
| KuaiSAR | 13,159 | 96,994 | 20,000 |
| Qilin | 1,544 | 6,135 | 6,135 |
| (a) Signed contribution under masking and matched replacement | |||||
|---|---|---|---|---|---|
| Latest episode | Full history on 66,139 shared instances | ||||
| Checkpoint | Mask (C0), 95% | Matched (C2), 97.5% | Mask, 95% | Matched, 97.5% | Mask matched, 97.5% |
| SESRec | n/a | n/a | n/a | n/a | |
| UniSAR seed 1 | |||||
| UniSAR seed 2 | |||||
| UniSAR seed 3 | |||||
Recommender Systems
Recommender systems predict and rank items for people using interaction histories, content, context, and feedback.
See all 573 papers