Keeping the Index Open: The Recommendation-Side Cost of Shared Search and Recommendation
Organizations: Amazon Web Services United States
Abstract
A shared search-and-recommendation index must score new items from features alone because search has no exploration slot. In a public log covering both surfaces over one catalog, of held-out query-search impressions show an item never previously shown or visited. For user-cold engagements, the feature-based tower serves this demand without measurable loss against sampled negatives ( Recall@20 versus warm). A lexical baseline reaches similar parity, while a full-catalog check remains statistically undecided. Dual-encoder retrieval therefore keeps the index open to new items, unlike an ID-softmax recommender that requires retraining. We price this openness on recommendation against six sequential baselines, each retrained and tuned through five rounds on corrected targets. A float32 timestamp bug had reordered leave-one-out targets for of users. On MovieLens-1M, warm accuracy trails the strongest retrained baseline by Recall@20 and NDCG@20. On MIND, the gap narrows to -- relative to the five strongest baselines, though the model ranks sixth of seven. Under strict zero-leakage cold-start evaluation, the content tower achieves Recall@20, the strongest retrained dedicated method () and a training-free floor, without cold-specific training. Exact full-softmax training raises Recall@20 by on MIND-small and on MovieLens-1M over sampled InfoNCE, but recomputes the full catalog each step and exhausts accelerator memory at K items. Approximate nearest-neighbor search explains none of the remaining gap, serving cost does not regress against ID-softmax retrieval, and a history-window sweep explains half the post-recipe remainder. Exact-quality training at catalog scale remains the open problem.
Figures & tables
| Model | Recall@20 | NDCG@20 | Echo@20 |
| GRU4Rec | 0.3687 0.0039 | 0.1799 | 3.11 |
| SASRec | 0.3893 0.0024 | 0.1876 | 3.87 |
| BERT4Rec | 0.3453 0.0024 | 0.1542 | 4.22 |
| SASRecF (genres) | 0.3851 0.0015 | 0.1911 | 3.33 |
| FDSA (genres) † | 0.3767 | 0.1742 | 4.12 |
| DIF-SR (genres) | 0.3944 0.0019 | 0.1883 | 3.99 |
| Model | Recall@20 | Seeds |
| DIF-SR | 0.3494 0.0011 | 3 |
| FDSA | 0.3446 | 1 |
| SASRec | 0.3433 0.0008 | 3 |
| SASRecF | 0.3430 0.0003 | 3 |
| GRU4Rec | 0.3396 0.0016 | 3 |
| BERT4Rec | 0.3275 0.0055 | 3 |
| Model | Cold R@20 | Warm R@20 | Overall |
| Floors | |||
| Last-item text cosine (no training) | 0.058 | 0.018 | — |
| MF (ID-based CF, tuned) | 0.031 0.004 | 0.276 0.001 | 0.125 |
| Dedicated cold-start baselines (tuned) | |||
| DropoutNet (meta) | 0.094 0.013 | 0.241 0.001 | 0.109 |
| Heater (meta) | 0.106 0.005 | 0.256 0.002 | 0.116 |
Recommender Systems
Recommender systems predict and rank items for people using interaction histories, content, context, and feedback.
See all 514 papers