Asked to name a running shoe brand, language models say Nike. Asked to recommend one, they say Brooks. This paper asks where models learn that taste. Across 102 models and 42 brand categories, the brand a model picks when recommending differs from the brand it names in 26. The field moves together to a shared recommendation. The Named brands are market leaders, and the Picked brands are mostly niche and specialist. Three open training pipelines show where this forms. Pretraining already attaches the recommended brand to advice. Supervised fine-tuning teaches the model to answer and mostly leaves the pick unchanged. Preference optimization (DPO) teaches the model which answer is better, and the pick moves in all three pipelines, by 11 to 24 points. The preference data contains almost no brands. DPO moves the model's judgments toward a second set of associations even when the preferred brand is not in the preference data, and the effect reaches beyond brands. Asked to choose a sauce, models switch from ketchup to pesto. We do not know exactly how these preferences arise. We suspect they come from the judge models that decide which answers count as better during preference training. For brands, a recommendation depends less on what a model knows about them than on whether they fit this taste.
Figures & tables
Question
Consensus moved
Answer unchanged
Name
0 of 42
89%
Choose
6 of 42
73%
Recommend
16 of 42
57%
Recommend, then pick
26 of 42
31%
Table 1: From naming to picking (102 models, 42 categories). Consensus moved: categories where the most common answer across models differs from the most common answer under Name. Answer unchanged: share of a model’s answers that match its own most common answer under Name (under Name itself, how consistently it repeats itself).
Category
Named
Chosen
Recommended
Picked
AI assistant
Siri
Claude
ChatGPT
Claude
airline
Delta
Delta
Singapore Airlines
Singapore Airlines
bank
Chase
Chase
Chase
Ally Bank
beer
Heineken
Guinness
Guinness
Sierra Nevada
browser
Chrome
Firefox
Firefox
Firefox
camera
Canon
Canon
Canon
Sony
Table 2: The field’s brand for each question, in the 26 categories where the Picked brand differs from the Named brand.
Pipeline
Base
SFT
DPO
RL
SFT → DPO
OLMo 3 7B
58%
52%
76%
74%
+24 [+16, +33]
Tulu 3 8B
54%
58%
69%
68%
+11 [+6, +16]
OLMo 3.1 32B
63%
57%
73%
73%
+16 [+7, +25]
Table 3: Fixed-list pick by stage: probability of picking the panel’s Picked brand over its Named brand, averaged over both list orders and over the 26 categories where the two differ. Last column: the SFT-to-DPO change with a 90% bootstrap interval over categories.
Pipeline
Base
SFT
DPO
RL
SFT → DPO
OLMo 3 7B
-0.00
+0.12
+0.50
+0.44
+0.39 [+0.11, +0.67]
Tulu 3 8B
+0.08
+0.17
+0.28
+0.15
+0.11 [+0.00, +0.21]
OLMo 3.1 32B
+0.30
+1.30
+1.47
+1.47
+0.17 [+0.07, +0.27]
Table 4: Census Choose: how far “Choose” moves the odds toward the panel’s Choose answer, relative to “Name” (nats), over the 23 categories where the two differ. Last column: the SFT-to-DPO change, 90% interval.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Contrast
Moved
Floor
Kept
Floor
Name → Choose
14%
0%
73%
85%
Name → Recommend
38%
0%
57%
85%
free Name → two-turn pick
64%
1%
30%
84%
one name → free Name
7%
0%
80%
85%
one name → free Choose
29%
0%
61%
82%
one name → free Recommend
31%
0%
60%
82%
Appendix
Table 5: Verb and one-name effects against split-half floors. “Moved” is the share of categories whose consensus brand changes. “Kept” is how often a model’s modal answer survives the change.
Pipeline
Base
SFT
DPO
RL
SFT → DPO
OLMo 3 7B
+1.48
+1.08
+1.44
+1.47
+0.36 [+0.08, +0.62]
Tulu 3 8B
+1.76
+1.71
+2.47
+2.25
+0.76 [+0.35, +1.20]
OLMo 3.1 32B
+0.79
+3.97
+4.88
+4.66
+0.92 [+0.58, +1.28]
Appendix
Table 6: How far “Can you recommend…” moves the odds from the Named brand toward the Picked brand, relative to “Name…” (nats), by stage. Last column: the SFT-to-DPO change, 90% interval.
Pipeline
Stage
Sampled
Log-probability
OLMo 3 7B
SFT
34%
37%
OLMo 3 7B
DPO
36%
51%
OLMo 3 7B
RL
42%
50%
Tulu 3 8B
SFT
38%
42%
Tulu 3 8B
DPO
49%
51%
Tulu 3 8B
RL
55%
51%
Appendix
Table 7: The pick on the SFT stage’s own lists: sampled picks (5 per list) against log-probabilities.
SFT
DPO chosen
DPO rejected
Data
Named
Picked
Named
Picked
Named
Picked
OLMo 3 (Dolci)
43
8
5
3
5
1
Tulu 3
16
2
10
3
11
2
Appendix
Table 8: Brand recommendations in the post-training data, over the 26 categories: first brand named in SFT answers, and in the chosen and rejected answers of preference pairs.