cs.CLSep 2, 2026
SaveDo LLMs Choose Like Humans? Using Cognitive Theory to Evaluate LLM Decision-Making
Organizations: Harvard University · Kempner Institute, Harvard University · Technion - Israel Institute of Technology
Abstract
Large language models (LLMs) exhibit a range of human-like decision-making behaviors, but whether these reflect similar underlying mechanisms or surface-level mimicry remains unclear. We evaluate whether LLM context sensitivity aligns with a cognitive economic theory that explains human behavior through problem categorization and attention allocation. Across 12 open-source and commercial LLMs on a novel 140,000-trial product choice benchmark, context induces human-like shifts in choice and problem categorization, but does not reliably reweight attention between features like price and quality. Neither scale nor chain-of-thought reasoning reliably attenuates context sensitivity or generates human-like behavior. These results suggest that LLM decision mechanisms are distinct from human ones.
Figures & tables
Figure 1: Product choice questions evaluate LLM decisions against human behavior and cognitive theory. ( Left ) Hedonic and functional context cues, drawn from the consumer choice literature, frame a binary choice between standard and premium products. ( Right ) Given human behavior explained by Bordalo et al. (2026b) , we predict that hedonic framing shifts LLM choices towards the premium option, reweights attention from price to quality, and produces larger effects when baseline categorization is ambiguous.
| Cue Type | Hedonic Cue | Functional Cue | References |
| Purchase trigger | Exploring the category out of curiosity or considering an upgrade | Routine replacement or replenishment without overpaying | Bell et al. (2011) |
| Retail architecture | Browsing a curated specialty page with staff picks and quality notes | Browsing a marketplace comparison page sorted by price with filters | Koschmann and Isaac (2018) |
| Payment source | Paying with a gift card or credit, keeping any unspent balance | Paying from a checking account, keeping any unspent balance | Thaler (1999) |
| Justification | Recently earned a small reward; this feels like a reasonable treat | Ordinary purchase with no special reason; just a practical choice | Okada (2005) |
| Reasoning frame | Focused on why the product would improve the user experience | Focused on how to complete the purchase and the concrete tradeoffs | Liberman and Trope (1998) |
Table 1: Contextual purchasing cues. Each cue type includes a hedonic/functional pair drawn from the consumer choice literature. Claude Sonnet-4.6 instantiates these templates into concrete cues for each product, designed to shift consumption framing without changing product utility.
Figure 2: Models match human context effects on choices, but diverge on feature sensitivity. ( Left ) Difference in premium choice rates between hedonic and functional cues. Shaded region indicates human-like behavior for all panels—here, hedonic cues lead to more premium choices. See Appendix Figure 5 for the difference in premium choice rates relative to the neutral condition. ( Right ) Difference in standardized price (x-axis) and quality (y-axis) sensitivity between hedonic and functional cues, estimated by linear regression on choice logits (see Section 4 ). Shaded region denotes behavior where hedonic cues attenuate (negative) price sensitivity and amplify quality sensitivity. Percentages report the share of model-cue combinations in each quadrant. All bars are 95% CIs.
Figure 3: Context effects follow a categorization process: in-context evidence shifts categorization (left), and ambiguity amplifies cue effects (center, right). Panels pool across Gemma 3 (4B, 12B, 27B) and Qwen 3 (4B, 8B, 14B, 32B). ( Left ) Difference in premium choice rate relative to (midpoint) as a function of the number of hedonic cues among eight in-context examples. ( Center ) Difference in premium choice rate between hedonic and functional cues (de-meaned by model) binned by product hedonic-ness scores. ( Right ) Difference in (premium - standard logit) between hedonic and functional cues as a function of (premium - standard logit) in neutral choice problems, both standardized to neutral-SD units. All bars are 95% CIs.
Figure 4: Scale and reasoning do not reliably reduce context sensitivity. ( Left ) Difference in premium choice rate between hedonic and functional cues for Qwen 3 (paired non-thinking/thinking) and Gemma 3 (non-thinking) model families. ( Right ) Difference in standardized price (x-axis) and quality (y-axis) sensitivity between hedonic and functional cues for Qwen models in thinking/non-thinking modes. Shaded region denotes human-consistent behavior. All bars are 95% CIs.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Premium choice rates by model and cue condition on the baseline dataset , for all sixteen configurations (twelve models plus the four thinking modes; (T) denotes thinking mode). ( Top ) Difference in premium choice rate between hedonic (functional) cues and the neutral condition. ( Bottom ) Difference in premium choice rate between hedonic and functional cues, averaged across cue types. All bars are 95% CIs.
Figure 6: Cue effects on feature sensitivity under alternate regression specifications. The same quantity as Figure 2 (right), re-estimated with differences in feature values instead of log ratios: absolute differences (left; price sensitivity per dollar, quality sensitivity per rating point) and per-product-SD standardized differences (right). Percentages report the share of model-cue combinations in each quadrant.
Figure 7: Distribution of hedonic-ness scores across the 100 products. Scores from the GPT 5.4 judge ( Section A.5 ); 0 = purely functional, 100 = purely hedonic. The distribution is right-skewed (median 33), with labeled products marking representative points.
Figure 8: Effect of thinking mode on price and quality sensitivity, by cue type. Difference in standardized price (x-axis) and quality (y-axis) sensitivity between hedonic and functional cues for Qwen models in thinking/non-thinking modes. Shaded region denotes human-consistent behavior. Figure 4 (right) shows the justification and reasoning frame dimensions. All bars are 95% CIs.
| Fishing reel outdoor & sporting goods | price ratio 2.33 | ||
| Premium | Aluminum spinning reel with sealed drag. Aluminum alloy body and rotor, 6 ball bearings, machined aluminum spool with sealed multi-disc drag, suited for light saltwater fishing. | $69.99 |
| Standard | Graphite spinning reel, entry level. Graphite composite body and rotor, 3 ball bearings, aluminum spool with basic anti-reverse mechanism, suited for freshwater fishing. | $29.99 |
| Electric toothbrush personal care & grooming | price ratio 3.13 | ||
| Premium | Rechargeable oscillating toothbrush. Single oscillating speed, USB charging cable, plastic handle, one replacement brush head and two-minute timer. | $24.99 |
| Standard | Basic battery-powered toothbrush. Single-speed vibrating motor, AA-battery powered, fixed plastic handle, one nylon bristle brush head included. | $7.99 |
Table 2: Two example product pairs. Each row shows the option name (bolded) and description generated by Claude Sonnet 4.6, along with its price. Descriptions are length-matched and brand-free, and option order (A=premium vs. A=standard) is counterbalanced across trials.
| Dimension | Pole | Cue |
| Purchase trigger | Hed. | I’m shopping for a chef’s knife because I’ve been curious about the category and I’m considering an upgrade to my everyday cooking setup. |
| Func. | I’m shopping for a chef’s knife because my current one has worn down and I want a solid replacement without overpaying. | |
| Retail architecture | Hed. | I’m browsing for a chef’s knife on a curated kitchenware shop page where staff picks include quality notes on blade feel and durability. |
| Func. | I’m browsing for a chef’s knife on a marketplace comparison page sorted by price, with filters, unit costs, and price-match language displayed. | |
| Payment source | Hed. | I’m looking for a chef’s knife and will be paying with a gift card I received as a birthday present. Any unspent amount remains mine. |
| Func. | I’m looking for a chef’s knife and will be paying with money from my regular checking account. Any unspent amount remains mine. |
Table 3: Cue templates instantiated for one product (chef’s knife). Each of the five dimensions has a paired hedonic/functional cue. Cues for the remaining 99 products are generated analogously via Claude Sonnet 4.6. Appendix A.3 provides the full prompt template for choice questions.
| Family | Trials | H. | F. | N. |
| Baseline | 8,800 | 4,000 | 4,000 | 800 |
| Price sweep | 88,000 | 40,000 | 40,000 | 8,000 |
| Quality sweep | 44,000 | 20,000 | 20,000 | 4,000 |
| ICL mix | 36,000 | 0 | 0 | 36,000 |
| Total | 176,800 |
Table 4: Trial counts per model. “H.”/“F.”/“N.” are hedonic, functional, and neutral conditions. The same 800 option pairs are reused across all four experiment families, but not every model is evaluated in each experiment.
| Panel A: baseline tier ratings ( ) | ||||
| Tier | Label | Mean | SD | |
| 1 | budget | 30.4 | 9.2 | 100 |
| 2 | economy | 54.6 | 11.8 | 100 |
| 3 | midrange | 72.9 | 8.7 | 100 |
| 4 | premium | 84.8 | 4.3 | 100 |
| 5 | luxury | 89.8 | 3.7 | 100 |
Table 5: GPT 5.4 quality ratings. Each option is rated five times independently from 0 to 100 with a rubric that ignores price and brand and scores only descriptive attributes ( Section A.5 ). Panel A: mean rating per baseline tier across the 100 products ( ). Panel B: mean rating across the five quality-sweep perturbations.
| Model | Access | Backend | Decoding | Choice DV |
| Gemma 3 4B/12B/27B-IT | Open | Local ( transformers ) | greedy | first-token A/B logit |
| Qwen 3 4B/8B/14B/32B | Open | Local (SGLang) | greedy | first-token A/B logprob |
| Qwen 3 4B–32B (Thinking) | Open | Local (SGLang) | T=0.6, top-p 0.95, 4096-tok CoT | A/B logprob after CoT |
| GLM 5.2 | API | Fireworks (reasoning off) | T=0 | A/B logprob (quantized) |
| DeepSeek v4 Pro | API | Fireworks (reasoning off) | T=0 | A/B logprob (quantized) |
| Qwen 3.7 Plus | API | Fireworks (reasoning off) | T=0 | A/B logprob (quantized) |
Table 6: Models evaluated.
More in this topic
LLM Decision-Making
LLM decision-making studies how language models choose, act, seek information, and allocate resources under goals, uncertainty, constraints, and social pressures.
See all 229 papers