When Should Decision Models Compare Candidates?
Organizations: William & Mary · International School of Krakow · University of California, San Diego · University of California, Davis ·
Abstract
Language-model agents frequently select among candidate actions, such as tools and web elements. Should decision models score candidates independently or allow their scores to depend on other candidates? We investigate this question by comparing three approaches: Pointwise scoring, Setwise scoring, and Candidate Attention. Across five agent-decision benchmarks, Pointwise scoring remains competitive, while Setwise scoring performs better on some tool-selection and binary-decision tasks. Candidate Attention does not consistently improve upon Pointwise scoring, despite allowing cross-candidate interaction. Using two controlled synthetic tasks, we further examine whether models learn to use candidate-set information. With targeted relational training, Setwise models learn candidate dependencies and generalize to shifted inputs, while Candidate Attention shows weaker transfer. Our results show that access to other candidates does not guarantee effective comparison, and that its benefits depend on both task requirements and training.
Figures & tables
| Method | Candidate encoding | LM Backbone input | LM Backbone fine-tuning | Additional head | Scoring rule |
|---|---|---|---|---|---|
| 1. Setwise LM (untuned) | Joint | No | No | Answer-label logits | |
| 2. Setwise LM (fine-tuned) | Joint | Yes | No | Answer-label logits | |
| 3. Setwise Pointer | Joint | Yes | Yes | Pointer head | |
| 4. Pointwise Yes/No | Independent | Each separately | No | No | Yes probability |
| 5. Pointwise MLP | Independent | Each separately | Yes | Yes | Shared MLP |
| 6. Pointwise Linear | Independent | Each separately | Yes | Yes | Shared linear head |
| Setwise | Pointwise | |||||||||
| Dataset | Subtask | LM (untuned) | LM (tuned) | Pointer | Yes/No | MLP | Linear | Pointer | Cand. Attn. | |
| MetaTool | Scenario | 1,450 | 12.62 | 84.48 | 84.97 | 45.66 | 82.28 | 80.48 | 80.41 | 80.48 |
| Similar | 802 | 0.00 | 76.68 | 75.94 | 51.50 | 74.19 | 74.81 | 74.19 | 74.31 | |
| Reliability | 802 | 100.00 | 94.89 | 85.16 | 0.00 | 72.44 | 87.16 | 60.10 | 76.68 | |
| Overall | 3,054 | 32.25 | 85.17 | 82.65 | 35.20 | 77.57 | 80.75 | 73.44 | 77.87 | |
| BFCL | Choice | 835 | 35.45 | 93.41 | 60.84 | 55.21 | 91.14 | 90.30 | 88.98 | 91.02 |
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| Architecture | Candidate access | Readout | Head configuration | Head params |
|---|---|---|---|---|
| Setwise LM | Joint | Option-label logits | Frozen LM output layer | 0 |
| Setwise Pointer | Joint | Query–option dot product | 2,048-to-256 projections | 1,053,184 |
| Pointwise Yes/No | Independent | Normalized Yes/No logits | Frozen LM output layer | 0 |
| Pointwise MLP | Independent | Scalar score | Width 256; FFN width 1,024 | 1,056,257 |
| Pointwise Linear | Independent | Scalar score | Layer norm plus linear map | 6,145 |
| Pointwise Pointer | Independent | Query–candidate dot product | 2,048-to-256 projections | 1,053,184 |
| ID dev | OOD dev | Source | ||||
| Training | Method | Acc | Pair | Acc | Pair | Acc |
| Untuned | Setwise LM | 23.5 | 7.7 | 23.0 | 5.7 | 38.7 |
| S100 | Setwise LM † | 25.8 | 3.3 | 22.5 | 6.7 | 85.9 |
| Pointwise MLP | 12.9 | 0.0 | 21.8 | 0.0 | 85.6 | |
| Cand. Attn. | 11.9 | 0.0 | 18.5 | 0.0 | 85.5 | |
| S90R10 | Setwise LM † | 99.7 | 99.3 | 90.0 | 83.3 | 85.2 |
| ID dev | OOD dev | OOD test | OOD resample ∗ | |||||
| Method | Acc | Pair | Acc | Pair | Acc | Pair | Acc | Pair |
| No task-specific training | ||||||||
| Setwise LM | 23.5 | 7.7 | 23.0 | 5.7 | 22.2 | 6.7 | 23.7 | 6.2 |
| S100 | ||||||||
| Setwise LM (seed 0) | 25.8 | 3.3 | 22.5 | 6.7 | 27.9 | 7.5 | 26.2 | 8.2 |
| Pointwise MLP | 12.9 0.8 | 0.0 0.0 | 21.8 4.1 | 0.0 0.0 | 25.6 4.5 | 0.0 0.0 | 24.6 4.6 | 0.0 0.0 |
| MetaTool | BFCL | Mind2Web, | Mind2Web, | Open-Jev | JevBench | |||
| Method | Overall | Overall | Retrieved | All | Retrieved | All | Overall | Overall |
| S100, seed 0 | ||||||||
| Setwise LM (fine-tuned) | 85.17 | 83.64 | 26.31 | 18.75 | 8.52 | 7.32 | 60.89 | 72.90 |
| Pointwise MLP | 77.57 | 71.27 | 21.71 | 15.47 | 15.08 | 12.96 | 64.19 | 74.19 |
| Candidate Attention | 77.87 | 73.45 | 21.18 | 15.09 | 15.17 | 13.03 | 65.93 | 71.61 |
| S50R50, mean std over seeds 0–2 | ||||||||
| Predicted Yes (%) | ||||||||
|---|---|---|---|---|---|---|---|---|
| Dataset | Gold Yes (%) | Untuned | Setwise LM | MLP (E) | Linear (F) | Attention (G) | G E Acc. | |
| BFCL | 1,048 | 16.5 | 0.0 | 26.1 | 52.3 | 39.0 | 34.7 | |
| Open-Jev | 1,205 | 30.0 | 0.0 | 30.5 | 38.4 | 38.3 | 37.5 | |
| MetaTool reliability | 802 | 0.0 | 0.0 | 5.1 | 27.6 | 12.8 | 23.3 | |
| JevBench | 49 | 42.9 | 0.0 | 61.2 | 57.1 | 57.1 | 63.3 | |
LLM Decision-Making
LLM decision-making studies how language models choose, act, seek information, and allocate resources under goals, uncertainty, constraints, and social pressures.
See all 279 papers