Language models are increasingly asked where real actors stand: what a party's platform prioritises, what an institution has committed to. Such judgments are gradable, carrying both an order and a magnitude. Nearly every benchmark that tests them asks about one entity at a time, then reads that score as the model's knowledge of the domain. Order and magnitude, however, need not be acquired together. Text about social entities carries far more comparison than measurement, so a model may hold a dependable ordering with no anchored scale beneath it. We ask whether a model can place a documented social position on a scale or only order the actors it compares, and what that means for how such knowledge is measured. We test this on party positions in Japan and South Korea, where the Comparative Manifesto Project's coded data supplies a rare external answer key. Asked to order four parties, models beat the strongest fixed-answer baseline by more than a factor of two; asked to place one party on the same scale, they fall below it. The ordering survives inside those failed placements, and supplying the coded values lifts placement to near ceiling. Comparative and absolute questions about the same fact therefore measure different things here, and for party issue salience a benchmark built from one cannot speak for the other.
Figures & tables
Figure 1: The life cycle of the data. Coded CMP percentages are quantised into ordinal buckets, giving each benchmark cell a gold level and a tie-aware gold ranking; the same cell is then put to a model two ways. The two lanes are separate API calls with no shared history, and both ask a four-alternative question: four emphasis levels in a , four named parties in b . Each lane starts from a structured prompt; the model answers in free prose that ends with one machine-readable tagged line (Section 4.2 ), and both lanes are parsed identically. Dashed paths are the only ones that read the gold column.
Model
Corr.
Part.
Rival
None
NoKn.
DeepSeek-V3
26.4
44.4
4.2
20.8
4.2
Gemini-Flash
25.5
30.9
7.3
36.4
0.0
gpt-oss-120b
25.0
38.9
2.8
20.8
12.5
Cohere-A
22.2
31.9
5.6
25.0
15.3
gpt-oss-20b
21.1
35.2
5.6
32.4
5.6
Claude-Sonnet
20.8
22.2
0.0
2.8
54.2
Table 1: Absolute-placement verdicts by model (%), sorted by accuracy. Corr.=Correct, Part.=one bucket off, Rival/None=two or more buckets off and matching a rival’s bucket / no party’s bucket, NoKn.=No knowledge. Models differ far more in calibration than in accuracy: Claude Sonnet declines on over half of items while others rarely do ( χ2=126.7 , p<10−24 , Cramér’s V=0.51 ).
Task and scoring
Model
Base
Ratio
Order four parties
58.1
22.2
2.62 ×
Place one, exact
22.4
37.9
0.59 ×
Place one, ± 1 bucket
58.8
80.7
0.73 ×
Table 2: The dissociation. “Base” is the best single constant answer under that task’s own scoring rule: always naming one fixed party (ordering) or always giving one fixed bucket (placement). The two 58% figures sit on opposite sides of their baselines, which is why raw accuracies must not be compared directly.
Measure
Ungr.
Gr.
Sig.
Absolute placement
22.4
95.6
—
Self-consistency (highest)
86.2
96.8
p=0.039
Self-consistency (lowest)
77.5
96.9
p=0.0006
Table 3: Effect of grounding. Supplying the coded values lifts placement to near ceiling and significantly raises agreement between a model’s comparative answer and the ordering implied by its own absolute answers (Fisher’s exact). Self-consistency n : 80/80 ungrounded, 63/64 grounded.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Work
Task
Ground truth
Source text given?
Liusie et al. (2024) , Qin et al. (2024)
Judge quality of a supplied text, pairwise vs pointwise
Human preference
Yes
Licht et al. (2025)
Measure a scalar construct in a supplied text
Expert-coded scales
Yes
Lehmann et al. (2026)
Compare two entities on a numerical attribute
Recorded attribute values
No (value is in text corpus)
Calderon et al. (2026)
Recall an encoded fact
Web-grounded fact set
No
Di Leo et al. (2025) , Caliskan et al. (2025) , Benoit and Laver (2026)
Estimate party position or salience
CMP / expert surveys
Yes
This work
Order vs place a named party on a coded issue
CMP coded manifesto data
No
Appendix
Table 4: The closest prior work by task, ground truth, and whether the text to be judged is supplied in the prompt. The last column is the axis this paper occupies alone: the model must answer from parametric knowledge, against an external key, in two framings.
Measure
All 9
Variant 6
Anchor 3
Order highest
58.1
69.6
34.2
chance
25.0
25.0
25.0
Order lowest
48.3
38.5
68.4
tie-aware chance
35.3
25.0
56.6
Place, exact
22.4
21.9
23.6
best constant
37.9
41.9
66.2
Appendix
Table 5: Headline measures split by issue type (%, per408 excluded). Ordering beats chance on the variant issues (highest p≈10−16 , lowest p=0.006 ) but not on the anchor issues (highest p=0.13 , lowest p=0.094 ), where all parties share a level; placement stays below its constant baseline in every subset.
Test
p
Holm α
Holds
Calibration across models, χ2
<10−24
0.0056
yes
Order highest, binomial
3.0×10−14
0.0063
yes
Read-out SC highest, Fisher
0.0001
0.0071
yes
Grounding SC lowest, Fisher
0.0006
0.0083
yes
Order lowest, binomial (tie-aware)
0.0028
0.0100
yes
Grounding SC highest, Fisher
0.039
0.0125
no
Appendix
Table 6: Holm correction over the nine tests reported in Section 5 . The last row is a null result by design: the implied and direct orderings are expected to agree.
Vermont Complex Systems Institute University of Vermont Burlington, Vermont 05403 · Vermont Complex Systems Institute Department of Statistics University of Vermont Burlington, Vermont 05403