Verifiable, Articulable, and Tacit Components of Preference

Alexander Spangher; Sheldon S. Huang; Andreas Haupt; Noah D. Goodman; Diyi Yang; Daniel E. Ho; Sanmi Koyejo

cs.AIOct 2, 2026

Verifiable, Articulable, and Tacit Components of Preference

Authors: Alexander Spangher, Sheldon S. Huang, Andreas Haupt, Noah D. Goodman, Diyi Yang, Daniel E. Ho, Sanmi Koyejo

Organizations: Stanford University · University of Toronto

Abstract

What makes a short story gripping; a news article newsworthy; or a math proof elegant? These constructs resist articulation or verification; their meaning is at least partially tacit. However, modern AI models are improved primarily via articulated constitutions, rubrics and verifiers (i.e. in RLAIF and RLVR); tacit components of preferences are typically understudied. We introduce a large, labeled preference dataset CreativePreferences, containing 2.8M texts labeled by 317M human preference judgments across 7 creative domains, with 42 benchmark tasks. We model these labels with executable programs, rubric banks and densely trained models (V, A and VAT, respectively). We observe robust articulability gaps, VAT-VA; and verifiability gaps, VAT-V; we estimate upper and lower bounds for each gap with a novel measurement approach that discovers articulable and verifiable metrics, identifies spurious variables and estimates the value of undiscovered metrics using capture-recapture. These gaps occur across all domains, even in domains traditionally treated as fully verifiable: correctness-centered domains (i.e. mathematics and software engineering) and claim- and novelty-centric domains (i.e. news, patents, peer review). The size of the gap varies based on domain (e.g. peer review and creative writing have the largest articulability gaps) and widens as more people take part in the judgment, consistent with Collins' collective tacit knowledge. We show two consequences: (1) on human generations, the full model more closely matches human preferences, often in disagreement with articulated criteria, and (2) in an analogy to Goodhart's law, articulating preference shifts it away from the tacit dimension. Articulability and verifiability gaps are consequential; we give recommendations on when tasks can be prompted; how learning mechanisms might improve; and when to leave judgments with humans.

Figures & tables

Figure 1: Verification and articulation gaps in preference prediction. Each blue and green point in LHS boxplot is a single task of 42, example: Predict whether academic paper X receives a prize ). RHS barplot shows how we calculate: first, we measure an upper bound of task performance with dense full models , estimated-optimal executable programs and articulated rubrics . Gaps are robustly estimated and comparable across tasks: we deconfound datasets, match preference labels and robustly estimate missing metrics. Light dots are subsample of $b{=}3{,}000$ bootstrap draws / task.

Panel of Table 1 — Table 1: CreativePreferences : seven domains (columns) with three preference variable types (rows): gatekeeper verdicts are when a text is evaluated by a decision-maker who accepts or rejects it. Expert curation are preference signals transmitted by selecting top items from a pool. Community votes preferences are aggregations of individual votes on items. Labels that list several corpora contain more than one task; we collect 42 tasks across 37 label-types. For details, see App. A.1 .

Panel of Figure 2 — Figure 2: How we mine criteria, discover spurious variables, and estimate effects of undiscovered metrics. (1) First, we discover metrics using metric discovery algorithms. (2) Each metric is tested for whether it is quality or spurious: we test if each metric (a) captures signal external to text itself ( channel test , e.g. the “time of day” of post); (b) is correlated with other metrics ( path test , e.g. “first-person” language correlates to favored genre); or (c) is qualitatively not valuable ( LLM-audit ). (3) We estimate the upper bound of performance from metrics not yet found yet using capture-recapture ( Good, 1953 ; Chao, 1984 ; Efron and Thisted, 1976 ) .

	Top verifiable checks $V$	Top articulated criteria $A$
software code $y$ : PR merged	no pass $\to$ fail regressions tests flip fail $\to$ pass patch applies; suite executes	message says what changed, why structured subject/body, issue links single-purpose, reviewable change
creative writing $y$ : story upvotes	plot-arc shape proxy: paragraphing, rising action density form/medium-fit check worldbuilding-specificity check	deliberate contrasts of image and tone no stumbles read aloud; rhythm holds micro-tension keeps pages turning
math $y$ : answer accepted	lint checks machine-checkable-proof check formula-correctness check	every inference warranted, no jumps brief, inevitable, clear argument airtight steps, no hidden assumptions

Table 2: What a program can check vs. what can be articulated to a judge: top three features per task, highlighted proportional to explanatory strength. The

\mathcal{V}

checks: an LLM writes a deterministic program implementing the named construct (e.g. the plot-arc check scores paragraph structure and rising verb-density as in Tian et al. (2024) ). Math’s correctness checks fail to predict answer acceptance (AUC

\leq.55

, against

.65

–

.66

for its articulated criteria). For details, see Appendix C.2 .

Figure 3: Shares by preference channel. Recoverable preferences by preference channel; one point per task, faint points are bootstrap draws.

task	regret (full model $-$ articulated)	95% CI	disagreement odds
peer review: citations	$+.171$	$[.125,.216]$	$4.4:1$
peer review: accept/reject	$+.109$	$[.078,.140]$	$2.1:1$
math.SE: accepted answers	$+.067$	$[.029,.103]$	$1.4:1$
peer review: spotlight	$+.065$	$[.020,.109]$	$1.3:1$
creative writing: upvotes	$+.061$	$[.013,.107]$	$1.5:1$

Table 3: Selection regret on a sample of tasks where groups of human text responses (e.g. StackOverflow questions) containing both a chosen and an unchosen item (Appendix I.1 ). Regret as defined in § 5.2 : the full model’s hit rate minus the articulated model’s, averaged over groups. Intervals are 3,000-sample bootstraps; regret is positive in 13 of 16 tasks (sign test one-sided

p{=}.011

; Appendix I.1 ). Disagreement odds is when the instruments pick different items, how often the community’s actual winner is

\mathcal{VAT}

’s pick vs.

\mathcal{VA}

’s. When the two instruments disagree, for example, in peer review:citations,

\mathcal{VAT}

’s pick matches humans’ recorded choice 4.4:1.

Appendix figures & tables57 assets

Supplementary material from the paper’s appendix.

Appendix

Field	Dataset	Status	$n$	Notes (1s/0s; sub-corpora)
Coding	GitHub PR merged (enriched)	inst.	44,751	594 repositories; within-repository balance
	Stack Overflow accepted answer	inst.	16,001	5,972 / 10,029
Math	mathlib PR merged	inst.	8,494	7,509 / 985; canonical clean slice 7,956
	Math.SE accepted answer	inst.	13,001	4,960 / 8,041; 4,960 questions
Law & policy	Patent application granted	inst.	579,084	290,198 / 288,886; nested claim-level corpus 59,937
	Comment received a response	inst.	9,521	7,439 / 2,082; 1,814 dockets

Appendix

Table 4: CreativePreferences inventory, gatekeeper verdict channel. Status: inst. = V/A instruments scored; coll. = collected but not yet instrumented; live = acquisition still running. Counts are verified against the primary artifacts (ledger files, build manifests, raw stores). Construction, filters, and confound handling per dataset are in Appendix B .

Field	Dataset	Status	$n$	Notes (1s/0s; sub-corpora)
Coding	Competition editorial-approach match	inst.	6,353	AtCoder 2,495 $\cdot$ LeetCode 1,995 $\cdot$ CodeChef 995 $\cdot$ CF 868
	Stack Overflow bounty award	inst.	23,498	7,103 / 16,395; within-question
	Release-highlight picks	coll.	4,654	Rust 4,031-PR pool $\cdot$ mathlib 623 picks
Math	AoPS editorial approach	inst.	25,454	17,560 / 7,894; 5,202-row scored task
	Math.SE bounty award	inst.	23,972	8,503 / 15,469; within-question
Law & policy	Comment: change made	inst.	7,084	2,824 / 4,260; second label agency-agreement 5,046

Appendix

Table 5: CreativePreferences inventory, expert curation channel (conventions as in Table 4 ).

Field	Dataset	Status	$n$	Notes (1s/0s; sub-corpora)
Coding	Stack Overflow answer votes	inst.	12,202	6,391 / 5,811; 137,994 ranked pairs collected
	PR linked-issue reactions	coll.	154,633	13 repositories
Math	Math.SE answer votes	inst.	100,000	50,000 / 50,000; 11,629-row scored task
Law & policy	Comment co-signing	inst.	9,520	342 positive
	Law SE answer votes	coll.	4,192	balanced; 72,369-post pool
Academia	Citation percentile	inst.	2,387	1,176 / 1,211; from 27,233-paper collected corpus

Appendix

Table 6: CreativePreferences inventory, community/crowd channel (conventions as in Table 4 ).

task	decision-maker	candidate set	information at decision time	prov.	$n$	positives	groups	gap 95% CI	label uncertainty / selection effects	maturity
GitHub PR merged	repository maintainers	pull requests opened on 594 repositories	diff, description, CI results, author identity, review thread	O	11,452	9,205	255	[+.020, +.073]	merge reflects maintainer priorities and timing as well as the patch	mature
Stack Overflow accepted answer	the question’s asker	answers to the same question	all answers, votes at the time, asker’s own problem	O	16,001	5,972	5,972	[+.070, +.085]	one person’s choice; asker may accept early or never	mature
Competition editorial-approach	contest editorial (official solution)	accepted solutions per problem	problem statement, editorial	A	999	n/r	n/r	n/r	label = similarity of the solution to the editorial approach; model-assisted, not a gatekeeper verdict on the item	mature
Stack Overflow bounty award	the bounty poster	answers to the bountied question	all answers, votes, poster’s own problem	O	23,498	7,103	7,036	[+.036, +.044]	one person’s choice under a deadline	mature
Stack Overflow answer votes	site voters (aggregate)	answers to the same question	answers in vote order, accepted mark	O	12,202	6,391	5,972	n/r	vote order and acceptance bias exposure; threshold on score	mature
mathlib PR merged	mathlib maintainers	pull requests to mathlib	diff, CI build, reviewer comments	O	7,956	7,502	31	n/r	94% positives; full model unstable at this balance (u/q)	mature

Appendix

Table 7: Audit table, one row per task of Table 23 . Columns are defined in the text above.

task	executable check	share of items the check applies to	AUC, executable columns alone	AUC, existing $V$ bank	$V$ + executable
Code (GitHub PR merged)	tests pass after patch (17 cols); patch well-formed; added Python/JS/Go code parses (“compiles”); lint errors	3.7% (tests); 99% (patch); 94% (compiles)	.553 /.548 within-repository (tests);.528 (compiles, univariate)	–	no change
Code (GitHub PR merged; deep)	added dependencies exist on PyPI/npm; bandit findings on added Python	5.3% (deps); 3% high findings	.518 /.494	–	–
Math (AoPS editorial approach)	final answer matches the reference key	46.2%	.572 (univariate.649)	.706	.706
Law (N&C, three labels)	cited CFR/USC/FR sections exist; cited part matches the rule; docket id matches	11–13% cite anything	.44–.50	.59–.60	.59–.60
Journalism (editorial pickup)	internal arithmetic identities; URLs resolve; dates and weekdays agree	0.1% / 20% / 50%	.462	.611	.607
Journalism (editorial pickup)	stated revenue, income and EPS match the filer’s SEC XBRL facts	0.6%	.458	.611	.610

Appendix

Table 8: Executable checks per task. Coverage is the share of items to which the check applies; AUCs are grouped five-fold out-of-fold under the

V

protocol. “Deep” rows are the second, more thorough pass. The Math.SE row’s coverage varies by check (see the project notes).

campaign	proposing rounds	proposers per round	families
joke upvotes	5	8 (rounds 1–5); 5 in round 6, Claude slots missing	Claude, Codex, GLM
N&C received a response	4	4 (Opus, Sonnet, two Codex); GLM in round 2 only	Claude, Codex (GLM once)
editorial pickup (press)	2	6	Claude, Codex, GLM
BBC most-read	6	8 (four Codex, four GLM)	Codex, GLM
spotlight/oral pick	5	4 in rounds 1–2; 6 from round 3	Claude, Codex, GLM
citation percentile	6	4 in rounds 1–2; 6 from round 3	Claude, Codex, GLM

Appendix

Table 9: Proposer fleet per mining campaign, from the header of each round’s proposal file. A slot is counted when it returned proposals. Campaigns absent from the table ran no mining rounds (“n/c” in Table 23 ).

task	articulated judge (Gemma-4-31B)	full model (Llama-3.1-8B LoRA)
creative writing, story upvotes (and other story tasks)	$3{,}600$ head $+$ $2{,}400$ tail characters ( $\approx 1{,}400$ tokens)	first $1{,}024$ tokens ( $\approx 4{,}000$ characters)
code, GitHub PR merged (v3, enriched record)	full record (median $6{,}801$ tokens)	first $4{,}096$ tokens
peer review (abstract tasks)	full abstract and metadata	full abstract and metadata
press releases, editorial pickup	first $4{,}000$ characters ( $\approx 1{,}000$ tokens; was $3{,}600$ under the retired Llama scoring)	first $1{,}024$ tokens ( $\approx 4{,}000$ characters)

Appendix

Table 10: Text budget per model for the audited tasks, from the run configurations. Columns: what the articulated judge (Gemma-4-31B) and the full model (Llama-3.1-8B LoRA) read of each item.

	math.SE bounty		SO bounty
Judge	$A$	$VA$	$A$	$VA$
Gemma-4-31B (record)	.696	.723	.753	.779
Llama-3.1-70B	.686	.711	.745	.772
Qwen2.5-14B	.680	.708	.736	.766
phi-4 (14B)	.672	.708	.726	.766
gemma-2-9b	.678	.708	.738	.765

Appendix

Table 11: Judge-family sweep on two bounty tasks. Rows: judge model; columns: held-out AUC of the articulated bank alone (

A

) and with the verifiable features (

VA

), on math.SE bounty and Stack Overflow bounty, with the same criteria and prompts for every judge.

Judge	family	$A$	$VA$	anchors (pos vs. neg / coherent vs. scrambled)
Gemma-4-31B (record rerun)	Gemma	.648	.660	.57 / 1.00
phi-4 (14B)	Phi	.634	.647	.66 /.98
Qwen-2.5-14B-Instruct	Qwen	.632	.640	.71 / 1.00
Mistral-7B-Instruct-v0.3	Mistral	.592	.630	.66 /.97
Llama-3.1-8B-Instruct	Llama	.560	.619	.57 /.99

Appendix

Table 12: Judge-family sweep on creative-writing story upvotes (

7{,}008

items,

45

criteria, identical prompts). Columns: judge; model family; grouped out-of-fold AUC of the articulated bank alone (

A

, best of linear and gradient-boosted heads) and with the verifiable features (

VA

;

V

alone is

.612

); anchor-battery AUCs (50 known positives vs. 50 known negatives, and coherent vs. scrambled text).

		joke upvotes ( $16{,}000$ )		N&C responded ( $9{,}521$ )
Judge	family	$A$	$VA$	$A$	$VA$
Gemma-4-31B (record)	Gemma	.748	.752	.672	.733
Gemma-4-31B (re-run, control)	Gemma	.749	.752	.670	.736
Qwen-2.5-14B-Instruct	Qwen	.687	.699	.631	.718
phi-4 (14B)	Phi	.671	.687	.644	.719
Llama-3.1-8B-Instruct	Llama	.662	.679	.669	.732

Appendix

Table 13: Judge-family sweep on two further tasks, full banks (base criteria plus every mined round), scored under each judge with identical prompts. The control row re-scores both full banks with the judge of record on a second machine and a second run; it reproduces the record within

.003

, so the drops below it are judge effects, not run effects. Columns: grouped out-of-fold AUC of the articulated bank alone (

A

) and with the verifiable features (

VA

), gradient-boosted head, on the full population of each task (joke upvotes: 50 topic groups; notice-and-comment responses: 1,814 dockets).

Figure 4: Species accumulation and Good–Turing missing mass by mining round , Track-A (quality) criteria, for every multi-round closure campaign. Orange: cumulative distinct criteria species discovered (each round’s two-judge-merged vocabulary; cross-round deduplication happens at routing, so the accumulation slightly overcounts). Slate, dashed: the per-round missing mass

\hat{\mu}=f_{1}/N

(fraction of that round’s proposals that were first-time species). The figure shows the contrast that motivates the value-weighted whisker construction below: the species vocabulary keeps growing (

\hat{\mu}

stays at

.4

–

.7

even in final rounds; new names never run out) while the held-out value of each round’s additions saturates (the stopping rule that ended each campaign).

campaign	truncated after	bound	realized after	used	holds
BBC most-read	round 1	.007	.018	255%	no
BBC most-read	round 2	.010	.011	104%	borderline
BBC most-read	round 3	.020	.005	24%	yes
BBC most-read	round 4	.029	$-$ .002	0%	yes
r/Jokes upvotes	round 1	.012	.010	86%	yes
r/Jokes upvotes	round 2	.039	.001	2%	yes

Appendix

Table 14: Truncation backtest of the missing-value bound on the two campaigns with full per-round records (Table 16 extends it to seven campaigns) (each evaluated in its own campaign frame). The bound fails only at the earliest truncations, where the observed gain is a single noisy round; from mid-campaign onward the bound holds at every truncation with substantial slack. Figure whiskers are computed at the final round of terminal campaigns, the regime in which the backtest validates the bound.

Task	$\hat{\mu}$	mined gain	whisker	$VA$ + whisker	status	$G_{c}$	whisker c
Editorial pickup (press)	.589	.091	$+.130$	.879	bound	.022	$+$ .032
Citation percentile	.456	.069	$+.058$	.782	bound	.048	$+$ .040
Math.SE votes	.283	.029	$+.011$	.651	bound	.016	$+$ .006
Joke upvotes	.400	flat $\times$ 3	$+.022$	.743	bound+tick	.028	$+$ .019
BBC most-read	.608	flat $\times$ 2	$+.065$	.843	bound+tick	.041	$+$ .064
Spotlight/oral pick	.589	.060	$+.086$	.675	bound	$-$ .003	$+$ .000

Appendix

Table 15: The missing-value bound on every eligible task, at the final sealed round of each mining campaign. Columns:

\hat{\mu}

, the Good–Turing missing mass (

f_{1}/N

at the final round); mined gain,

G=VA_{new}-VA

, the increase in

VA

that the campaign realized (flat

\times k

: the last

k

rounds realized no gain); whisker,

G\hat{\mu}/(1-\hat{\mu})

, the bound of Eq. 5 ;

VA

+ whisker, the bounded articulated ceiling; status, whether the task reports the bound alone or the bound with the calibrated terminal tick (

+.005

, the realized future gain in the backtest’s terminal regime). n/e = not eligible (no sealed campaign). The two right-hand columns re-express the bound in the campaign’s own frame:

G_{c}

is the gain the campaign realized on its honest population between its first and final round (paired grouped bootstrap, Table 18 ), and whisker

{}_{c}=\max(0,G_{c})\hat{\mu}/(1-\hat{\mu})

. The mined-gain column is the master-table difference (the

+

mined column minus

VA

), which mixes the campaign frame with the bakeoff frame; for spotlight, press and HashtagWars the two frames disagree, and the campaign-frame bound is the smaller one.

campaign	$k/K$	$\hat{\mu}_{k}$	bound	realized	used	holds
BBC most-read	1/6	0.45	+.007	+.020	297%	no
BBC most-read	2/6	0.41 ∗	+.010	+.014	130%	no
BBC most-read	3/6	0.49 ∗	+.020	+.008	39%	yes
BBC most-read	4/6	0.51 ∗	+.029	+.001	4%	yes
BBC most-read	5/6	0.49 ∗	+.025	+.003	12%	yes
r/Jokes upvotes	1/6	0.42	+.012	+.011	94%	yes

Appendix

Table 16: Truncation backtest of the missing-value bound on seven campaigns. Rows: a campaign cut at round

k

K

; columns:

\hat{\mu}_{k}

, the missing mass at round

k

; bound, the whisker computed from rounds

\leq k

; realized, the gain actually obtained in rounds

k{+}1..K

; used, the mined gain entering the bound at round

k

; holds, whether realized

\leq

bound. n/d: the bound at that truncation was zero because no gain had yet been realized, so the share used is undefined and the bound fails by construction.

campaign / round	$\hat{\mu}$ similarity-only	$\hat{\mu}$ merged	$\hat{\mu}$ strict two-judge	leave-one-proposer-out range
HashtagWars r4	0.73	0.73	0.37	[0.73, 0.91]
Spotlight r5	0.78	0.78	0.59	[0.75, 0.84]
Citations r5	0.57	0.57	0.60	[0.52, 0.60]
AoPS r1	0.50	0.56	0.49	[0.53, 0.64]
BBC r1	0.55	0.45	0.45	[0.38, 0.53]
BBC r6	0.54	0.61	0.61	[0.58, 0.67]

Appendix

Table 17: Sensitivity of the missing mass

\hat{\mu}

to the species-merge rule. Rows: campaign and round; columns:

\hat{\mu}

under the similarity-only partition, under the default two-judge merge, under the strict two-judge partition, and its range across leave-one-proposer-out recomputations. Values in red were recomputed from the saved verdicts and re-embedded proposals (bge-large, cosine

.79

); where a round has only one two-judge partition, the strict and merged columns coincide by construction, and for the HashtagWars, spotlight, citations and press rows the merged column is the similarity-only partition.

task	$\hat{\mu}$	95% CI	$G$	se $(G)$	whisker	95% CI
Editorial pickup (press)	.589	[.405,.772]	.022	.015	$+$ .032	[ $+$ .000, $+$ .108]
Citation percentile	.456	[.288,.623]	.048	.015	$+$ .040	[ $+$ .012, $+$ .093]
Math.SE votes	.283	[.166,.401]	.016	.007	$+$ .006	[ $+$ .001, $+$ .015]
Joke upvotes	.400	[.219,.581]	.028	.003	$+$ .019	[ $+$ .008, $+$ .040]
BBC most-read	.608	[.458,.759]	.041	.003	$+$ .064	[ $+$ .034, $+$ .133]
Spotlight/oral pick	.589	[.411,.767]	$-$ .003	.013	$+$ .000	[ $+$ .000, $+$ .037]

Appendix

Table 18: Inputs to the missing-value whisker and their uncertainty, at the final round of each campaign. Columns:

\hat{\mu}

(Good–Turing missing mass) with its 95% interval; the mined gain

G

in the campaign’s own frame and its standard error from a 2,000-draw paired grouped bootstrap of the campaign’s reproduced heads; the whisker

G\hat{\mu}/(1-\hat{\mu})

with its 95% Monte-Carlo interval (3,000 draws).

\hat{\mu}

and

G

are the values of Table 15 ; whiskers may differ in the third decimal by rounding. The citation-percentile

\hat{\mu}

is the pooled-partition value of that table; its standard error uses the final round’s

N{=}90

and the strict partition’s doubleton count.

task	$n$	$T$	ours	HypotheSAEs	HypoGeniC	HypotheSAEs pkg	ExperiGen	union	ours+union	whisker	ceiling	full	gap
Joke upvotes	3,163	.747	.689	.604	.686	.556	.691	.707	.716	$+$ .019	.734	.747	$+$ .013
Math.SE accepted	2,600	.644	.574	.532	.554	.526	.582	.579	.575	$+$ .001	.576	.644	$+$ .068
Math.SE votes	2,326	.654	.603	.535	.611	.569	.628	.632	.631	$+$ .006	.637	.660	$+$ .023
N&C rule change	1,417	.624	.612	.545	.607	.487	.632	.644	.633	0	.633	.634	$+$ .001
N&C responded	1,904	.817	.791	.501	.675	.470	.723	.736	.802	$+$ .013	.815	.841	$+$ .026
ICLR spotlight/oral	1,571	.594	.529	.517	.557	.455	.514	.556	.573	0	.573	.594	$+$ .021

Appendix

Table 19: Metric-discovery head-to-head on the fourteen tasks with a shared evaluation frame: held-out AUC of the full model

T

, our bank, each external bank, their union, and the union stacked on ours; the campaign whisker; the committed articulated ceiling

\max(\text{ours},\text{ours+union})+\text{whisker}

; the full model

\max(T,\mathcal{VAT})

with

\mathcal{VAT}

refit on ours+union; and the articulability gap, full model minus ceiling. ExperiGen is the strongest external method (best on ten tasks, ahead of our bank on six); the package-faithful HypotheSAEs is last on every task. The gap survives on twelve of the thirteen bounded tasks; on BBC most-read the whisker covers it (

-.005

), and on N&C rule change the two are at parity (

+.001

Panel of Figure 5 — Figure 5: The spurious-feature audit. A candidate metric goes through three tests. Channel tests ask whether it captures signals external to the text, by scrambling the text and by probing its top n-grams. Path tests ask whether its association with the label is reducible to a known spurious variable, by stratifying on that variable and checking whether the metric survives. The LLM adjudication asks whether the metric measures the capture, selection or context of the item rather than the quality of the work. Each test writes one row of an evidence table (result plus the assumption it depends on), and the adjudication routes the metric, citing those rows, either to the criterion bank as a genuine metric or to the spurious-variable ledger, where it is deconfounded: stratified on if present in more than $60\%$ of the data, stripped by regular expression if it is a removable surface feature, and otherwise residualized out of the models’ scores.

case	truth	scramble AUC	surface $R^{2}$	fingerprint $\rho$	AUC pooled $\to$ within stratum	stage 5	routed
planted token, disguised criterion	spurious	.51	.89	.60	.70 $\to$ .50	incidental	yes
planted token, literal criterion	spurious	.51	1.00	.62	.71 $\to$ .50	incidental	yes
control: reveal is present and locatable	quality	.49	.05	.22	.59 $\to$ .59	quality	no
control: point survives as a quotable formulation	quality	.49	.03	.13	.62 $\to$ .63	quality	no
112 record quality criteria (47 base, 65 mined)	quality		0 of 112 flagged				no
68 record nuisance criteria	spurious		6 of 68 flagged by statistics			incidental	yes (stage 5)

Appendix

Table 20: Validation of the spurious-feature audit on planted artifacts and historical leaks. Columns are the appendix’s tests: label AUC on word-scrambled items; variance explained by the length/format family (plus the token indicator); Spearman correlation with a character-

n

-gram capture classifier; AUC pooled and inside strata of the suspect channel; the blind construct audit’s label; and whether the battery routes the case to

M_{spurious}

. Flag thresholds: surface

R^{2}>.5

|\rho|>.5

, within-stratum loss

>.08

AUC, scrambled AUC more than

.10

from

.5

Task	Grouping unit	Declared	Largest declared spurious features (by label AUC on their own)
Creative writing (crowd)	story prompt	70	Fragmented staccato paragraphing / line-break repetition; Blank-line and short-paragraph density; Dialogue punctuation mark count
N&C responded	docket	56	First-hand local exposure; First-person credential or stake self-disclosure; Identified accountable author with stated standing
Peer review (citation)	paper	57	Extent of currently fashionable subfield vocabulary; Trend-Aligned Vocabulary; Count of enumerated evaluation settings and resource markers
Press releases (pickup)	company	23	Extent of departure from templated corporate register; Extent of newswire dateline, ticker and boilerplate formatti; Formal wire-service dateline / distribution convention
Jokes (crowd)	topic cluster	58	Extent of overlap with stock circulated joke material; Pre-owned chestnut: the pun arrives already familiar; Canonical-chestnut retelling fingerprint (position in retell
Math SE (accepted)	question	36	number of answers; answer position; is the first answer

Appendix

Table 21: Declared spurious variables per task. “Declared” is the size of the task’s nuisance ledger (

M_{spurious}

criteria plus structural columns such as posting time and answer position); the last column names the three ledger entries with the largest label AUC on their own, as recorded in the deconfounding run. The full ledgers are in the run artifacts.

Task	$K$	Full-model AUC	Residual AUC (all $K$ )	Asymptote	placebo drop (share)
Creative writing (crowd)	70	.792	.737	.738	+.008 (0.15)
N&C responded	56	.817	.733	.733	+.035 (.42)
Peer review (citation)	57	.884	.773	.777	+.029 (0.26)
Press releases (pickup)	23	.774	.729	.726	+.034 (0.73)
Jokes (crowd)	58	.747	.641	.641	+.004 (0.04)
Math SE (accepted)	36	.644	.579	.560 e	+.016 (0.24)

Appendix

Table 22: Full-model performance under progressive nuisance partial-out.

K

= number of declared nuisance channels; “Residual AUC” = held-out AUC of the full model’s readout after cross-fitted linear residualization on all

K

channels; “Asymptote” = fitted floor

A

, quoted only where the curve plateaus (

|A-\text{full-}K|<.01

); “—” marks tasks still declining at full

K

, for which the full-

K

residual is an upper bound on the floor. One task (BBC most-read) is omitted: its dense ledger is not yet built. e : extrapolated from a curve still declining at full

K

, so read as an upper estimate; n/i: no finite asymptote is identifiable from the fitted curve (the full-

K

residual is the quotable bound). “Placebo drop” = the drop in AUC produced by residualizing on a row-permuted nuisance block with the same

K

and folds (mean of the placebo runs), i.e. the part of the drop that is first-stage estimation noise rather than shared variance, with its share of the archived drop in parentheses. † : New Yorker caption-contest task; not included in any main-text table or average (multimodal).

Field	task	Channel	$V$	$VA$	$+$ mined	$T$	$VAT$
Coding	GitHub PR merged	verdict	.550	.631	.638	u/q m	.692
	Stack Overflow accepted answer	verdict	.639 f	.733	n/c	.743	.764 [.736, .791]
	Competition editorial-approach	curated	.743	.734	n/c	.724 m	.737 m
	Stack Overflow bounty award	curated	.739 f	.787	n/c	.804	.811 [.791, .829]
	Stack Overflow answer votes	community	.638	.710	n/c	.723	.728
Math	Math.SE accepted answer	verdict	.591	.632	.618 f	.644	.630

Appendix

Table 23: Per-task provenance, sizes and paired intervals are in Table 7 . Every task, every instrument (held-out AUC; grouping units and construction in Appendix B ). “

+

mined” is the articulated bank after criteria-mining rounds, where a campaign ran; “n/c” means no mining campaign ran, “n/r” that the model was not run, “u/q” that it is unquoted, and “—” that it was not built for that task, not that it scored at chance (the two tasks marked 0 are at chance; see the notes and Appendix G.6 ). Bracketed intervals are group-bootstrap

95\%

CIs on the fused test-split value. The

\mathcal{VAT}

column is the fused candidate chosen on the evaluation split and scored on test; it is below

T

on 7 of the 26 rows that have both and below the articulated model on 3, and

V

exceeds

VA

on 5 rows; the full model used for every gap and share is

\max(T,\mathcal{VAT})

. Of the

32

rows, the

28

with a full model are the points of Figures 1 and 3 and the units of every cross-task correlation and test; rows with no full model (n/r or — in both full-model columns) are assigned no gap.

Figure 6: Best-instrument AUC (the ceiling) for the tasks behind Figure 1 , grouped by domain and by preference channel. The raw gap co-moves with this ceiling (

r{=}.49

p{=}.008

); the normalized share does not (

\rho{=}-.09

p{=}.65

;

n{=}28

Figure 7: Decision-maker verdicts , unified-frame ladder. Green bars are the verifiable model

V

; blue bars the articulated model

VA

, with pale stacks marking criteria-mining increments where a mining campaign ran; purple bars the full model, the larger of the

T

and

VAT

columns of Table 23 . Every verdict task of that table is drawn, with its markers; n/r = no full model on record. Thin whiskers on the amber bars are the Good–Turing missing-value bounds of Appendix D.2 , drawn under one standard on every task with a sealed mining campaign (editorial pickup

+.130

, notice-and-comment

+.013

); the peer-review campaign was exploratory and carries none.

Figure 8: Expert curation , unified-frame ladder (same encoding as Figure 7 ). Every curated task of Table 23 is drawn; on Wigleaf the full-model bar is the fused value (

.593

, marker

f

), since the text-only model (

.559

) sits below the articulated model. Whiskers are the missing-value bounds (spotlight

+.086

, AoPS

+.001

, HashtagWars

+.101

); the spotlight bound exceeds the task’s full model, so its gap is treated as unresolved (Table 23 ).

Figure 9: Community preference across seven fields (eight tasks), unified-frame instruments. Per task: verifiers

V

(green), the articulated model

VA

(blue) with its criteria-mining increment stacked on top (pale), and the full model, the larger of

T

and

VAT

(purple). Thin solid whiskers are missing-value upper bounds : observed mining gain scaled by the ratio of unmined to mined criteria mass (Good–Turing, last mining round); bars without whiskers lack a campaign or a mined column, not evidence of zero remaining gain. Humor and journalism show their bounds like every other task (

.022

and

.065

) and additionally a dashed tick at the calibrated residual (

\approx.005

): both campaigns closed

\geq 2

consecutive proposing rounds with held-out gain below

\varepsilon{=}.005

, the regime in which the backtest we report in Appendix D.2 validates the realized-gain estimate. The tick is a point estimate and the whisker a bound. The two campaigns’ Good–Turing bounds at closure are

.022

(humor, mass

.564{\to}.400

after strict cross-proposer species merging) and

.065

(journalism, mass

.492{\to}.608

; the bound widens because unseen-species mass rose even as realized gain stayed at zero, the mass

\neq

value pattern the backtest anticipates). Tasks grouped by field; the annotated creative-writing spans mark the verification and articulation gaps.

Panel of Figure 10 — Figure 10: The articulability gap: channels and anatomy. (a) A written input $x$ reaches a binary judgement $y$ through two channels we can measure: verifiable metrics $m_{\mathrm{ver}}(x)$ (checkable by a program) and articulable metrics $m_{\mathrm{art}}(x)$ (statable in words, scored by an LLM judge); the full model trained end-to-end on $(x,y)$ bounds the total shared preference. (b) On a given task, the span from $V$ to $VAT$ is the verifiability gap (what can be said but not checked), and the step from $VA$ to $VAT$ is the articulability gap (what is shared but resists statement), which is our operationalization of tacit preference. The training-signal spectrum this implies is discussed in § 7 .

task	$n$	groups	AUC	boot. 95% CI	LOGO $\|\Delta\|_{\max}$	half sd	pairwise
GitHub PR merged (transition frame)	7,563	184	.630	[.587,.677]	.012	.035	.646
Homepage placement	12,998	1,229	.734	[.722,.746]	.001	.006	.708
Kindle Scout	726	3	.724	[.714,.790]	.059	.028	.715
Math.SE bounty	23,972	8,278	.723	[.716,.729]	n/r	.003	.711
RoyalRoad followers	3,604	149	.630	[.611,.648]	.003	.011	.655
RoyalRoad market pickup	1,274	1,274	.562	[.530,.593]	.001	.021	n/r

Appendix

Table 24: Robustness of the articulated model’s AUC on every task with per-item held-out scores. Columns: items; bootstrap groups; the AUC; its 3,000-draw grouped-bootstrap 95% interval; the largest absolute change in AUC when one group is left out (n/r where groups exceed 3,000 and the bootstrap covers it); the standard deviation of the AUC over 30 random halves of the groups; and the pairwise AUC computed only within groups (n/r where groups are singletons). The retired v1 moderation task is shown because its interval illustrates the leak the information view flagged.

Task	$n$	groups	$V$	$VA$	$\Delta(VA-V)$
math.SE bounty	23,972	8,278	.691 [.684,.699]	.723 [.716,.730]	[+.028,+.035]
SO bounty	23,498	7,036	.739 [.732,.746]	.779 [.772,.785]	[+.036,+.044]
SO accepted	16,001	5,972	.639 [.631,.648]	.717 [.708,.725]	[+.070,+.085]
r/Jokes removal	18,879	55	.603 [.542,.857]	.701 [.651,.903]	[+.043,+.111]
Kindle Scout	726	3	.612 [.608,.685]	.724 [.714,.790]	[+.105,+.142]
Macro-norm humor (pooled)	73,268	5	.561 [.548,.572]	.714 [.690,.742]	[+.135,+.185]

Appendix

Table 25: Standardized uncertainty for the articulated ladder. Rows: tasks; columns: number of items and of bootstrap groups, the 3,000-draw grouped-bootstrap 95% intervals for

V

and

VA

, and the interval of their paired within-draw difference.

Source	Side	Measured bound	Where
Judge family (7 judges, 6 families)	$A$ $\downarrow$	$\leq.031$	App. C.5
Judge panel (7-family ensemble)	$A$ $\uparrow$	$+.005$	App. C.5
Per-judge template optimization	$A$ $\uparrow$	$\leq.003$	App. C.5
Per-judge prompt optimization	$A$ $\uparrow$	$\leq.05$ (peer review)	App. H.3
Prompt/implementation headroom	$A$ $\uparrow$	${\sim}.006$	parallel forms ( Spangher, 2026 )
Criteria-mining exhaustion	$A$ $\uparrow$	asymptote $\pm.006$ + whisker	§ 4 , App. D.2

Appendix

Table 26: Systematic (non-sampling) sensitivity budget for the articulated side. Rows: sources of variation we measured; columns: which side of the comparison they move, the measured bound on their effect, and the appendix that measures it.

Figure 11: Data-scaling of the Llama-3.1-8B LoRA full model (best validation AUC vs. fraction of training data; one panel per sweep family). Most sweeps use ungrouped splits, so levels can be inflated by group identity (dashed grouped variants show the level without group leakage where run, e.g. homepage newsworthiness

.86\to.82

; the code-review and peer-review ungrouped levels are known leak artifacts); the shapes are the informative part: creative writing is still climbing at full data while math Q&A, legal outcomes, and press releases saturate by 30–60%.

Figure 12: Residual scaling: full models (

VAT

) retrained on nested fractions of two curated tasks’ training data (identical eval/test rows throughout). Left: the raw dense bound grows log-linearly with data (

\sim

.03 AUC per doubling) with no plateau in range, and the later story-upvotes scale-up (raw text, three training sizes up to the full 76,867 rows, dashed) follows the same slope; a saturating power-law fit leaves the asymptote unidentified. Right: the deconfounded residual (the full model’s margin over the articulated bank plus declared channels, grouped bootstrap) is indistinguishable from zero at 12.5% data and grows to

+.013

[.007,.018] (math.SE bounty) and

+.006

[.002,.011] (Stack Overflow bounty) at full data: the articulated bank absorbs roughly

90\%

of what added data gives the dense model, and the unabsorbed part is still growing. Articulated-share estimates are therefore upper bounds that decline slowly with full-model data. The Llama-8B sweep above predates the current corpora and is retained for provenance only.

Figure 13: Verifiable vs. articulable discrimination share, one point per task of the earlier 24-task set (

n{=}24

, including the withdrawn patents task; marker = preference channel). The dotted diagonal is verifiable

=

articulable (checks carry everything statable). Spearman

\rho=.83

(

p<10^{-5}

) on the plotted points; on the 28 tasks of Figure 1 ,

\rho=.63

(text). Humor sits far below the trend: highly articulable, weakly verifiable.

field	tacit share	jargon fraction	distinctiveness	most discipline-specific words
peer review	$.392$	$.406$	$.118$	llms, neural, adversarial, outperforms
creative writing	$.386$	$.179$	$.052$	wp, didn, sighed, nodded, stared
humor	$.153$	$.359$	$.099$	contest, prompt, entry, hashtag
code	$.139$	$.408$	$.074$	import, com, self, github, def
notice and comment	$.135$	$.259$	$.089$	docket, FAA, NPRM, airworthiness
mathematics	$.121$	$.486$	$.062$	frac, math, sqrt, mathbb, infty

Appendix

Table 27: Scholarly jargon, measured as in Lucy et al. (2023) , against the tacit share. Jargon fraction is the mean share of an item’s tokens with field-level type NPMI above

0.1

; distinctiveness is the mean of

\max(\mathcal{T}_{f}(t),0)

over the field’s vocabulary. Rank correlation with the tacit share is

+.11

for the fraction and

+.39

for distinctiveness over seven fields, and

-.13

(

p{=}.60

) over the nineteen tasks. The two largest-gap fields sit at opposite ends of the jargon scale.

property	all ( $n{=}19$ )	community (6)	verdict (6)	curated (7)	with the object control
rare-word share	$-.39$	$-.26$	$+.09$	$-.75$	$+.015$ ( $t{=}0.5$ )
out-of-vocabulary share	$-.43$	$-.37$	$+.20$	$-.68$	$-.003$ ( $-0.1$ )
mean word length	$+.35$	$+.03$	$+.94$	$+.82$	$+.027$ ( $1.1$ )
lexical diversity	$+.53$	$+.14$	$+.26$	$+.71$	$-.088$ ( $-3.1$ ), sign flips
compressibility	$-.07$	$-.94$	$-.31$	$+.21$	$-.092$ ( $-6.2$ ), length proxy
reading grade	$+.21$	$+.09$	$+.83$	$+.46$	$+.044$ ( $2.0$ )

Appendix

Table 28: Surface properties of the texts against the tacit share: Spearman

\rho

over all nineteen tasks and within each channel, and in the last column the coefficient per standard deviation once the composition-versus-answer indicator of § G.4 is in the model. Only length survives that control with its sign intact.

task	tacit	who decides	deciders per item	producers	institutions
story upvotes	$.435$	anonymous crowd	$219$ mean / $47$ median upvotes ℓ	not in data	none by construction
citation percentile	$.423$	the citing community	$59.7$ / $10$ citing works ℓ	not in data	not in data
ICLR accept/reject	$.394$	reviewers + area chair	$3.81$ / $4$ reviews	$62{,}442$ authors, $4.78$ /paper	not in data
spotlight/oral	$.360$	programme committee	committee absent from data	$60{,}689$ authors, $5.05$ /paper	not in data
RoyalRoad followers	$.337$	anonymous readers	$21.4$ / $2$ followers ℓ	not in data	none by construction
pull request merged	$.281$	repository insiders	$37.7$ / $29$ review comments	not in data	$8{,}982$ orgs, HHI $.002$

Appendix

Table 29: Participants behind each task, counted from the raw corpora. ℓ marks a count that is label-derived: the binary outcome is a threshold on it, so it must not be correlated with the gap. Producer and institution counts are floors where identity is sparsely recorded (notice-and-comment names an organisation on

13

–

16\%

of comments; the Style Invitational entrant count comes from a name-extraction heuristic with a

73\%

match rate). Affiliation is absent from every peer-review file we hold and does not exist for the Reddit and RoyalRoad tasks, which is why the institutional-tightness question is left open rather than answered.

task	$n$	full model	articulated	memory alone	articulated + memory	gap	share closed
citation percentile	$478$	$.884$	$.627$	$\mathbf{.739}$	$.746$	$+.257$	$\mathbf{+46\%}$
#HashtagWars	$924$	$.732$	$.553$	$.546$	$.542$	$+.179$	$-6\%$
story upvotes	$7{,}008$	$.792$	$.658$	$.587$	$.675$	$+.134$	$+12\%$
ICLR accept/reject	$1{,}244$	$.777$	$.647$	$.574$	$.657$	$+.130$	$+7\%$
Math.SE accepted	$2{,}600$	$.644$	$.549$	$.497$	$.552$	$+.095$	$+4\%$
Math.SE votes	$2{,}326$	$.654$	$.576$	$.511$	$.572$	$+.077$	$-6\%$

Appendix

Table 30: The corpus-memory ablation on every task whose text we hold, ordered by gap size. The articulated model is refitted with three added features that encode only the labels of an item’s nearest training-fold neighbours, so nothing new is articulated; the articulated column is this ablation’s own refit and differs from Table 23 by a few points. Share closed is

(\mathrm{AUC}_{VA+\mathrm{mem}}-\mathrm{AUC}_{VA})

over the gap, and it is unstable wherever the gap is small: the three notice-and-comment rows are ratios on denominators of

.03

–

.05

, and on notice-and-comment responses the augmented articulated model passes the full model outright. Across the fourteen live tasks the share closed does not correlate with gap size (

\rho=+.02

), nor does the memory-alone AUC (

\rho=-.07

). On citation percentile a memory of the corpus alone (

.739

) beats the entire criteria bank (

.627

) and closes

46\%

of the gap; on joke upvotes it closes nothing. Remaining tasks were still running when this table was made.

Panel of Figure 14 — Figure 14: Metric infilling: fields share an articulated core; subfields introduce their own criteria. Schematic of the discovery procedure: a field carries a shared metric set ( $m_{1}$ – $m_{3}$ ); mining on a subfield’s residual (using labels only on the training and monitoring splits; the test split is never touched by any discovery decision) proposes candidate criteria, and a candidate is kept only if it increases scores on a held-out set. Preference vocabularies are heavily shared across related decisions (per-metric rank agreement $\rho{=}.76$ between regulatory outcome and agreement; $\rho{=}.47$ between academic curation and citations) while the weights and a small set of subfield-specific criteria differ (metric names schematic).

Figure 15: Organizational scale and the two layers of verifiability (regulatory rulemaking, agency-agreement label). Each point is a federal agency. Headcount predicts how well shallow surface verifiers work (

\rho{=}.50

p{=}.022

) while practice (lifetime rulemaking-docket throughput) predicts deep verifier signal (

\rho{=}.64

p{=}.003

; robust to excluding FAA and to partialling sampled-docket count); the two cross-pairings are null. Merging the layers dilutes the throughput signal (

\rho{=}.30

, ns) while the full

V{+}V_{deep}{+}A

stack retains it (

\rho{=}.48

p{=}.036

). The GitHub counterpart, where an earlier analysis found the opposite sign (repos with

\leq

5 maintainers more internally consistent,

p{=}.035

, Mann–Whitney), rests on artifacts that were since lost and a chance-floor reconstruction; its rebuild is queued and it is not shown.

Task	Community	$n$	$V$	$A$	$VA$
Humor	pooled (5 subs)	73,268	.561	.712	.714
	r/tifu	12,650	.605	.798	.802
	r/Showerthoughts	17,942	.616	.758	.769
	r/me_irl	7,324	.582	.749	.752
	r/nottheonion	12,332	.570	.727	.737
	r/funny	23,020	.595	.715	.727

Appendix

Table 31: Per-community articulated recovery inside the pooled macro-norm moderation tasks. Rows: one subreddit per line, grouped by pooled task; columns: number of items and held-out AUC of the

V

A

and

VA

banks refit within that community, with the same criteria, judge and estimators as the pooled fit.

Task	Channel	$VA$	$T$	$VAT$	95% CI
r/Jokes moderator removal	verdict	.904	.922	.923	[.908,.939]
Kindle Scout publisher acceptance p	verdict	.744	.744	.792	[.627,.935]
StackOverflow accepted answer	verdict	.733	.743	.764	[.736,.791]
StackOverflow bounty award	curated	.787	.804	.811	[.791,.829]
math.SE bounty award	curated	.714	.733	.738	[.717,.759]
News-article tweet engagement	community	.600	.643	.646	[.627,.665]

Appendix

Table 32: Fused preference recovery on the boundary tasks. Fusion variants are selected on the evaluation split and reported on the test split. p Pilot-scale task (

n{=}726

); the wide interval is the value to report. b The articulated model’s anchor battery is weak on this population and

VA

coincides with a word-

n

-gram baseline, so the dense-over-bank margin here measures an instrument limit, not tacitness.

field	density ( $\log_{10}$ count)	task	tacit share
math Q&A	4.79	accepted answers; answer votes	.083;.255
creative writing	5.62	story upvotes	.435
humor	5.85	joke upvotes; HashtagWars	.150;.125
notice and comment	6.32	responded; outcome	.133;.145
peer review	6.57	verdict; curation; revealed	.394;.360;.417
press releases	7.10	pickup	.109

Appendix

Table 33: Pretraining density against the tacit share. Density is

\log_{10}

of the summed Dolma counts of four field-diagnostic phrases; the tacit share is

(\mathrm{AUC}_{VAT}-\mathrm{AUC}_{VA})/(\mathrm{AUC}_{VAT}-.5)

from Table 23 . Spearman

\rho{=}.12

over the eleven tasks, permutation

p{=}.73

; only six density values exist, so the effective sample is six fields.

task	subfield (levels)	rate range	artic.	+subfield	full	artic.	+subfield	separate	full	gap pooled / within / separate
			pooled AUC			within-subfield AUC
creative writing: upvotes	genre tag (9)	.07–.57	.668	.684	.792	.653	.656	.645	.779	.124 /.126 /.135
N&C: received response	agency (14)	.39–.89	.796	.827	.817	.772	.768	.691	.804	.021 /.032 /.113
N&C: rule change	agency (13)	.22–.56	.618	.619	.624	.605	.594	.545	.615	.006 /.009 /.070
N&C: cosigner agreement	agency (12)	.43–.87	.562	.556	.603	.538	.506	.523	.593	.041 /.055 /.070
peer review: accept/reject	subject area (6)	.32–.55	.647	.653	.777	.649	.654	.643	.776	.130 /.127 /.133

Appendix

Table 34: Subfield conditioning under pooled and within-subfield readouts. Rate range is the positive rate across subfield levels. Pooled AUC is over the whole task; within-subfield AUC is computed inside each level and averaged with level weights (levels with fewer than twenty items or one class are skipped). Artic. is the articulated bank alone, +subfield adds a one-hot of the level, separate fits one articulated model per level on that level’s training rows and pools the out-of-fold scores, full is the full model on the same items. Gap is full minus the articulated model under each readout, and full minus the separate fit within subfields. Under the pooled readout the one-hot gains on notice-and-comment responses and story upvotes are between-subfield base rates; within subfields neither the one-hot nor separate per-subfield models help in any task.

agency	rows	own criteria	pooled bank	+ own	+ others’	+ all	full model
DOT	476	7	0.879	0.865	0.860	0.856	0.604
FWS	416	8	0.677	0.671	0.716	0.720	0.868
FDA	415	8	0.653	0.655	0.673	0.658	0.655
CMS	413	8	0.693	0.694	0.694	0.699	0.836
NHTSA	408	8	0.722	0.707	0.714	0.701	0.780
ED	406	8	0.755	0.771	0.788	0.778	0.884

Appendix

Table 35: Per-agency criterion mining on notice-and-comment responses. Rows: fit-and-mine rows of the agency; own criteria: its mined, audited, non-collapsed criteria. AUCs are grouped out-of-fold inside the agency for the articulated model refit on the pooled terminal bank, on the bank plus the agency’s own criteria, plus the other thirteen agencies’ criteria, and plus all

98

; the full model’s AUC is on the agency’s dense-held-out rows. Fixed-parameter gradient-boosted head, one seed.

pair	readout	task	anchor	bank	anchor	bank	$\Delta$ bank	bank of record
			base		instruct
Llama-3.1-8B	free generation	peer review: accept/reject	.99	.524	.996	.582	+.058	.654
		N&C: received response	.78	.545	.94	.677	+.132	.796
		creative writing: upvotes	.94	.558	.99	.543	$-$ .015	.668
Gemma-3-12B	first-token log-prob	peer review: accept/reject	.75	.559	.999	.663	+.104	.654
		N&C: received response	.47	.602	.81	.650	+.048	.796

Appendix

Table 36: Base-versus-instruct judge pairs on the twenty-four strongest criteria of three tasks. Anchor is the coherent-versus-scrambled AUC (gate

.80

); bank is the grouped out-of-fold AUC of the twenty-four criteria under the frozen head; bank of record is the full bank scored by the judge of record (Table 23 ). The 8B legs pass or nearly pass the anchor but are degenerate scorers (base: only

0

10

; instruct: one modal value on

67

–

95\%

of calls). The 12B base legs have real spread and fail the anchor in every task. No row is read as a post-training effect.

		peer review: accept/reject		N&C: received response		creative writing: upvotes
judge	training	record	optimized	record	optimized	record	optimized
Gemma-3-12B	base	.559	.568	.601	.596	.564	.573
Gemma-3-12B	instruct	.663	.668	.651	.653	.642	.646
Llama-3.1-8B	base	.546	.570	.601	.638	.603	.601
Llama-3.1-8B	instruct	.625	.632	.674	.691	.599	.592
Gemma-4-31B (judge of record)	instruct	.642	.694	.681	.672	.640	.652

Appendix

Table 37: Bank AUC of the twenty-four strongest criteria under the record instructions and under instructions optimized for that judge (grouped out-of-fold, frozen head, first-token log-probability readout for every judge). Optimization is label-free, so the columns are comparable. The upper block is the base-versus-instruct pairs, the lower block post-trained judges of different families.

task	cutoff	late rows	$T$ paper	$T$ early	$T$ drift	$\mathcal{VA}$ paper	$\mathcal{VA}$ early	$\mathcal{VA}$ drift	gap early
press pickup	2021	203	.734	.668	$+.066$ [ $+.005$ , $+.106$ ]	.691	.718	$-.008$	$-.050$
joke upvotes	2018	781	.783	.741	$+.042$ [ $+.020$ , $+.057$ ]	.724	.734	$-.004$	$+.007$
ICLR accept/reject	2024	413	.796	.785	$+.011$ [ $-.007$ , $+.030$ ]	.640	.641	$+.008$	$+.144$
BBC most-read	2022	1,535	.779	.772	$+.007$ [ $-.001$ , $+.015$ ]	.730	.727	$+.011$ [ $+.002$ , $+.019$ ]	$+.046$
spotlight/oral	2024	397	.595	.587	$+.009$ [ $-.041$ , $+.060$ ]	.489	.505	$-.028$ [ $-.102$ , $+.042$ ]	$+.081$
story upvotes	2021	1,385	.762	–	–	.635	.617	$+.023$ [ $+.004$ , $+.042$ ]	–

Appendix

Table 38: Temporal holdout. Every model is read on the paper’s held-out rows with year after the cutoff. “Paper” is the model trained across all years; “early” is the same model refit on years up to the cutoff. Drift is paper minus early (positive: the model went stale), with group bootstrap intervals where the interval excludes zero or nearly does. “Gap early” is the early full model minus the early articulated model. “

\mathcal{VA}

early” is the three-seed mean of the refit; the

\mathcal{VA}

drift estimate and interval come from the paired bootstrap on the first seed, so the two columns do not subtract exactly. The full model was retrained on the five tasks with a “

T

early” entry; the other four are read with the articulated refit only.

Task	Channel (how computed)	Items with it	Gap, raw	Gap, within strata
Press pickup	issuing company (group id; read within company)	–	$.107$	$.065$
Creative writing (crowd)	non-[WP] feature-thread tag in the prompt line (pattern)	9.8%	$.121$	$.116$
Homepage placement	section kicker or format label glued to the headline (pattern)	5.6%	$.084$	$.082$
Math SE (vote)	answer says it supplements earlier answers (pattern)	4.4%	$.031$	$.029$
Math SE (accepted)	same	5.0%	$.073$	$.072$
GitHub PR merged	bot dependency update; status token ([WIP], do not merge); maintainer share of review comments (pattern)	7.6%, 1.1%	$.046$	$.041$

Appendix

Table 39: Exposure channels named by the item-level reads, declared as nuisances and read on every held-out item (raw frame; the text-only full model and the bank refit with the frozen stack). Gap is

\max(T,\mathcal{VAT})-\mathcal{VA}

in AUC. “Within strata” pools AUC within the channel’s strata (press: within company, 8 companies with both outcomes; pull requests: within the joint strata of the two binary channels).

Rival account	Tasks	Key result	Verdict
One decider’s taste (decider holdout, offerer-disjoint retrain)	Math.SE bounty, SO bounty, SO accepted	seen vs unseen deciders $-.003$ $[-.024,+.019]$ , $+.003$ $[-.017,+.020]$ ; retrain $-.004$ $[-.015,+.006]$	not supported
Timing and exposure	jokes, stories, SO votes, RoyalRoad, tweets	jokes $-.000$ ; stories $+.097\to+.037$ , within-quintile shrink $\approx 30\%$ ; others $\leq.016$	mixed (stories)
Coarse criterion readout	jokes, stories, ICLR, citations	$\Delta$ $+.009$ , $+.012$ , $-.004$ , $+.033$ [CI crosses 0]	not tripped; instrument
Judge noise	same four	paraphrase ICC $.83/.84/.62/.69$ ; six-readout average $+.008$	not tripped; instrument
One holistic judgment	citations, jokes, stories, ICLR	$\mathcal{VA}$ +holistic $.868$ vs full $.884$ ; stories below bank	instrument only (reading retracted)
Vocabulary limit (description bottleneck)	stories	description $.746$ vs raw $.714$ at 6k rows (three seeds), $.763$ vs $.787$ at 24k, $.771$ vs $.802$ on all rows; neutral rewrite $.741$	description holds 77% of the gap at full size

Appendix

Table 40: Rival accounts of the articulability gap. Gaps are held-out AUC differences

\mathcal{VAT}-\mathcal{VA}

on the frozen frame; brackets are group-paired bootstrap intervals (2,000 draws). “Instrument” marks tests that bound our measuring instrument rather than human articulability.

Task	Property	Measured by	Share of items	High-gap / matched (%): preferred $\cdot$ not	Label AUC	Scope	Flag
Story upvotes	Dialogue-heavy: 25% or more of the story’s characters in quotes	pattern	35.4%	16/30 $\cdot$ 56/28	$0.56$	text
	Payoff on a reference the reader must already know	judge	16.8%	14/20 $\cdot$ 13/11	$0.52$	outside	N
	Written as a document or account (report, chronicle, log, letter)	judge	14.9%	19/21 $\cdot$ 9/6	$0.48$	text
	Scenes of dialogue that do not change the situation	judge	11.2%	7/5 $\cdot$ 13/10	$0.51$	text
HashtagWars pick	Minimal-edit substitution into a known title or lyric	judge	50.5%	– $\cdot$ 73/61	$0.50$	outside	N
BBC most-read	Curiosity, oddity or explainer	judge	41.6%	65/12 $\cdot$ 15/68	$0.37$	text	C

Appendix

Table 41: Text properties of high-gap items that survived a hand check, measured on every held-out item of the task. “Share of items” is the share of all held-out items with the property. The next column gives its share among high-gap and matched items, first among preferred items, then among not-preferred ones (– where the high-gap group has no items of that class). Label AUC is that of the property’s indicator over all items (above .5 means more common among preferred items). Flag: N marks a novelty or familiarity property, C a curiosity or surprise property.

Task	added	gap: raw $\to$ with them	declared	gap within strata	both	$\mathcal{VA}$ : raw $\to$ with	$\mathcal{VAT}$ : raw $\to$ with
	descriptions in the bank		new spurious channels
Story upvotes	9	$.121\to.122$	3	$.109$	$.109$	$.667\to.666$	$.783\to.784$
Joke upvotes	7	$.025\to.031$	3	$.026$	$.034$	$.716\to.712$	$.740\to.743$
HashtagWars pick	4	$.175\to.218$	3	$.181$	$.226$	$.527\to.485$	$.572\to.577$
BBC most-read	11	$.066\to.061$	2	$.066$	$.061$	$.754\to.759$	$.816\to.816$
Press pickup	3	$.107\to.118$	7	$.087$	$.076$	$.644\to.633$	$.720\to.702$

Appendix

Table 42: Implementing every description from the item-level reads, on all held-out items. “Descriptions in the bank”: genuine and measurable descriptions, verified or not, added as bank columns and the stack refit. “New spurious channels”: channels not already in the task’s ledger, with both models read within the strata of up to four of them. Gap is

\max(T,\mathcal{VAT})-\mathcal{VA}

in AUC on the raw frame (the bank and the text-only full model refit together), so the raw values differ from Table 23 .

Task	Excerpt (preferred item; the full model rates it well, the bank poorly)	$p_{T}$	$p_{\mathcal{VA}}$
Story upvotes	“The creatures were all smiles. A bunch of tall, pale, ghostly things with these dark smiles.” (prompt: humans are a thousand times more sensitive to the smell of petrichor than sharks are to blood)	$.81$	$.18$
Story upvotes	“Before we first discovered alien life, our best astronomers believed that we were the only ones in our galaxy.” (a future history in essay form, no scene and no dialogue)	$.82$	$.10$
Joke upvotes	“I like to imagine the guy who invented the umbrella was going to call it the ‘brella’… But he hesitated…”	$.88$	$.28$
Joke upvotes	“These no nut November memes / They’re really getting out of hand”	$.72$	$.18$
Math.SE answer votes	“Yes. Since $\mathbb{Z}$ is free, the sequence splits and you get what you want.”	$.65$	$.27$
Math.SE accepted answer	“This may actually be a matter of ‘just practice more.’ If you’ve done enough factoring, you’ll recognize the coefficients…”	$.51$	$.21$

Appendix

Table 43: Examples of preferred high-gap items.

p_{T}

and

p_{\mathcal{VA}}

are the calibrated probabilities of the true label under the text-only full model and the articulated model.

Task	Description	Verdict	Reason
Story upvotes	Uses the prompt as a springboard and goes elsewhere	invalid	fires on 73% of stories; most flagged stories carry out the prompt
	Single narrating voice (under 3% of characters in quotes)	noisy	dialogue marked with dashes passes; the cut splits mostly narrated stories
	Withholds a key fact about its situation or narrator	noisy	fires on 49%; misses stories that do withhold one
	Serialised part or excerpt	noisy, process	records how the story was posted
Joke upvotes	Pun by sound or spelling, including one left unspelled	noisy	flags non-puns; misses puns whose answer is not written out
	Deadpan literal statement that refuses to be a joke	noisy	reads title-only fragments as deadpan

Appendix

Table 44: Descriptions set aside after the hand check. “Noisy”: between two and five of eight flagged items show the property, or two or more unflagged items do. “Invalid”: the flag does not track the property. “Process”, “format” and “authorship”: the property describes how or by whom an item was posted or produced, not what it says. “Data”: the flag tracks missing text. Noisy is not refuted; several of the readers’ strongest descriptions are here because our patterns and criteria did not measure them cleanly.

task	readout	units	hit (artic.)	hit (full)	hit (random)	regret	95% CI	swap rate	wins full:artic.
peer review: citations ∗	pairwise	56,901 pairs	.714	.884	.500	+.171	[+.125, +.216]	0.27	12565:2852
humor: #HashtagWars picks	grouped	8 groups	.125	.250	.099	+.125	[ $-$ .250, +.500]	1.00	2:1
peer review: accept/reject ∗	pairwise	383,426 pairs	.668	.777	.500	+.109	[+.078, +.140]	0.31	80645:38878
N&C: rule change	grouped	69 groups	.420	.522	.453	+.101	[ $-$ .043, +.246]	0.65	16:9
humor: joke upvotes	grouped	10 groups	.900	1.000	.492	+.100	[+.000, +.300]	1.00	1:0
N&C: received response	grouped	20 groups	.750	.850	.684	+.100	[ $-$ .100, +.300]	0.65	3:1

Appendix

Table 45: Selection regret on all sixteen tasks. Columns: readout (grouped or pairwise); units (decidable groups or pairs); hit rate of the articulated model, of the full model and of a uniform random pick; regret with its 95% grouped-bootstrap interval; swap rate (share of units on which the two models name different items); and, on those units, how often the full model’s pick vs. the articulated model’s was the community’s choice. ∗ : interval excludes zero. † : New Yorker caption-contest tasks; we do not include these results in any main-text table or average, as the task is multimodal (the cartoon each caption answers is unavailable to every instrument).

task	criteria	decidable groups	induced gap	regret	$\rho$ [95% CI]	$p$
math.SE: answer votes	92	1,140	.17–.77	.021–.119	+.87 [.57,.97]	$<$ .001
creative writing: upvotes	144	523	.43–.80	.061–.174	+.77 [.40,.95]	$<$ .001
math.SE: accepted answers	80	992	.41–.77	.033–.107	+.71 [.28,.93]	.002
humor: joke upvotes	124	10	.10–.59	.000–.600	+.65 [.22,.87]	.007
humor: #HashtagWars picks	106	8	.56–1.18	$-$ .125–.250	+.64 [.16,.91]	.007
journalism: editorial pickup	155	8	.25–1.06	.000–.500	+.54 [.03,.88]	.031

Appendix

Table 46: Within-task dose-response between the induced gap and selection regret. Sixteen points per task: random

k

-criterion subsets of the articulated bank (

k\in\{4,8,16,32,64\}

, three draws) plus the full bank. Columns: bank size; decidable groups (groups containing both a chosen and an unchosen item); range of the induced gap and of regret across the sixteen points; Spearman

\rho

over the sixteen points with its 95% bootstrap interval (3,000 resamples of the points) and its

p

-value. Peer-review tasks use pairwise readouts, where regret equals the pairwise-AUC difference and

\rho{=}1

carries no information; they are excluded from the eight-task summary.

task	removed ( $k{=}32$ )	bank AUC	induced gap	regret	swap rate
creative writing: upvotes	none (full bank)	.665	.435	.061	.46
	32 most predictive	.652	.478	.121
	32 least predictive	.662	.446	.096
	32 random (mean of 3)	.662	.446	.091
math.SE: accepted answers	none (full bank)	.569	.518	.067	.45
	32 most predictive	.550	.652	.080

Appendix

Table 47: Criterion removal at matched induced gap on the three tasks with enough decidable groups. Columns: the articulated bank’s held-out AUC after removal; the induced gap

(\mathrm{AUC}_{VAT}-\mathrm{AUC}_{VA})/(\mathrm{AUC}_{VAT}-.5)

; selection regret (full model minus articulated model, per group); and the full bank’s swap rate. Random removal is the mean of three draws; the refit noise on regret is about

.01

(creative writing) and

.001

(Math.SE).

task	pools	real base rate	agree	full on artic.	artic. on full	$A^{\prime}$ [95% CI]	artic. / full	artic. / full
				top pick above the floor			real vs. generated	picks a real item
creative writing (150 prompts)	150	.11	.26	+.210	+.176	$+.034~[-.025,.095]$	.74 /.54	.39 /.19
creative writing (402 prompts)	402	.25	.25	+.264	+.216	$+.047~[.009,.084]$	.71 /.53	.74 /.55
joke upvotes	300	.08	.26	+.234	+.183	$+.051~[.004,.096]$	.80 /.58	.44 /.23
Math.SE answer votes	150	.15	.13	+.121	+.133	$-.011~[-.087,.065]$	.46 /.48	.26 /.27

Appendix

Table 48: Pool readouts on three tasks, Llama-3.1-8B-Instruct candidates excluded (see text). Columns: pools; share of real items per pool; how often the two models name the same top item; the rank, above the pool’s mean, that the other model’s top pick reaches on each channel; the asymmetry

A^{\prime}

(full model’s pick on the articulated channel minus the articulated model’s pick on the full channel) with a 3,000-draw bootstrap over pools; real-versus-generated AUC of each model’s score; share of pools in which each model’s top pick is a real item. Creative writing is the run of Section 5.2 restated without the Llama generator; jokes use twelve generated completions of a real joke’s title plus the joke; Math.SE uses twelve generated answers plus both real answers to the question.

Explore similar work

Jun 4, 2026cs.CL

What Do People Actually Want From AI? Mapping Preference Plurality

Large Language Models (LLMs) are often fine-tuned through Reinforcement Learning from Human Feedback (RLHF) to align with people's preferences and values. However, this method has known limitations: it aggregates conflicting preferences, often relies on unrepresentative samples, and uses only binary comparisons. Analysing 1,500 open-ended responses from the PRISM dataset across 75 countries, we examine what people actually want from AI systems and reveal concrete failures of current methods. We find that different people want different things: most values are requested by fewer than a quarter of respondents, with truthfulness the sole exception at 49%. Furthermore, the same words hide divergent meanings: when people describe what they mean by "truthfulness", they reveal distinct, potentially incompatible, epistemological bases, as some ask for sourced claims, some for expert opinions, and some even ask for unpopular views. Certain capabilities, namely how human-like a model behaves, and some features, like AI guardrails, are outright controversial, with some desiring them and others rejecting them. We additionally find that people often use contextual distinctions (what AI should do "by default" versus "if requested") that binary comparisons cannot capture. These findings expose fundamental problems in current alignment practices. When 49% request truthfulness but define it differently, this is unlikely to be captured by a single reward model. The persistence of high hallucination rates in well-funded models, despite users' clear demands for accuracy, suggests that current methods fail to identify actual preferences. This paper sheds light on the situated, contested, imperfect signals that are currently being flattened into universal preference models, a practice others have characterised as epistemic violence.

Julia Sepúlveda Coelho, Scott A. Hale

Oxford Internet Institute, University of Oxford, United Kingdom · Meedan, United States

Artificial Intelligence Alignment Preference Alignment Learning Truth Open-Ended Responses

Ask PDF

Jun 26, 2026cs.LG

Democratic ICAI: Debating Our Way to Steering Principles from Preferences

Preference-based alignment often struggles to capture the reasoning that underlies human judgments. Many evaluations rely on multiple interacting criteria, yet pairwise labels reveal only the final choice rather than the considerations that shape preferences. Inverse Constitutional AI (ICAI) improves interpretability in decision making by summarizing preferences into natural-language principles, but its single-pass explanations miss much of the nuance involved in complex decisions. We introduce Democratic ICAI (DICAI), a novel approach that gathers multiple competing rationales through structured persona debate, offering a broader and more expressive account of the factors influencing each comparison. From these richer signals, we derive clearer and more comprehensive steering principles and use them to guide preference decision modeling through both LLM-based and decision-tree judges, as well as downstream model training via constitution-induced preference labels. Experiments on creative preference benchmarks, MuCE-Pref and LiTBench, across multiple creative task categories show that Democratic ICAI yields a more faithful preference structure. It improves average preference prediction across tasks relative to deliberative prompting and principle-based baselines, while producing constitutions that LLM annotators prefer.

Kevin Kingslin, Anish Natekar, Ashutosh Ranjan +3

TCS Research, India

Preference Alignment Learning Deliberation Democracy Decisions Steering

Ask PDF

May 15, 2026cs.LG

Learning What Evaluators Value: A Reliable Approach to Modeling Evaluator Preferences

In many applications, human and LLM evaluators use assessments of relevant criteria to create an overall evaluation for an item or individual. For example, in admissions, committees assess candidates on attributes such as test scores, GPA, and research experience to evaluate their overall fit for the program. Another example arises in medical care where clinicians use patient reports of symptoms to consider preliminary diagnoses and assess risks. Each setting involves mapping multiple criteria to an overall evaluation -- a process that reflects the evaluator's underlying preferences. We focus on the fundamental question of learning these preferences. Many applications of this problem make specific modeling assumptions on evaluator preferences that may be substantially violated in the real world. We make the minimal assumption that the preference function is coordinate-wise non-decreasing, which is reasonable in a large number of evaluation settings. We theoretically characterize the severity of model mismatch for many common assumptions and show that it can lead to significant issues for learning evaluator preferences and other important downstream tasks. We then present an algorithm for learning evaluators' preferences that is robust to model mismatch. We prove theoretically that our algorithm can learn any preference function without sacrificing performance when the linearity assumption holds. Evaluations of our algorithm with synthetic simulations and real-world data confirm its ability to learn preferences robustly and illustrate key aspects of LLM and human preferences.

Madeline Celi Kitch, Nihar B. Shah

Carnegie Mellon University

Preference Alignment Learning Rater Mismatch Reliable

Ask PDF