DelegationBench: Measuring When AI Agents Should Ask Before Acting

Shiva Pochampally

cs.AIOct 4, 2026

DelegationBench: Measuring When AI Agents Should Ask Before Acting

Authors: Shiva Pochampally

Abstract

AI agents that send emails, edit files, and make purchases must decide when to act on their own and when to check with the user first. This decision is usually evaluated by showing a model a proposed action, asking whether it should proceed, and scoring agreement with human labels. We introduce DelegationBench to test whether such scores can be trusted. It has 156 scenarios with four possible responses (act, ask for permission, ask for missing information, refuse), and most scenarios come in matched pairs that change a single feature: whether the action was requested, what is at stake, whether it can be undone, or who will see it. Across ten models from five families, agreement scores mislead in three ways. A simple keyword rule, which we wrote after seeing the benchmark, agrees with our annotators more often than eight of the models, yet its decision changes in only 9 of 48 matched pairs. Equivalent ways of asking the same question change how often a model acts by up to 52.5 percentage points. And every model stops to ask the user less often when it must carry out the task with tools than when it judges a proposed action. When rules are stated explicitly, the same models follow them almost perfectly, so the gaps are not explained by a general inability to follow rules. We release the benchmark and evaluation tools and recommend reporting these properties separately rather than as one score.

Figures & tables

Figure 1: Overview. Each scenario gives an instruction, context, and one proposed action. Matched pairs change a single feature, here whether the deletion can be undone. In the first setting a model judges the shown action and picks one of four responses; in the second it must act with tools. The bottom row lists the three gaps we measure.

Figure 2: Study design. The judgment study and the elicitation-format study show the model a proposed action. Tool-Loop gives the same scenarios as tasks, Post-Approval continues every episode that ended with a question, and two blinded reviewers audit a sample of the resulting tool calls.

Figure 3: Agreement with the annotators (x-axis) versus pair responsiveness, the share of matched pairs in which the act rate moves in the designed direction (y-axis). The keyword rule (star) agrees more often than eight models but responds to only 9 of 48 pairs. Responsiveness measures reaction to the change, not correctness.

Model	4-way agree	Act/non-act agree	Under-intervention	Over-intervention
Gemini 3.5 Flash-Lite	73.4	75.7	20.2	29.0
Gemini 3.6 Flash	71.9	75.6	25.3	23.5
Claude Sonnet 5	68.7	71.1	36.5	20.0
Claude Opus 4.8	65.5	68.7	40.3	20.9
GPT-5.6 Terra	60.4	66.3	43.8	22.0
GPT-5.6 Luna	60.0	67.4	41.5	22.3

Table 1: Agreement with the annotators’ majority label on 149 scenarios (%). Under-intervention: acting where the majority would not. Over-intervention: not acting where the majority would.

Figure 4: (A) Share of the 48 scenarios ending in a request to the user, when judging a shown action (circles) and when acting with tools (squares). Every model asks less when acting. (B) On the 29 scenarios where exact arguments are measurable, the ordering of models by judged act rate differs from their ordering by how often they call the proposed tool. Inset: strict argument matching versus human-judged correspondence.

Model	Natural policy	Structured policy	Decision given	$\tau$ -bench
GPT-5.6 Luna	100.0	100.0	100.0	100.0
Claude Sonnet 5	97.6	100.0	99.4	100.0
Gemini 3.5 Flash-Lite	97.3	98.5	100.0	100.0

Table 2: Balanced accuracy (%) when the correct decision follows from an explicit rule. The synthetic columns use 192 calls per model; the

\tau

-bench column uses 68 calls per model on 17 decision points. Invalid responses count as errors. “Decision given” supplies the correct decision and tests only whether the model carries it out.

Appendix figures & tables36 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: One matched pair for each feature. Within a pair, only the text in the “Low” and “High” boxes differs.

System	Judges a proposed action	Separates permission from information	Tests format sensitivity	Tests acting with tools	Continues after the agent asks
DelegationBench	Yes	Yes	Yes (5 formats)	Yes (Tool-Loop)	Yes (Post-Approval; denial checked against state)
When2Call ( Ross et al., 2025 )	Partial (whether to call a tool)	Not reported	Yes (multiple choice vs. free form)	Not reported	Not reported
Suri et al. ( Suri et al., 2026 )	Not reported	Partial (information side only)	Not reported	Not reported	Not reported
Janus ( Brigham et al., 2026 )	Yes	No	Not reported	Not reported	Partial
AuthBench ( Yan et al., 2026 )	Yes	Not reported	Not reported	Not reported	Not reported
AgentAbstain ( Liu et al., 2026 )	No (agent behavior only)	No	Not reported	Yes	Not reported

Appendix

Table 3: Measurements reported by related benchmarks. The table compares scope, not quality.

Provider	Model identifier	Setting
OpenAI	gpt-5.6-sol	temperature 1; medium reasoning
OpenAI	gpt-5.6-terra	temperature 1; medium reasoning
OpenAI	gpt-5.6-luna	temperature 1; medium reasoning
Groq	openai/gpt-oss-120b	temperature 1; medium reasoning
Groq	openai/gpt-oss-20b	temperature 1; medium reasoning
Anthropic	claude-sonnet-5	adaptive thinking; medium effort

Appendix

Table 4: Model identifiers and settings. We also ran GPT-OSS-120B on a second provider (Cerebras, 780 requests) to check that results do not depend on the serving backend; it is not counted as an eleventh model.

Figure 6: Decision composition for the ten models. Only Qwen produces invalid outputs. Models with similar Act rates can split their non- Act decisions very differently.

Model	ACT	ASK	REQ	REF	ASK $\mid\neg$ ACT	REF $\mid\neg$ ACT	Entropy
GPT-5.6 Sol	65.6	22.2	10.1	2.1	64.6	6.0	0.049
GPT-OSS-120B (Groq)	64.0	16.9	11.7	7.4	47.0	20.6	0.137
GPT-OSS-20B (Groq)	65.3	13.1	12.4	9.2	37.6	26.6	0.157
GPT-5.6 Terra	58.5	22.9	13.7	4.9	55.2	11.7	0.067
GPT-5.6 Luna	57.6	23.6	14.1	4.7	55.6	11.2	0.087
Claude Sonnet 5	55.8	31.2	9.1	4.0	70.4	9.0	0.037

Appendix

Table 5: Decision composition (%) and mean normalized entropy across the five trials. The two conditional columns divide by the number of valid non- Act decisions.

Model	5/5	$\geq$ 4/5	Pairwise	Norm. entropy
GPT-5.6 Sol	87.8	96.2	0.944	0.049
GPT-OSS-120B (Groq)	68.6	84.6	0.840	0.137
GPT-OSS-20B (Groq)	66.0	81.4	0.821	0.157
GPT-5.6 Terra	84.6	92.9	0.922	0.067
GPT-5.6 Luna	80.1	92.3	0.901	0.087
Claude Sonnet 5	91.7	96.2	0.958	0.037

Appendix

Table 6: Stability across repeated generations, over 156 scenarios per model. 5/5 and

\geq

4/5 give the share of scenarios (%) where at least that many of the five trials return the same decision. Pairwise is the mean agreement over all pairs of the five trials.

Domain	ACT median [range]	ASK median [range]	Majority agreement median [range]	Stability median [range]
Calendar	65.2 [56.6, 72.4]	13.8 [8.3, 24.8]	68.6 [59.3, 73.8]	82.8 [48.3, 96.6]
Code/system	77.1 [46.7, 86.7]	9.6 [5.0, 45.8]	55.2 [50.0, 73.6]	87.5 [79.2, 95.8]
Communication	62.2 [51.1, 63.7]	25.4 [16.3, 30.4]	69.2 [63.1, 73.1]	90.7 [74.1, 100]
Files	30.3 [18.9, 42.3]	42.6 [26.3, 59.4]	58.2 [37.6, 70.9]	75.7 [57.1, 97.1]
Publishing/social	92.1 [85.7, 100]	4.3 [0, 11.8]	81.4 [72.9, 87.1]	85.7 [57.1, 100]
Purchasing	50.4 [12.6, 77.0]	35.7 [7.4, 76.3]	49.2 [20.0, 90.4]	79.6 [48.1, 96.3]

Appendix

Table 7: Behavior by domain: median and range over the ten models (%). Majority agreement is agreement with the annotators’ majority label. Stability is the share of scenarios where all five trials agree. Domains differ in size, so these ranges are descriptive.

Family	Lexical cue	Consequential context	Interaction
Calendar change	5.0	$-33.0$	10.0
Code change	6.0	$-94.0$	12.0
Email send	1.0	$-1.0$	2.0
File cleanup	2.0	$-28.0$	$-4.0$
Purchase commitment	10.0	$-10.0$	20.0
Social publish	7.0	$-11.0$	14.0

Appendix

Table 8: Mean Act -rate contrasts over the ten models in the six lexical 2-by-2 families (percentage points). Each family has four scenarios and one contrast per model. Signs follow each family’s coding.

Model	Consistency	Large shifts	Model	Consistency	Large shifts
Sol	93.8	1	Sonnet	87.5	2
GPT-OSS-120B	81.3	2	Opus	97.5	0
GPT-OSS-20B	82.5	1	Gemini Flash	100.0	0
Terra	90.0	2	Flash-Lite	92.5	1
Luna	82.5	2	Qwen	83.8	0

Appendix

Table 9: Decision consistency between the 16 paraphrases and their parent scenarios, matched by trial index (%). Large shifts counts paraphrases whose Act rate differs from the parent’s by at least 60 points.

Probe	ACT	ASK	REQUEST	Stable models	Majority label
Inbox cleanup 1	96	2	2	8	ACT
Inbox cleanup 2	36	54	8	5	ACT
Storage cleanup 1	54	46	0	6	ASK
Storage cleanup 2	12	86	0	7	ASK
Calendar management 1	94	4	0	8	ASK
Calendar management 2	44	44	10	5	ASK

Appendix

Table 10: The eight boundary probes, each over 50 trials (ten models, five trials each; %). Stable models counts models that give the same decision on all five trials. The small remainder is invalid; no trial chose Refuse .

Model	Offline ASK	Tool-Loop ASK	Difference
Sol	19.2	15.8	$-3.3$
GPT-OSS-120B	15.8	4.6	$-11.3$
GPT-OSS-20B	10.0	0.4	$-9.6$
Terra	17.5	12.5	$-5.0$
Luna	24.6	14.6	$-10.0$
Sonnet	32.5	20.0	$-12.5$

Appendix

Table 11: Rate of ending in Ask on the same 48 scenarios, offline and in Tool-Loop (%). Qwen and GPT-OSS-20B use their valid-cell denominators, 209 and 239. The two settings differ in several ways at once: whether the candidate action is shown, the response format, and whether the model must construct the call itself.

Model	Factor	Mean	95% CI	$-$	0	$+$	Sign-flip $p$
GPT-5.6 Sol	scope	-0.917	[-1.000, -0.750]	11	1	0	0.0010
GPT-5.6 Sol	stakes	-0.417	[-0.667, -0.183]	7	5	0	0.0156
GPT-5.6 Sol	reversibility	-0.467	[-0.733, -0.200]	7	5	0	0.0156
GPT-5.6 Sol	external_visibility	+0.000	[-0.150, +0.183]	2	9	1	1.0000
GPT-OSS-120B (Groq)	scope	-0.883	[-1.000, -0.700]	11	1	0	0.0010
GPT-OSS-120B (Groq)	stakes	-0.600	[-0.800, -0.400]	11	1	0	0.0010

Appendix

Table 12: Pair effects for each model and factor. Mean is the high-minus-low difference in Act rate, averaged over the 12 pairs for that factor. The 95% interval resamples pairs. The

-

, 0, and

+

columns count pairs whose Act rate falls, stays the same, or rises from the low to the high variant. The sign-flip

p

-value is exact, enumerating all 4,096 sign assignments.

Source	Scheme	$n$	ACT	ASK	REQUEST	REFUSE
gpt-5.6-sol	emitted	780	65.6	22.2	10.1	2.1
gpt-5.6-sol	joint_consensus	775	66.1	22.2	9.7	2.1
gpt-5.6-sol	claude	779	65.7	22.3	9.6	2.3
gpt-5.6-sol	gemini	780	65.6	22.1	10.0	2.3
openai/gpt-oss-120b	emitted	780	64.0	16.9	11.7	7.4
openai/gpt-oss-120b	joint_consensus	766	65.1	19.8	10.4	4.6

Appendix

Table 13: Decisions as emitted by each model and as recoded from the written rationale. Recoding was done blind to the emitted label by two model judges (Claude and Gemini); joint_consensus keeps records where both judges agree. Values are percentages of the retained records.

Held out	$n_{4}$	Four-way [95% CI]	$n_{2}$	Binary [95% CI]	$n_{r}$	Reason [95% CI]
A	101	.782 [.703,.861]	115	.896 [.835,.948]	48	.729 [.604,.854]
B	107	.738 [.654,.822]	125	.824 [.752,.888]	41	.854 [.732,.951]
C	99	.798 [.717,.879]	122	.844 [.779,.910]	37	.946 [.865,1.000]

Appendix

Table 14: Agreement of each held-out annotator with the other two, on scenarios where those two agree.

n_{4}

n_{2}

, and

n_{r}

are the four-way, binary, and reason denominators described in the text.

Model	Annotator A	Annotator B	Annotator C
Gemini 3.5 Flash-Lite	65.4	57.3	78.8
Gemini 3.6 Flash	64.4	55.6	77.9
Claude Sonnet 5	60.4	52.7	77.9
Claude Opus 4.8	56.2	51.2	72.8
GPT-5.6 Terra	56.8	49.1	68.6
GPT-5.6 Luna	56.4	47.8	67.4

Appendix

Table 15: Four-way agreement (%) with each individual annotator, over all 156 scenarios and five trials. Rows are sorted by agreement with the annotators’ majority label, so reorderings are visible.

Model	Multiclass Brier	Binary Brier	ACT bias
Gemini 3.5 Flash-Lite	.336	.292	$-.018$
Gemini 3.6 Flash	.415	.337	$+.032$
Qwen 3.6 27B	.455	.405	$+.088$
Claude Sonnet 5	.463	.417	$+.109$
GPT-5.6 Terra	.545	.463	$+.136$
Claude Opus 4.8	.546	.484	$+.127$

Appendix

Table 16: Calibration over all 156 scenarios. Brier scores compare the model’s decision frequencies with the annotators’ vote shares (lower is better). The binary score uses Act versus non- Act . ACT bias is the model’s Act rate minus the annotators’ Act vote share.

Policy	4-way	All pairs	Scope	Unchanged	Paraphrase
Ten-model range	50.6–73.4%	45.8–70.8%	11–12/12	25.0–50.0%	81.3–100%
Keyword rule	69.1%	9/48	3/12	39/48	15/16
Annotator ACT vote share	–	20/48	9/12	27/48	83.3% ∗

Appendix

Table 17: Agreement and pair behavior for the models, the keyword rule, and the annotators’ vote share. 4-way is agreement with the annotators’ majority label. All pairs is pair responsiveness: the share of the 48 matched pairs in which the Act rate moves in the direction the pair was designed to test (a strict decrease from the low to the high variant). Scope counts this for the 12 scope pairs. Unchanged counts pairs whose Act rate is exactly equal across the two variants. Paraphrase is decision agreement between paraphrases and their parent scenarios. ∗ Exact decision agreement, matched by annotator.

Figure 7: Under- and over-intervention against the annotators’ majority label (lower is better on both axes). Starred points are Pareto-efficient.

Model	Majority agreement	Brier $\downarrow$	Responsiveness	Under	Over	Elicit. shift	TL tool/strict	Approve act
GPT-5.6 Sol	58.1	0.599	56.3	54.8	19.4	8.3	64.8/32.4	76.3 (29/38)
GPT-OSS-120B	54.5	0.599	64.6	55.3	23.5	47.5	66.9/36.6	72.7 (8/11)
GPT-OSS-20B	50.6	0.648	56.3	58.8	25.8	52.5	77.1/37.5	– †
GPT-5.6 Terra	60.4	0.545	62.5	43.8	22.0	30.0	62.8/45.5	90.0 (27/30)
GPT-5.6 Luna	60.0	0.547	64.6	41.5	22.3	8.3	60.0/34.5	77.1 (27/35)
Claude Sonnet 5	68.7	0.463	56.3	36.5	20.0	11.7	60.7/38.6	83.3 (40/48)

Appendix

Table 18: DelegationBench report card. Each column is a separate raw metric; none is combined into a score. Majority agreement: four-way agreement with the annotators’ majority label (%). Brier: multiclass Brier score against the three annotators’ vote distribution over all 156 scenarios (lower is better). Responsiveness: pair responsiveness (%; Appendix I ). Under and Over: under- and over-intervention (%). Elicit. shift: largest absolute change in Act rate across the five elicitation formats (percentage points). TL tool/strict: in Tool-Loop, the rate of calling the candidate tool and the rate of exactly reconstructing the candidate call (%). Approve act: share of Post-Approval continuations that act after approval (%, with numerator/denominator). Exact reconstruction understates correspondence: a normalized matcher accepts 89.0% of Tool-Loop candidate-tool calls, and two blinded reviewers from the authors’ lab judge 93.9% and 96.0% of them equivalent to the candidate. A dash marks the two models whose Post-Approval results are incomplete (see Table 21 ).

Stage	Question	Recommended measurement
Candidate judgment	Is this action authorized?	ACT/ASK/REQUEST/REFUSE against matched controls, not aggregate agreement alone
Elicitation	Does presentation change the operating point?	Counterbalanced/alternative formats; report the shift, not one number
Permission resolution	What does an ASK actually require?	Classify permission vs. information vs. mixed before scoring resolution
Constructed action	What tool/action does the agent propose?	Candidate-tool dispatch plus argument checks, not judgment alone
Semantic correspondence	Does the call implement the candidate?	Audited correspondence; strict matching is a conservative floor
State update	Did the authorized/denied outcome occur?	Deterministic state transition check when an environment exists

Appendix

Table 19: Recommended reporting protocol. Each stage is reported separately; no stage substitutes for another, and the stages are not combined into one score.

Model	Valid / scheduled	Offline ACT	Strict match	Cand.-tool	Any-tool	$r_{\rm strict}$	$r_{\rm tool}$
Sol	145/145	72.4	32.4	64.8	64.8	.467	.671
GPT-OSS-120B	145/145	70.3	36.6	66.9	66.9	.376	.583
GPT-OSS-20B	144/145	68.8	37.5	77.1	80.6	.490	.610
Terra	145/145	69.7	45.5	62.8	62.8	.396	.440
Luna	145/145	64.8	34.5	60.0	60.0	.488	.595
Sonnet	145/145	64.1	38.6	60.7	60.7	.387	.437

Appendix

Table 20: Tool-Loop results on the 29 main scenarios. Offline ACT is the model’s Act rate on the same scenarios in the main study. Strict match is the rate of exactly reconstructing the candidate call; the human audit (Appendix L ) shows this rule is conservative. Cand.-tool and Any-tool are the rates of calling the candidate tool and any tool. Rates are percentages. The

r

columns correlate, across scenarios, the offline Act rate with the strict-match or candidate-tool rate.

Model	Parent	ASK/eligible	Valid	Act	Cand. tool	Meas. eligible	Meas. tool	Strict	Ask again	Request	Refuse	Other	Tech.
Sol	240	38/38	38	29	29	23	19	11	1	4	0	4	0
GPT-OSS-120B	240	11/11	11	8	8	8	6	0	1	0	1	1	0
Sonnet	240	48/48	48	40	40	25	22	9	3	4	0	1	0
Gemini Flash	240	16/16	16	15	15	10	10	5	0	1	0	0	0
Terra	240	30/30	30	27	27	19	17	9	3	0	0	0	0
Luna	240	35/35	35	27	27	26	23	10	3	3	1	1	0

Appendix

Table 21: Post-Approval outcomes by model. Parent is the number of valid Tool-Loop trajectories. ASK/eligible gives the trajectories that ended in Ask and were continued. Valid is the number of valid continuations. Act and Cand. tool count continuations that call any tool and that call the candidate tool, respectively. Meas. eligible counts valid continuations whose scenario allows the strict reconstruction check; Meas. tool further requires a candidate-tool call. Strict counts exact reconstructions. The last five columns split the continuations that do not act (Valid minus Act). † marks the two models with incomplete parent trajectories, excluded from the main totals.

Rule stratum	Audit $n$	R1 equiv.	R1 non-equiv.	R1 uncertain	R2 equiv.	R2 non-equiv.
Strict ACCEPT	100	100	0	0	100	0
Strict REJECT	100	88	9	3	92	8

Appendix

Table 22: Unweighted audit counts by strict-rule outcome. Reviewer 2 never used UNCERTAIN. Population estimates below use inverse-probability weights from the sampling design.

Probability-weighted quantity	Reviewer 1	95% interval	Reviewer 2	95% interval
Strict-ACCEPT precision	100.0	96.3–100.0	100.0	96.3–100.0
Strict-REJECT false rejection	87.1	80.7–92.5	91.4	86.1–95.8
Overall semantic correspondence	94.6	91.9–96.9	96.4	94.1–98.2
Tool-Loop correspondence $\mid$ candidate tool	93.9	90.9–96.5	96.0	93.4–98.0
Overall uncertain mass	1.3	0.4–2.6	0.0	0.0–0.0

Appendix

Table 23: Population estimates for each reviewer (%), weighted by inverse sampling probability. Strict-ACCEPT precision is the share of strict accepts judged equivalent. Strict-REJECT false rejection is the share of strict rejects judged equivalent. Intervals use 10,000 stratified bootstrap resamples, except for strict-ACCEPT precision, where every sampled call succeeds; there we report a Wilson binomial interval.

Automated category	$n$	Eq.	Non-eq.	Unc.	Eq.	Non-eq.	Unc.
		Reviewer 1			Reviewer 2
Representation candidate	79	78	0	1	78	1	0
Changed value	12	7	3	2	11	1	0
Narrower or safer	6	0	6	0	0	6	0
Changed target	3	3	0	0	3	0	0

Appendix

Table 24: Human labels for the sampled strict rejects, by automated taxonomy category. The taxonomy was fixed before either review and is used for explanation only.

Family	Overall acc.	ACT acc.	ASK acc.	Balanced acc.	Cell consistency
GPT-5.6	89.2%	78.3%	100%	89.2%	97.9%
Claude	87.9%	75.8%	100%	87.9%	97.9%
Gemini	85.8%	71.7%	100%	85.8%	97.9%
GPT-OSS	83.8%	69.2%	98.3%	83.7%	91.7%
Qwen	86.2%	72.3%	100%	86.1%	97.9%
Pooled (1,199 valid)	86.6%	73.5%	99.7%	86.6%	–

Appendix

Table 25: Accuracy on the 48 standing-policy cases (5 trials, 5 families). The lower accuracy on cases where acting is permitted comes mostly from four templates with known construction problems; see text.

Condition	N valid	Accuracy (lenient / strict)
Permission present	250	90.4% / 89.6%
Permission absent	250	79.6% / 64.8%
Information complete	250	91.6% / 91.6%
Information missing	250	78.4% / 62.8%

Appendix

Table 26: Permission

\times

information control (5 templates, 20 cases, 5 trials, 5 families). “Lenient” counts a question that asks for both the fact and permission as a correct information request.

Condition	Family	Compliance rate	N valid/scheduled
APPROVAL	GPT-5.6	100%	15/15
	Claude	66.7%	15/15
	Gemini	100%	15/15
	GPT-OSS	40.0%	15/15
	Qwen	100%	13/15
MODIFICATION	GPT-5.6	100%	15/15

Appendix

Table 27: Communication templates after the repair: share of episodes that carry out exactly the approved or modified action. No episode fabricated message content. The remaining non-compliance (Claude and GPT-OSS under approval) consists of asking again after approval.

Model	Condition	$N$	Valid	Balanced acc.	Completion
Gemini 3.5 Flash-Lite	Oracle decision	192	192	100.0	100.0
Gemini 3.5 Flash-Lite	Natural language	192	189	97.3	96.4
Gemini 3.5 Flash-Lite	Structured policy	192	189	98.5	98.8
GPT-5.6 Luna	Oracle decision	192	192	100.0	100.0
GPT-5.6 Luna	Natural language	192	192	100.0	100.0
GPT-5.6 Luna	Structured policy	192	192	100.0	100.0

Appendix

Table 28: Policy format. Rates are percentages. Unauthorized is the rate of attempting the action on gated cases; completion is the rate of reaching the target state on permitted cases. Invalid responses count as failures in balanced accuracy.

Model	Condition	$N$	Valid	Balanced acc.	Unauthorized	Completion
Gemini 3.5 Flash-Lite	Eligibility rule	192	192	100.0	–	–
GPT-5.6 Luna	Eligibility rule	192	192	97.7	–	–
Claude Sonnet 5	Eligibility rule	192	192	98.1	–	–

Appendix

Table 29: Ordinary-rule control (%). Invalid responses count as failures.

Model	Condition	$N$	Valid	Balanced acc.	Completion
Gemini 3.5 Flash-Lite	Hidden / QUESTION	144	141	97.9	100.0
Gemini 3.5 Flash-Lite	Hidden / ask_user	144	143	99.3	100.0
Gemini 3.5 Flash-Lite	Shown / QUESTION	144	144	100.0	100.0
Gemini 3.5 Flash-Lite	Shown / ask_user	144	140	97.2	97.2
GPT-5.6 Luna	Hidden / QUESTION	144	144	100.0	100.0
GPT-5.6 Luna	Hidden / ask_user	144	144	100.0	100.0

Appendix

Table 30: Candidate visibility and question interface. Columns as in the policy-format table (%).

Model	Condition	$N$	Valid	Balanced acc.	Completion
Gemini 3.5 Flash-Lite	Natural language	68	68	100.0	100.0
Gemini 3.5 Flash-Lite	Structured policy	68	68	100.0	100.0
GPT-5.6 Luna	Natural language	68	68	100.0	100.0
GPT-5.6 Luna	Structured policy	68	68	100.0	100.0
Claude Sonnet 5	Natural language	68	68	100.0	100.0
Claude Sonnet 5	Structured policy	68	68	100.0	100.0

Appendix

Table 31: External decision points adapted from

\tau

-bench. Columns as in the policy-format table (%).

Model	Condition	$N$	Valid	Balanced acc.	Completion
Gemini 3.5 Flash-Lite	Oracle decision	32	32	100.0	100.0
Gemini 3.5 Flash-Lite	Natural language	32	32	100.0	100.0
Gemini 3.5 Flash-Lite	Structured policy	32	31	97.2	100.0
GPT-5.6 Luna	Oracle decision	32	32	100.0	100.0
GPT-5.6 Luna	Natural language	32	32	100.0	100.0
GPT-5.6 Luna	Structured policy	32	32	100.0	100.0

Appendix

Table 32: Development pilot. Columns as in the policy-format table (%).

Model	Study / contrast	Cluster	95% interval
GPT-5.6 Luna	Structured minus natural	Family	[0.0, 0.0]
GPT-5.6 Luna	Structured minus natural	Domain	[0.0, 0.0]
GPT-5.6 Luna	Candidate visibility	Family	[0.0, 0.0]
GPT-5.6 Luna	Question tool	Family	[0.0, 0.0]
GPT-5.6 Luna	Interaction	Family	[0.0, 0.0]
GPT-5.6 Luna	Structured minus natural	Task	[0.0, 0.0]

Appendix

Table 33: Paired contrasts with cluster-bootstrap intervals. Policy-format and external contrasts use balanced accuracy. The interface contrasts (candidate visibility, question tool, and their interaction) use the question rate. Task clusters are the external study’s source tasks.

Model	Fact wording	$N$	Valid	Balanced acc.	Completion
GPT-5.6 Luna	Natural	192	192	99.4	98.8
GPT-5.6 Luna	Paraphrased	192	192	98.8	97.6
Claude Sonnet 5	Natural	192	192	97.0	94.0
Claude Sonnet 5	Paraphrased	192	192	97.0	94.0
Gemini 3.5 Flash-Lite	Natural	192	191	98.9	98.8
Gemini 3.5 Flash-Lite	Paraphrased	192	192	98.8	97.6

Appendix

Table 34: Paraphrased versus natural fact wording, with fresh natural-wording controls. Balanced accuracy counts invalid responses as failures; completion is measured on permitted cases. These rates are scored against the executable policy and are not comparable to agreement with the annotators.

Model	Cluster	Accuracy change (pp)	95% interval
GPT-5.6 Luna	Family	-0.3	[-1.0, 0.0]
GPT-5.6 Luna	Domain	-0.6	[-1.8, 0.0]
Claude Sonnet 5	Family	0.0	[-1.6, 1.6]
Claude Sonnet 5	Domain	0.0	[-1.8, 1.8]
Gemini 3.5 Flash-Lite	Family	-0.2	[-1.6, 1.0]
Gemini 3.5 Flash-Lite	Domain	-0.1	[-1.8, 1.4]

Appendix

Table 35: Paraphrased minus natural-wording balanced accuracy, with paired cluster-bootstrap intervals. The paraphrases are few and were not checked by human raters, so this does not show general robustness to paraphrase.

Explore similar work

Sep 10, 2025cs.CY

HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistants

As humans delegate more tasks and decisions to artificial intelligence (AI), we risk losing control of our individual and collective futures. Relatively simple algorithmic systems already steer human decision-making, such as social media feed algorithms that lead people to unintentionally and absent-mindedly scroll through engagement-optimized content. In this paper, we develop the idea of human agency by integrating philosophical and scientific theories of agency with AI-assisted evaluation methods: using large language models (LLMs) to simulate and validate user queries and to evaluate AI responses. We develop HumanAgencyBench (HAB), a scalable and adaptive diagnostic tool for six behaviors related to human agency. HAB measures the tendency of an AI assistant to Ask Clarifying Questions, Avoid Value Manipulation, Correct Misinformation, Defer Important Decisions, Encourage Learning, and Maintain Social Boundaries. We find low-to-moderate agency support in contemporary LLM-based assistants, with substantial variation across system developers and behaviors. For example, while Anthropic LLMs most support human agency overall, they are the least supportive LLMs in terms of Avoid Value Manipulation. These behaviors do not appear to consistently result from increasing LLM capabilities or instruction-following (e.g., RLHF); we encourage further study of these behaviors so that developers and users can better understand the complexities of modern human-AI interaction.

Benjamin Sturgeon, Daniel Samuelson, Jacob Haimes +1

Apart Research · AI Safety Cape Town · University of Chicago +2

Human-Ai Interaction Human Agency Artificial Intelligence Assistants Analytics

Ask PDF

Jul 28, 2026cs.AI

HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following

Language-model agents are increasingly deployed under standing instructions: a system prompt, a policy file, or a skills document is placed in context, and the agent is trusted to let that document govern every action that follows. Existing benchmarks rarely test this deployment pattern directly; they measure whether an agent can complete a task, not whether a long, binding policy document constrains its behavior over an extended tool-use horizon. We present HANDBOOK_md, a benchmark of 65 agentic tasks modeled on how employees follow company handbooks. Each task places an agent in a self-contained company environment (a file workspace with mock email, chat, calendar, issue-tracking, and commerce services exposed over the Model Context Protocol) and instructs it to carry out routine professional work governed by an expert-written standard operating procedure of 20-124 pages. Tasks span five domains (finance, medical billing, insurance, logistics, and HR) and 10 fictional companies. To resist memorization, every task modifies one of 10 base handbooks, altering the specific rules and thresholds on which grading depends, so no two tasks share the same set of policies. Grading is fully deterministic: each task carries a rubric of programmatic criteria (824 in total) that check both that required actions occurred and that prohibited actions did not. Under strict grading, where a trial passes only if every criterion is satisfied, the strongest evaluated model passes 36.2% of trials, and most frontier models remain below 25%. Failures follow consistent patterns: agents let a plausible but unauthorized in-environment request override the standing policy, perform a required check and then act against its result, lose rule details over long horizons, and report compliance they did not achieve. We release the tasks, environments, and evaluation harness.

Liudas Panavas, Sebastian Minus, Bradley Monton +4

Surge AI*

Model Context Protocol

Ask PDF

May 18, 2026cs.SE

Overeager Coding Agents: Measuring Out-of-Scope Actions on Benign Tasks

Coding agents now run autonomously with shell, file, and network privileges. When a user issues a benign request, the agent sometimes does more than asked: it deletes unrelated files, wipes a stale credentials backup, or rewrites configuration the user never mentioned. We call these scope expansions overeager actions, an authorization problem distinct from capability failures, prompt injection, or sandbox escapes. We present OverEager-Gen, a benchmark dedicated to overeager behavior on benign tasks. Building it surfaces a measurement-validity issue: if a benchmark spells out the authorized scope inside the prompt, the agent stops inferring boundaries and starts pattern-matching declaration text. On Claude Code, stripping the consent declaration alone raises the overeager rate from 0.0% to 17.1% on paired scenarios (McNemar exact p = 2.4 x 10^-4). OverEager-Gen therefore certifies each scenario's discriminative power before admission via a behavioral-gradient validator, audits internal tool calls through a dual-channel stack (PATH-injected shim plus per-agent event streams), and ships byte-identical consent_kept and consent_stripped variants. OverEager-Bench contains 500 validated scenarios and ~7,500 runs across four agent products (Claude Code, OpenHands, Codex CLI, Gemini CLI) and six base models; a 50-sample re-annotation gives Cohen's kappa = 0.73 and rule-judge recall = 1.00. Stripping consent multiplies the overeager rate on every shared base model (Delta in [11.9, 17.2] pp). The framework axis dominates effect size: a permissive cluster (Claude Code, Codex CLI, Gemini CLI) runs at 5.4-27.7% while the ask-to-continue framework (OpenHands) sits at 0.2-4.5% (Fisher p <= 10^-5). Within-framework base-model variance reaches 15.9 pp, indicating that model-layer alignment does not fully propagate through permissive permission gating.

Yubin Qu, Ying Zhang, Yanjun Zhang +4

Griffith University · Wake Forest University · Nanyang Technological University +2

Kernel-Resident Tool Governance Gateway Coding Agents Benign

Ask PDF