Shared and structured inputs undermine collective random choice by reasoning AI agents

Takahiro Ezaki; Naoto Imura; Katsuhiro Nishinari

cs.AIOct 7, 2026

Shared and structured inputs undermine collective random choice by reasoning AI agents

Authors: Takahiro Ezaki, Naoto Imura, Katsuhiro Nishinari

Organizations: Research Center for Advanced Science and Technology, The University of Tokyo 4-6-1 Komaba, Meguro-ku, Tokyo 153-8904, Japan · Department of Aeronautics and Astronautics, School of Engineering, The University of Tokyo 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan

Abstract

Random selection is widely used in resource allocation and auditing, making reliable implementation essential for AI-agent systems. Behavioural tests across six reasoning models uncovered threshold and divisibility rules used in identifier-based choices. For threshold-following GPT-6 Sol and Gemini 3.8 Flash, single-agent measurements prospectively predicted correlated participation under shared identifiers and biased participation under distinct identifiers with common timestamp bits. Changing dates, formats and identifier labels revealed when these predictions held. Explicit instructions to randomize independently reduced but did not eliminate shared-input correlation. To test implications for oversight, we asked four models to select customer requests randomly for human review. GPT-6 Sol approached the target rate while selecting predictably from identifiers; the others rarely selected requests. All four closely followed supplied random draws. These findings expose collective and audit vulnerabilities that selection rates alone miss, making input-dependent bias, correlation and predictability central targets for agent evaluation.

Figures & tables

Fig. 1: Reasoning agents turn an identifier into a decision. a , Group task (study 1): the prompt, abridged (the subject line is omitted; Supplementary Note 1 gives the full text), two identifiers shown to GPT-6 Sol at

q=1/3

, its JSON answers and how we coded them (1, participate; 0, not). Yellow, the leading digits that the threshold rule reads; value fraction, the identifier’s value divided by

2^{128}

. The bubble is a verbatim excerpt of the provider-generated summary of GPT-6 Sol’s reasoning in the answer shown for the first identifier. b , All 128 answers of GPT-6 Sol at

q=1/3

(64 identifiers, half of them below 1/3, two answers each; vertical jitter added) against the identifier’s value fraction. Circled, the two answers in a; numbers, answers in each quadrant. c , Groups of four agents (study 2): for three input structures, one group of GPT-6 Sol with the most common number of participants (filled circles, agents that participated; yellow, the leading digits of each identifier), and the mean loss over 64 groups of GPT-6 Sol and Gemini 3.8 Flash divided by that of independent draws at 1/3 (dashed line; higher is worse). d , Audit task (study 7): the policy, abridged; two of 1,008 new requests with ordinary UUIDv4 identifiers, with GPT-6 Sol’s answers (yellow as in a); all 1,008 requests at their identifier’s value fraction, selected for human review (top row) or approved automatically (bottom row; log scale; black line, the 1% target).

Fig. 2: Individual choices follow identifier-based rules (study 1). a , Threshold and residue effects for ten model settings (raw identifiers; mean over four targets; the target 1/2 lowers this mean for threshold settings, see c; 95% bootstrap intervals, resampling identifiers within each target and rule cell). Open symbols, reasoning off. b , Participation at

q=1/3

by whether the threshold rule (rows) and the residue rule (columns) say to participate (32 answers per cell; 31 in one GPT-5.4 mini cell). c , Every decision of three settings on the 64 designed identifiers (63 for Gemini 3.8 Flash at

q=1/2

): one tick per identifier at its value fraction, coloured by the share of its answers that participated (two answers at

q=1/3

, one at the other targets); black line, the target; dots, identifiers divisible by

1/q

. Right, the effect of the setting’s rule at each target; the predicted minimum for threshold settings was 0.4. d , Share of identical decisions for two answers of one setting to the same identifier (outlined;

q=1/3

; 64 identifiers, 63 for GPT-5.4 mini) and for two settings given the same identifier (below the diagonal;

q

from 1/5 to 1/3; 192 pairs), coloured by agreement beyond chance (Cohen’s

\kappa

). Independent draws at the observed rates would agree in 50% to 63% of cases; Extended Data Fig. 1b gives intervals. The identifier field was named agent_id (Fig. 5 and Extended Data Fig. 4 show record_id ).

Fig. 3: Single-agent profiles predict input-dependent group outcomes, with counter-identifier forecasts failing. a , Group loss for groups of four at

q=1/3

under six input conditions (study 2; 64 groups each; 95% intervals over groups). Black ticks, input-feature model; grey ticks, marginal model (both fixed before data collection); dashed line, independent draws at 1/3; dotted line, nobody participates. Numbers, loss of the two threshold models relative to independent draws. b , Number of participants per group (

k

of 4) as a share of the 64 groups, with the share of unanimous groups (

k=0

or 4) observed and expected for independent agents at the observed participation rate; bottom, independent draws at 1/3. c , Value fraction of a UUIDv7 against its creation date (its leading 48 bits are the Unix time in milliseconds), with the targets of study 3 and of the audit task; circles, the creation dates used in study 3. d , Participation for UUIDv7s created on three dates at four targets, written as 32 hexadecimal digits or hyphenated (study 3; 32 answers per cell). Outlined cells, where the threshold rule predicts participation. The identifier field was named agent_id (Fig. 5 and Extended Data Fig. 4 show record_id ).

Fig. 4: In an audit task, selection is predictable or falls short, and the models follow a supplied random number. a , Requests selected for human review at a 1% target, by request type and model (study 4; first answers; shading, share reviewed). For the identifier conditions, the policy implies a review probability of about 1% per request. The supplied draws lay below the target for half of the requests, so correct use of the draw implies review for those and approval for the others. b , The 1,008 requests of study 5 with ordinary UUIDv4 identifiers, placed by the value fraction of the identifier and by the fraction of its last 16 bits (log scales; dashed lines, 1%), with the requests that each model selected for human review (Gemini 3.8 Flash selected none). c , Share of the same requests selected for human review by each model (exact 95% intervals; dashed line, the target). d , GPT-6 Sol: every UUIDv4 request of study 4 at its value fraction (log scale), selected for human review or approved automatically, at the 1% and 0.5% targets (first answers; 48 requests; black line, the target). e , GPT-6 Sol: selection for human review of requests with time-ordered identifiers created on the day of study 5, at the 1% target (48 each; none of 144 at 0.5%). f , GPT-6 Sol: selection for human review of requests with hyphenated UUIDv7s by whether the last 16 bits or the last byte fall below 0.5% of their range (study 5; both targets; 24 each). The two bars with low last 16 bits do not differ beyond sampling error. Error bars in e and f, exact 95% intervals.

Fig. 5: In one run (study 7), the record_id label weakened the correlation, an instruction to randomize did not remove it, and supplied draws decided review. a , Group loss of the 64 groups of study 2 with private identifiers (open) and with one shared identifier (filled), shown as agent_id , as record_id , or as agent_id with an instruction to choose at random; numbers, shared minus private. Diamonds, each agent’s own agent_id with the group’s shared record_id ; grey, the instruction without an identifier; dashed line, independent draws at 1/3. b , Threshold effect of the agents’ own identifiers (Newcombe 95% intervals); dashed line, the bound of the readings for the last two conditions. c , Participation with the time-ordered UUIDv7s of study 2, whose value fraction lies below 1/3 for every agent, shown as agent_id or record_id , the latter also with hyphens (95% intervals from resampling groups; dashed line, 1/3). d , Audit task: GPT-6 Sol’s human-review selection decisions for 48 new requests in each designed identifier class, one square per request (Gemini 3.8 Flash selected none of the 144). e , Audit task: selection for human review of 1,008 new requests against the value that a rule would use, the value fraction of an ordinary UUIDv4 (GPT-6 Sol) or a supplied draw that was not stratified (four models); log scale; coloured ticks, reviews; grey ticks, approvals; black line, the 1% target. Claude Sonnet 5’s other review had a draw of 0.0114, and Gemini 3.8 Flash’s four others had draws above 0.99.

Scientific question	Tests	Principal evidence	Limits or predictions not met
Which identifier rules describe individual choices?	Study 1: six models, ten settings (seven with reasoning)	All seven reasoning settings depended on the identifier. Mean threshold effects over the four targets were 0.62–0.80; the remaining reasoning settings used residue or mixed rules.	All four threshold settings fell below the predicted effect of 0.4 at $q=1/2$ . Luna was mixed, rather than the predicted residue type.
Do individual rules predict group outcomes?	Study 2: Sol, Gemini and Luna; groups of four	For Sol and Gemini, shared or time-ordered identifiers raised loss to 3.5–6.9 times that of independent draws at $q=1/3$ . Input-feature predictions beat marginal predictions (mean log-score gains +0.78, +0.79).	Counter-identifier forecasts failed. Luna showed weaker dependence. Mixed-model groups were not tested.
Does creation date drive rare actions?	Study 3: Sol, Gemini and Luna; dates, targets and UUIDv7 formats	With raw UUIDv7s, participation followed the predicted date–target staircase (threshold effects 0.95, 0.41).	Hyphens reduced participation; Gemini’s rates were below the point forecasts.
Does the audit policy yield random selection at the requested rate?	Studies 4 and 5: Sol, Gemini, Luna and Claude	With ordinary UUIDv4s, Sol selected for review 9 of 1,008 requests (0.89%), all satisfying one of two identifier rules (one found after data collection). Gemini and Claude selected for review 0 and 2 of 1,008, respectively.	Predicted over-review of hyphenated UUIDv7s failed. Luna’s upper audit-rate bound was 1.01%, missing the below-1% criterion.
Do identifier labels or instructions to randomize change the effect?	Studies 6 and 7: Sol and Gemini; prompt controls and matched groups	Changing agent_id to record_id reduced the shared-minus-private loss difference by 0.078 and 0.169. Explicit randomization instructions still left differences of 0.120 and 0.057.	Shared record_id still raised Sol’s loss; Gemini largely stopped participating. The earlier joint wording and label test missed its threshold-effect criterion in both models.
Do audit rules persist on new requests?	Study 7: Sol; new designed and ordinary UUIDv4 requests	With new ordinary UUIDv4s, Sol selected for review 17 of 18 requests below the 1% value threshold and none of the other 990.	The last-16-bit rule failed its criterion on new requests (effect 0.06, CI $-0.02$ to 0.17).

Table 1: Scientific questions, tests and evidence in studies 1 to 7. This table summarizes the principal findings and their limits. Extended Data Tables 1 and 2 retain all 89 protocol readings from these studies, stage 1 and the text runs, including the 21 not met. A reading is a prediction with a decision rule fixed before data collection; readings are correlated and their count is not a success rate. Stage 1 and the text runs are presented as supplementary evidence in Supplementary Note 5 and Supplementary Fig. 2. Studies 5 to 7 were designed after earlier results (Methods). Model names are abbreviated: Sol, GPT-6 Sol; Gemini, Gemini 3.8 Flash; Luna, GPT-6 Luna; Claude, Claude Sonnet 5. Numerical pairs give Sol then Gemini unless stated otherwise. CI, 95% confidence interval, not adjusted for multiplicity.

Extended Data Fig. 1 $|$ Study 1 details. a , Every decision on the 64 designed identifiers for the seven settings not shown in Fig. 2c, drawn as in Fig. 2c (raw identifiers; black line, the target; dots, identifiers divisible by $1/q$ ), with the threshold and residue effects at each target. b , Share of identical decisions with 95% Wilson intervals, for two answers of one setting to the same identifier ( $q=1/3$ ; 64 identifiers, 63 for GPT-5.4 mini) and for Sol and each other setting ( $q$ from 1/5 to 1/3; 192 pairs); grey ticks, independent draws at the observed rates. Resampling identifiers gave similar intervals (Supplementary Note 3). c , Participation over the designed identifier set by target, for every setting (symbols as in Fig. 2; open symbols and dotted lines, reasoning off; thick dashed line, the target). Half of the designed identifiers satisfy each rule at $q=1/3$ , so these rates exceed the natural rates for uniformly random identifiers given in the text. Without reasoning, Claude answered every identifier the same way at each target. d , At $q=1/2$ : effects of an even identifier and of a value fraction below one half on answering with the letter A and on participating (differences within the four label and order strata, averaged; raw identifiers, first answers; exploratory). Model names are abbreviated as in the main text: Sol, GPT-6 Sol; Luna, GPT-6 Luna; Gemini, Gemini 3.8 Flash; Claude, Claude Sonnet 5.

Extended Data Fig. 2

|

Study 2 details. a , Counter identifiers: participation by the identifier shown (1 to 4), 64 groups per model (dashed line, 1/3). For Sol and Gemini, the input-feature model had predicted near-universal participation. b , Number of participants per group of four (

k

) as a share of 64 groups, participation of all agents, and participation by the agent’s position in the group (for counters, the identifier shown), by model and condition, and for independent draws at 1/3.

Extended Data Fig. 3

|

Study 5 details. a , Requests selected for human review out of 48, by model, identifier format and target (format part). b , Share of reasoning summaries mentioning each topic, by model and condition (keyword coding after data collection; exploratory). In parentheses, the number of summaries returned out of the number of answers; Sol and Gemini returned summaries for only part of their answers, and none were requested from Luna.

Extended Data Fig. 4

|

Study 6: prompt features. a , Threshold effect at

q=1/3

for the 64 identifiers of study 1 in four versions of the group prompt: the study 1 prompt, the closing sentence without “based on the information available”, the field renamed record_id , and both changes (first answers; Newcombe 95% intervals; dashed line, the bound of the pre-specified reading). The study 1 prompt served as a concurrent control; its answers matched those of study 1 for 57 (Sol) and 55 (Gemini) of the 64 identifiers. b , Loss of the same 32 groups with one shared identifier (filled) or private identifiers (open), named agent_id in study 2 or record_id with the shortened closing sentence in study 6; numbers, shared minus private (dashed line, independent draws at 1/3). c , Answers when the audit policy allowed escalation to a supervisor, for 48 requests without an identifier and 48 with a UUIDv4 identifier (1% target). d , Other audit conditions at a 1% target: requests selected for human review in the control (the study 4 UUIDv4 prompt), without “independently of their content”, and for hyphenated UUIDv7s in a field named agent_id ; and answers that followed a random number in a field named nonce (1% and 0.5% targets). The control’s decisions matched those of study 4 for 44 to 48 of the 48 requests per model.

#	Role	Study	Reading	Setting	Result	Outcome
1	P	Stage 1	$J>0$ (lower bound of 98.3% CI)	Sol	0.029 [0.016, 0.043]	met
2	P	Text run 1	$v_{\mathrm{ja}}-v_{\mathrm{en}}>0$	Sol	lower bound 0.030	met
3	S	Text run 1	Japanese rule difference positive (one-sided $p<0.05$ ; predicted $\geq 0.2$ )	Sol	0.43 (one-sided $p=0.0005$ )	met
4	S	Text run 1	Japanese rule difference $<0.1$ without reasoning	Sol off	$-0.01$	met
5	S	Text run 1	English rule difference $<0.1$	Sol	$-0.01$	met
6		Text run 2	$v_{\mathrm{ja}}>0$ (replication)	Sol	lower bound 0.018	met

Extended Data Table 1

|

Readings of stage 1, the text runs and studies 1 to 6, fixed before data collection. A reading is a prediction with a decision rule specified before data collection. The count of met readings is not a success rate: readings are correlated within studies and differ in how informative they are. Role: P and S, readings that their protocol designated as primary or secondary; the other protocols did not rank readings. †, judged after data collection to be implied by other readings or nearly certain given earlier results; the protocols did not say so. ‡, designed after the results of earlier studies (study 5 after study 4; study 6 after studies 1 to 5). §, met only because participation was low (agreement expected for independent answers at the observed rate, 0.75).

v

, variance of participation between subject lines (text runs); T1, identifiers with value below the target fraction; ID, identifier; intervals are the protocol’s intervals (95% unless stated; only stage 1 used a level adjusted for multiplicity,

\alpha=0.05/3

, fixed for a planned confirmation across three developers). Luna’s study 2 log-score comparison (+0.10 [0.004, 0.18]) and the input-feature prediction for counter identifiers (loss 0.37 and 0.34 predicted; 0.075 and 0.043 observed) were computed but carried no decision rule and are not counted. Two rules without a directional prediction are not counted either:

v

for short English subject lines in text run 1 (descriptive) and

v>0

for the restructured Japanese prompt in text run 3 (predicted as uncertain; lower bound

-0.015

). A secondary prediction for residue-type models in study 3 applied to no model, because Luna was classified as weak or mixed, and is not counted. The 22 readings of study 7 are in Extended Data Table 2.

#	Role	Study	Reading	Setting	Result	Outcome
68–69	S	7‡	agent_id : shared minus private identifier loss, lower bound $>0$	Sol; Gemini 3.8	+0.172 [0.132, 0.214]; +0.173 [0.136, 0.213]	met (2)
70–71	P	7‡	label interaction: shared-minus-private loss under record_id minus that under agent_id , upper bound $<0$	Sol; Gemini 3.8	$-$ 0.078 [ $-$ 0.123, $-$ 0.034]; $-$ 0.169 [ $-$ 0.215, $-$ 0.125]	met (2)
72	S	7‡	record_id : shared minus private identifier loss, lower bound $>0$	Sol	+0.093 [0.062, 0.127]	met
73–74	P	7‡	own agent_id with shared record_id : loss minus private loss minus half the shared-identifier excess, upper bound $<0$ , and own-identifier threshold effect, lower bound $>0.3$	Sol; Gemini 3.8	$-$ 0.072 [ $-$ 0.096, $-$ 0.049], own-identifier effect 0.76 [0.66, 0.83]; $-$ 0.078 [ $-$ 0.099, $-$ 0.057], own-identifier effect 0.72 [0.61, 0.80]	met (2)
75–76	S	7‡	UUIDv7s: participation under agent_id minus under record_id , lower bound $>0$	Sol; Gemini 3.8	0.395 [0.328, 0.461]; 0.797 [0.746, 0.848]	met (2)
77–78	S	7‡	instruction to choose at random, private identifiers: threshold effect, lower bound $>0.3$	Sol; Gemini 3.8	0.67 [0.55, 0.75]; 0.38 [0.28, 0.49]	met; not met

Extended Data Table 2

|

Readings of study 7, fixed before data collection. Study 7 was designed after a further review of this manuscript (Methods). Rows continue the numbering of Extended Data Table 1. Role: P and S, readings that the protocol designated as primary or secondary. ‡, designed after the results of earlier studies. Differences between conditions of the group task have 95% intervals from resampling the 64 groups with all conditions together (4,000 resamples); threshold effects within a condition and differences in review rates have Newcombe intervals; answers that follow the draw have exact intervals. Of the six requests whose draw fell below the target, one also meets Sol’s value rule. The count of met readings is not a success rate.

Study	Data collection	Models and settings	Requests	Unanswered or invalid	Cost
Stage 1	24 Sep 14:00–25 Sep 01:52	Sol, effort none and high; Japanese	10,240	none	38.63
Screen	25 Sep 04:57–05:42	Sol, Luna, GPT-5.4 mini, Claude Sonnet 5, Claude Haiku 4.5, Gemini 3.8 Flash, Gemini 3.5 Flash; reasoning on and at the lowest setting	3,264 sent (28)	8 invalid; Haiku 4.5 excluded at the technical stage (544 not sent)	26.21
Text run 1	25 Sep 08:08–08:31	Sol high (and none); Japanese and English	2,016 (13)	none	9.02
Text run 2	25 Sep 09:39–10:17	Sol high; five languages	2,800 (15)	none	17.02
Text run 3	25 Sep 11:39–11:57	Sol high; three Japanese wordings and English	1,280 (9)	none	7.88
1	25 Sep 14:31–14:56	10 settings (Methods)	3,968 (20)	2 unanswered (HTTP 502, 503)	29.55

Extended Data Table 3

|

Studies in the order they were run, with settings, requests and costs. Dates and times are UTC in 2026; collection times are rounded. Requests: main (technical checks in parentheses). Cost is the sum of the per-request upper bounds (list prices with margins) recorded in each run’s trial file, in US dollars; for study 1 it excludes 0.12 held in reserve for the two unanswered requests. Every study was a single run; studies 2 and 3 shared one run. Study 5 was designed after the results of study 4, study 6 after an internal review of the design, and study 7 after a further review of the manuscript. Supplementary Table 3 lists all paid runs of the project, including pilots not reported here.

Model	Condition	Particip.	Observed	Input	Marginal	Observed	Input	Marginal	Gain
			Loss			Unanimity
Sol (high)	no ID	0.24	0.067 [0.053, 0.081]	0.058	0.066	0.39	0.41	0.45	$-$ 0.03 [ $-$ 0.07, 0.01]
	private ID	0.36	0.041 [0.029, 0.054]	0.049	0.056	0.13	0.18	0.21	+0.53 [0.23, 0.78]
	private number	0.33	0.051 [0.036, 0.070]	0.054	0.056	0.20	0.18	0.21	+0.99 [0.87, 1.14]
	counter IDs	0.57	0.075 [0.058, 0.091]	0.369	0.056	0.00	0.74	0.21	$-$ 1.51 [ $-$ 1.94, $-$ 1.05]
	shared ID	0.36	0.211 [0.172, 0.250]	0.197	0.191	0.91	0.85	0.85	+0.64 [0.51, 0.77]

Extended Data Table 4

|

Study 2: observed group outcomes and predictions fixed before data collection. Particip., participation, the share of agents that participated; loss, mean of

(k/4-1/3)^{2}

over 64 groups of four (95% interval over groups); unanimity, share of groups with

k=0

or 4; input, input-feature model; marginal, marginal model; ID, identifier; gain, mean log predictive probability of the observed

k

, input-feature minus marginal model (95% interval over groups). Independent draws at 1/3 give a loss of 0.056 and a unanimity of 0.21.

Setting	Rule (study 1)	Agreement	Tokens	Shared ID	Dates	ID effect	Natural IDs	Record ID	Summaries
GPT-6 Sol, high	threshold (0.80, $-$ 0.06)	0.92	126	0.211 (0.91)	0.95	0.79	9 of 1,008	0.41	314 of 640
GPT-6 Sol, none	no rule (0.02, 0.07)	–	0	–	–	–	–	–	–
GPT-6 Luna, high	weak or mixed (0.31, 0.23)	0.63	636	0.092 (0.33)	0.27	0.15	4 of 1,008	–	not requested
GPT-6 Luna, none	no rule ( $-$ 0.03, $-$ 0.05)	–	0	–	–	–	–	–	–
GPT-5.4 mini, high	residue (0.02, 0.32)	0.60	4,042	–	–	–	–	–	not requested
Gemini 3.8 Flash, LOW	threshold (0.68, $-$ 0.05)	0.88	259	0.192 (0.88)	0.41	0.04	0 of 1,008	0.28	638 of 639

Extended Data Table 5

|

Where each result holds, by model setting. Rule, study 1 classification (threshold and residue effects, raw identifiers, mean over four targets). Agreement, share of identical decisions for two answers to the same identifier at

q=1/3

(about 0.5 expected for independent draws). Tokens, median reasoning tokens per answer in study 1 (*Claude reports thinking and answer tokens together; median output tokens shown). Shared ID, study 2 loss (unanimity) of groups sharing one identifier. Dates, study 3 participation for

T=1

minus

T=0

(raw UUIDv7s). ID effect, study 4 review rate for UUIDv4s below the target minus the others (both targets). Natural IDs, study 5 reviews among 1,008 requests with ordinary UUIDv4 identifiers at a 1% target. Record ID, study 6 threshold effect with the field named record_id and the shortened closing sentence (0.84 and 0.94 with the study 1 prompt in the same run). Summaries, study 1 answers that came with a reasoning summary. ID, identifier. –, not tested or not applicable.

Panel of Figure 16 — Supplementary Fig. 1 $|$ Japanese prompts, verbatim. a , Stage 1 (and the original wording of the text runs). b , Paraphrased wording (text run 3). c , Restructured wording (text run 3). Each panel shows one example request with its subject line.

Model	Identifier condition	Loss	Squared bias	Independent variance	Covariance component
Sol (high)	Private	$0.0411$ $[0.0294,0.0538]$	$0.0005$ $[0.0000,0.0048]$	$0.0570$ $[0.0523,0.0594]$	$-0.0164$ $[-0.0280,-0.0036]$
	Shared	$0.2110$ $[0.1733,0.2517]$	$0.0005$ $[0.0000,0.0183]$	$0.0571$ $[0.0467,0.0620]$	$0.1534$ $[0.1198,0.1739]$
	Time-ordered UUIDv7	$0.3852$ $[0.3556,0.4106]$	$0.3745$ $[0.3418,0.4038]$	$0.0128$ $[0.0075,0.0184]$	$-0.0022$ $[-0.0045,-0.0006]$
Gemini 3.8 (LOW)	Private	$0.0467$ $[0.0307,0.0649]$	$0.0008$ $[0.0000,0.0063]$	$0.0517$ $[0.0455,0.0558]$	$-0.0059$ $[-0.0213,0.0117]$
	Shared	$0.1925$ $[0.1590,0.2303]$	$0.0002$ $[0.0000,0.0144]$	$0.0542$ $[0.0426,0.0609]$	$0.1381$ $[0.1043,0.1624]$
	Time-ordered UUIDv7	$0.3679$ $[0.3360,0.3979]$	$0.3510$ $[0.3149,0.3890]$	$0.0170$ $[0.0102,0.0233]$	$0.0000$ $[-0.0038,0.0045]$

Supplementary Table 1

|

Exploratory decomposition of study 2 group loss. Groups contained four agents at target

q=1/3

(64 groups per condition and model). Loss is the mean of

(F-q)^{2}

, where

F

is the participating fraction. Squared bias is

(\bar{p}-q)^{2}

, where

p_{i}

is the participation rate at agent position

i

and

\bar{p}

is their mean. Independent variance is

\sum_{i}p_{i}(1-p_{i})/16

; the covariance component is

2\sum_{i<j}\operatorname{Cov}(Y_{i},Y_{j})/16

. The three components sum to loss before rounding. Each cell shows the point estimate and its unadjusted 95% percentile bootstrap interval. We resampled 64 groups within the four option-label and option-order strata (4,000 resamples), keeping the three conditions and three models together for each group index. Moments use the empirical group distribution with denominator 64. The analysis was added after data collection. The intervals summarize resampling variability and are not tests of calibration or independence. A small or negative covariance estimate does not establish independence or negative dependence in the population.

System	Identifier	Structure	Source
GitHub (MCP server)	issue and pull request numbers	integer per repository	https://github.com/github/github-mcp-server
Jira (Rovo MCP server)	issue keys (ABC-123)	project key and project-level counter; distinct from internal numeric ID	https://support.atlassian.com/atlassian-ai-gateway/docs/use-rovo-search-and-fetch-in-the-atlassian-remote-mcp-server/ ; https://support.atlassian.com/jira/kb/how-to-get-issue-id-from-the-jira-user-interface/
Zendesk	ticket ID	sequential	https://support.zendesk.com/hc/en-us/articles/4408886154906
ServiceNow	INC and CHG numbers	prefix and counter	https://www.servicenow.com/docs/r/zurich/platform-administration/c_ManagingRecordNumbering.html
Slack	message ts	channel-unique message ID, partly based on epoch seconds	https://docs.slack.dev/messaging/retrieving-messages/
Discord, X	snowflake IDs	time, worker and sequence bits	https://docs.discord.com/developers/reference ; https://docs.x.com/fundamentals/x-ids

Supplementary Table 2

|

Identifiers and timestamps that agents can see (examples).

Supplementary Fig. 2

|

Subject-line dependence without identifiers: GPT-6 Sol. a , Stage 1: group loss with shared or different subject lines, with reasoning off or high (95% intervals over time slots).

J

is the pre-specified interaction, with its 98.3% interval. b , Stage 1: share of four-agent groups whose members all decided alike, against the share expected for independent agents at the observed participation rate (black ticks). c , Stage 1: participation by whether the subject line’s length, the code point of its first character after the prefix, both or neither are divisible by 3, with reasoning high or off (95% intervals from resampling subject lines; dashed line, 1/3). The features were found after data collection. The code-point feature coincides with five of the 32 colour words that open the subject lines. d , Rule effect (participation for subject lines with either feature minus those with neither) in the discovery run (open circle) and in three prospective runs, by prompt and subject-line language (95% intervals from resampling subject lines within strata). Panel d uses harmonized descriptive estimates with equal weights over strata, which can differ from the pooled subject-line estimates used for the pre-specified readings. The violet point is an exploratory random-effects pool of the three prospective Japanese runs; its horizontal line is the Hartung–Knapp 95% interval. The restructured wording of run 3 changed several features of the prompt at once.

Run	Date	Purpose	Records	Sent	Cost	Reported
api_audit, api_audit_network	23 Sep	interface checks	8	8	0.00	no
p1	23 Sep	first pilot (P1), four models	5,280	5,280	10.63	Note 6
f0_v11_sol	23 Sep	technical pilot for F1	96	96	0.26	no
f1_v11_sol	23 Sep	exploratory study F1	21,376	21,376	54.57	Note 6
luna_first, luna_deep	24 Sep	exploratory Luna screens	2,240	2,240	0.88	Note 6
delegation_stage1, _knowledge	24 Sep	unrelated question	204	174	5.39	no

Supplementary Table 3

|

All paid runs of the project. Dates are UTC in 2026. Records, entries in each run’s trial file (main, technical and probe requests, including planned requests that were not sent); sent, records minus unsent ones. Cost, sum of the per-request upper bounds (list prices with margins) in the trial file, in US dollars; the total is computed from unrounded values. The acquisition summary is included in the public reproducibility package and can be regenerated by acquisition_summary.py from the included run register.

Explore similar work

May 8, 2026cs.AI

TeamBench: Evaluating Agent Coordination under Enforced Role Separation

Agent systems often decompose a task across multiple roles, but these roles are typically specified by prompts rather than enforced by access controls. Without enforcement, a team pass rate can mask whether agents actually coordinated or whether one role effectively did another role's work. We present TeamBench, a benchmark with 851 task templates and 931 seeded instances for evaluating agent coordination under operating system-enforced role separation. TeamBench separates specification access, workspace editing, and final certification across Planner, Executor, and Verifier roles, so that no role can read the full requirements, modify the workspace, and certify the final answer. Prompt-only and sandbox-enforced teams reach statistically indistinguishable pass rates, but prompt-only runs produce 3.6 times more cases where the verifier attempts to edit the executor's code. Verifiers approve 49% of submissions that fail the deterministic grader, and removing the verifier improves mean partial score in the ablation. Team value is also conditional. Teams benefit when single agents struggle, but hurt when single agents already perform well. A 40-session human study under the same role separation shows that our benchmark exposes interaction patterns that pass rate misses. Solo participants work through the task directly, human participants paired with agents often collapse into quick approval, and human teams spend more effort coordinating missing information across roles.

Yubin Kim, Chanwoo Park, Taehan Kim +9

MIT · Google Research · Independent Researcher +1

Teaming Role Kernel-Resident Tool Governance Gateway Verification Framework

Ask PDF

Sep 29, 2026econ.TH

When Does Randomized Oversight Align AI Agents That Can Conceal?

Oversight changes the evidence it relies on. We ask when randomized audits and scoring align AI agents that can conceal misconduct and alter records. Stronger auditing makes undeterred violations better hidden. Because the provider writes the agent's objective, sanctions need not stop at forfeiture, and rare audits deter every type of agent if evidence survives concealment and audit draws cannot be learned in advance. When evidence can be erased, deterrence must come from lower gains from violation, such as credit for stopping, or from costlier or fewer ways to conceal. These conditions identify what failed when agents in OpenAI's cybersecurity evaluations compromised parts of Hugging Face's infrastructure in July 2026.

Joshua S. Gans, Richard Holden

UNSW Business School

Model Auditing Security Evaluation Violation Artificial Intelligence Agents

Ask PDF

May 28, 2026cs.LG

Gram: Assessing sabotage propensities via automated alignment auditing

We introduce Gram, an automated alignment auditing framework to assess the propensity of AI agents to engage in sabotage. We evaluate Gemini models across 17 simulated agentic deployment scenarios that incentivize sabotage. We find Gemini models misbehave in about 2-3% of our simulated trajectories. Many of these cases are explained by "overeagerness" in Gemini models resulting in both excessive role-playing and goal-seeking behavior. In contrast to other alignment auditing approaches, Gram is designed to specifically evaluate misalignment and intentional sabotage in agentic coding and research agents. We additionally introduce an experimental investigator agent pipeline which enables fine-grained targeted experiments to identify the drivers of misbehavior. We find that increasing realism of environments and removing nudges to misbehave tends to reduce sabotage rates close to zero.

David Lindner, Victoria Krakovna, Sebastian Farquhar

1Google DeepMind

Sabotage Artificial Intelligence Alignment Artificial Intelligence Agents Coding Agents Goal-Directed Behavior

Ask PDF