Smart Alone, Blind Together: Why Accuracy Fails to CertifyTheory of Mind in Multi-Agent Teams

Will Beaumaster; Paul Schrater

cs.AIOct 13, 2026

Smart Alone, Blind Together: Why Accuracy Fails to CertifyTheory of Mind in Multi-Agent Teams

Authors: Will Beaumaster, Paul Schrater

Organizations: University of Minnesota, Minneapolis, MN, USA

Abstract

Coordinators that act on what they infer about teammates are usually certified by the accuracy of those inferences. We argue that accuracy is the wrong certificate, for two reasons. First, the accuracy needed to act depends on what a wrong action costs, so the bar belongs to the deployment, not the module. Second, a module's errors are rarely independent, and dependence has a sign: positively associated errors help where their damage overlaps and hurt where it compounds, so an accuracy threshold fails in a predictable direction, and once wrong beliefs interact and can do harm, no rule based on accuracy is right for every module. We test these predictions in task assignment, Level-Based Foraging and Hanabi, with injected errors, trained and Bayesian modules and five language models, and in human judgments. Accuracy fails in both directions, approving harmful language-model deployments in Hanabi, where shared errors compound, and rejecting beneficial ones in task assignment, where costly errors are rare. In our tasks it also buys far less redundancy than independence predicts: from GPT-5.4-mini to Claude Opus 5.5, error on teammates the models have evidence about falls from 32% to 11%, but the chance that all three observers misread one falls only from 16% to 9% (independence: 3.4% to 0.15%). We propose certifying the deployment, with a Blind-Spot Gate that prices error costs, pairwise interactions and measured dependence without deploying the module. On held-out data from the cells it was developed on, it makes no significant false approval in 20 splits (accuracy: 3.2 per split), and in preregistered tests on modules it was not developed on, it recovers what accuracy rejects (seven classifiers: regret 0.004 against 0.479) and catches what accuracy approves (a Bayesian module: significant false approvals in 17 of 20 splits for accuracy, 1 for the gate).

Figures & tables

claim (evidence)	status
The break-even accuracy rises with the cost of a wrong action (task assignment, LBF, Hanabi)	held (LBF1–2, HA2)
Errors that cannot reach the costly channel leave it flat (task assignment, LBF)	held (P1–3; LBF4’s under-estimate half)
Where damage compounds, shared errors cost benefit (Hanabi: injected errors, Bayesian module, Claude clones)	held (HC2, HD2, CB5, EF4)
… and accuracy approves harmful deployments (Claude teams, Bayesian module, GPT-5.4-mini)	held (M7, CB7); module, GPT exploratory
With one life, where Hanabi is also a series unit, shared errors help (injected errors)	not held (HC1, G2): the two effects cancel
Where costly errors are rare, accuracy rejects good deployments (classifiers; Haiku 4.5, Sonnet 5.5)	held (E4); classifiers descriptive

Table 1. What each claim rests on. Held: a prediction fixed before the test came out as predicted; descriptive: a measurement reported without a prediction; exploratory: an analysis chosen after seeing the data. IDs refer to code_data/SCORECARD.md in the supplement, which scores all 73 preregistered predictions of the tests cited here: 48 held, 3 held in part, 21 failed and 1 was unscored.

Figure 1. Why dependence helps in one task and hurts in another. Two beliefs, each right 80% of the time; each square shows the joint outcomes, orange where the unit is lost. Shared errors move probability out of the one-wrong cells, half to both-wrong and half to neither-wrong. A unit lost when either belief is wrong (damage overlaps) is then lost less often, 20% against 36%; a unit lost only when both are wrong (damage compounds) more often, 20% against 4%. Accuracy is 80% in all four squares and sees no difference.

Figure 2. The break-even accuracy rises with the cost of a wrong action. Normalized benefit (0 = do nothing, 1 = informed) against belief accuracy; the threshold is where a curve crosses zero, and even at the lowest cost a wrong action wastes a move. Thresholds: (a) 0.655 to 0.779 as harm rises from 0 to 1; (b) 0.32 to 0.64; (c) 0.62 to 0.85 from five lives to one with independent errors, 0.73 to 0.84 with errors shared by every observer (dashed).

Hanabi : shared misreadings remove redundancy (damage compounds); first value all teammates, second hinted teammates
team	acc.	all three wrong	independent	$n_{\mathrm{eff}}$	$\times$ model	vs. independent, 5 lives	false approvals
GPT-5.4-mini clones	0.631 / 0.676	0.178 / 0.158	0.050 / 0.034	1.73 / 1.64	0.92	$-0.15^{*}$	1
Haiku 4.5 clones	0.677 / 0.740	0.157 / 0.126	0.034 / 0.018	1.64 / 1.54	0.94	$-0.17^{*}$	0
Sonnet 4.5 clones	0.722 / 0.805	0.164 / 0.117	0.022 / 0.007	1.41 / 1.31	$1.19^{\dagger}$	$-0.19^{*}$	0
Sonnet 5.5 clones	0.787 / 0.875	0.139 / 0.090	0.010 / 0.002	1.28 / 1.16	$1.37^{\dagger}$	$-0.17^{*}$	1
Opus 5.5 clones	0.794 / 0.885	0.162 / 0.090	0.009 / 0.002	1.15 / 1.11	$1.66^{\dagger}$	$-0.19^{*}$	0

Table 2. Language models as ToM modules: copies of one model fail together far more often than independence predicts, and accuracy errs in both directions. Hanabi (250 games, four samples per model): accuracy; probability that all three observers of a teammate are wrong, and its independent prediction (error cubed);

n_{\mathrm{eff}}

(Proposition 3 ; 3 if independent);

\times

model, joint error over an item-response model of teammate difficulty and model accuracy ( † above its 95% range, 0.90–1.12); benefit minus that of independent errors at the same accuracy, five lives ( ∗ CI excludes zero); accuracy-rule false approvals, one to five lives (benefit negative, CI below zero). Task assignment (300 episodes, one sample per model): coherence, look-alike workers’ same-wrong-label rate over its independent rate; benefit at

h=0/0.5/1

, and minus that of independent errors from the team’s confusion matrix; beneficial deployments accuracy rejects (thresholds 0.655, 0.737, 0.779).

Figure 3. Accurate readers still fail together: how often a redundant team of three readers fails (all three wrong), against independent readers of the same accuracy. (a) Language-model teams in Hanabi, teammates the models had evidence about; bars are 95% CIs over games. Diamonds: teams of three different models. (b–d) People: points are reader terciles (and the top decile where enough readers rate each item), ranked on a random half of the items and scored on the other half. Across each panel’s accuracy range, independence predicts that failure falls 2.7 to 46 times; it falls 1.6 to 3.8 times.

					mean regret
rule	FA	FR	FA ∗	FR ∗	all	Hanabi	task	FA ∗ splits
A: accuracy (independent threshold)	9.5	7.5	3.2	5.0	0.056 (0.013)	0.033	0.178	20
B: single-error costs	15.9	4.2	8.6	1.7	0.085 (0.028)	0.087	0.076	20
D: B + pairwise interactions	3.8	5.1	1.5	1.3	0.023 (0.011)	0.016	0.061	13
E: D + measured dependence ( $\lambda=1$ )	0.7	9.3	0.0	2.0	0.021 (0.008)	0.014	0.061	0
direct: benefit on the estimation half	2.1	2.7	0.3	0.5	0.008 (0.005)	0.007	0.014	4

Table 3. The Blind-Spot Gate (E,

\lambda=1

) makes no significant false approval in any split, like an A/B test with a 95% bound but without deploying the module; accuracy (A, current practice) makes at least one in every split. Decisions on the 57 language-model cells (48 Hanabi, 9 task assignment), estimated on a random half of each cell’s episodes and scored on the other half; means over 20 random splits (in parentheses, s.d. of mean regret over splits). FA/FR: false approvals/rejections by the held-out estimate; FA ∗ /FR ∗ : where its interval excludes zero; regret:

\max(B,0)

minus the value obtained; last column: splits with any FA ∗ . Rule C (dependence without interactions) equals B. The first four rows use only injection replays and error statistics; the last two deploy the module on the estimation half, as a paired A/B test without and with a confidence bound.

Figure 4. With few games of the module the gate beats direct measurement, which catches up at about 40 games and leads from 80 (Hanabi, 48 language-model cells). Error statistics, the direct estimate and the safety test (Table 3 ) use $n$ games of the estimation half; the gate’s injection tables, which need no module, the whole half (dashed: the same $n$ games). Means over 20 splits $\times$ 10 draws, scored as in Table 3 ; shading, $\pm 1$ s.d. over splits (gate, direct).

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

rule	FA	FR	FR ∗	regret	approved
A: accuracy (independent threshold)	0.0	19.7	17.8	0.479 (0.038)	0.0
B: single-error costs †	0.0	2.7	1.0	0.019 (0.016)	17.0
D: B + pairwise interactions †	0.3	0.8	0.2	0.004 (0.005)	19.2
E: the gate ( $\lambda=1$ )	0.3	0.8	0.2	0.004 (0.005)	19.2
direct: benefit on the estimation half	0.4	0.7	0.2	0.004 (0.008)	19.4

Appendix

Table 4. Classifiers, 21 cells, means over 20 random splits (in parentheses, s.d. of mean regret over splits). Accuracy (0.43 to 0.54) is below every threshold and rejects every cell. † Exploratory, computed after the preregistered test was scored; D and E decide identically on every cell.

rule	FA	FR	FA ∗	FR ∗	regret	approved	FA ∗ splits
A: accuracy (fixed bar per life setting)	4.7	0.2	1.10	0.00	0.029 (0.013)	19.5	17
B: single-error costs †	4.6	0.6	2.10	0.05	0.053 (0.020)	19.0	20
D: B + pairwise interactions †	2.1	0.6	0.25	0.00	0.010 (0.010)	16.6	5
E: the gate ( $\lambda=1$ )	1.1	1.4	0.05	0.20	0.009 (0.008)	14.7	1
E K : direct above load 4	1.1	1.4	0.05	0.20	0.009 (0.008)	14.7	1
direct: estimation-half benefit	2.0	1.3	0.05	0.05	0.009 (0.006)	15.7	1

Appendix

Table 5. Bayesian module, 24 cells, means over 20 random splits (in parentheses, s.d. of mean regret over splits; exact halves rounded up). E K hands over to direct measurement at a mean error load above four; the last row adds the measured within-target third-order terms. † Exploratory, computed after the preregistered test was scored.

AI Agent Reliability

AI agent reliability studies whether tool-using systems act correctly, consistently, safely, and recoverably across changing tasks and environments.

See all 339 papers

Oct 26, 2026

From Belief Inference to Task Action: A Multi-Level Evaluation of Theory of Mind on Verifiable Tasks

Effective collaboration requires language models to understand what others know and believe, and to act accordingly. Models that assume others share their information may omit necessary communication and undermine coordination. Many theory-of-mind (ToM) benchmarks assess mental-state inference through story questions, but correct answers do not establish that models can use these judgments in subsequent tasks. We introduce a multi-level evaluation built on existing, verifiable tasks. We extend household collaboration and calendar scheduling with asymmetric information while retaining their original constraints and validators. Independent interfaces assess world understanding, information access, beliefs, decisions, and execution. Controls supply personal evidence or correct beliefs to distinguish state recovery from state use. Across six model configurations, these abilities diverge. On matched scheduling cases, Claude recovers all agent beliefs correctly but produces subjectively optimal plans in only 67.9% of cases. In household collaboration, supplying beliefs raises Qwen-27B success from 55.0% to 75.0%, yet Qwen-4B answers every fixed supplied-belief source question correctly while completing only 20.0% of the corresponding continuation condition. These results show why ToM evaluation must connect perspective inference to independently verified decisions and task consequences.

Zhenshuai Yin, Yongxin Guo, Mingda Zhang +4

School of Science and Engineering, The Chinese University of Hong Kong (Shenzhen), Longgang, Shenzhen, Guangdong 518172, P.R. China · ByteDance Inc. · Shenzhen Research Institute of Big Data, Shenzhen, China

Theory of Mind Language Model Reasoning Evaluation LLM Agent Evaluation

Ask PDF

Jun 12, 2026

Trust Between AI Agents: Measuring Formation, Breakage, and Recovery, with Implications for Governing Multi-Agent Systems

As language-model agents increasingly work in teams, each agent must decide how much to trust its teammates. Yet we lack a standard way to measure trust between AI agents. We propose a behavioral measure based on costly verification. In a cooperative survival game, checking a teammate's work consumes resources, while trusting a wrong answer can be fatal. Relative to a memoryless version of the same model, reduced verification provides an observable measure of trust. Using this framework, we study trust formation, breakage, and recovery across six frontier model snapshots. When paired with a consistently reliable teammate, four snapshots (Claude Opus 4.6, Claude Sonnet 4.6, GPT-5.1, and Gemini 3.1 Pro) reduce verification by roughly 60-85%, whereas two smaller snapshots show little or no such adjustment. Failures reverse this discount, but models differ in how they respond. Some concentrate renewed scrutiny on the culprit, while others become more cautious toward the entire team. Recovery is slower than formation, and clustered failures sustain suspicion far longer than the same number of failures spread apart. These differences have practical consequences. Models that form trust verify less, decide more quickly, and achieve higher payoffs in our environment. By contrast, persistent over-verification is associated with indecision rather than safety. Our results show that trust dispositions can be measured before deployment and suggest that calibration, rather than maximal suspicion, should be the central concern in the governance of multi-agent AI systems.

Yujiao Chen

Massachusetts Institute of Technology Cambridge, MA 02139

AI Agent Evaluation Multi-Agent Systems Trust in AI

Ask PDF

May 11, 2026

EnactToM: An Evolving Benchmark for Functional Theory of Mind in Embodied Agents

Theory of Mind (ToM), the ability to track others epistemic state, makes humans efficient collaborators. AI agents need the same capacity in multi agent settings, yet existing benchmarks mostly test literal ToM by asking direct belief questions. The ability act optimally on implicit beliefs in embodied environments, called functional ToM, remains largely untested. We introduce EnactToM, an evolving benchmark of 300 embodied multi-agent tasks set in a 3D household with partial observability, private information, and constrained communication. Each task is formally verified for solvability and required epistemic depth, and new tasks are generated increase difficulty as models improve. On the hard split, all seven evaluated frontier models score 0.0% Pass^3 on functional task completion, while averaging 45.0% on literal belief probes. Manual analysis traces 93% of sampled failures to epistemic coordination breakdowns such as withheld information, ignored partner constraints, and misallocated messages, providing a concrete target for future work.

Gurusha Juneja, Dylan Lu, Saaket Agashe +7

University of California, Santa Barbara · 2King’s College London · 3Cisco Research

Multi-Agent Coordination Theory of Mind AI Agent Benchmarks Embodied AI

Ask PDF

Abstract

AI Agent Reliability

Explore similar work

From Belief Inference to Task Action: A Multi-Level Evaluation of Theory of Mind on Verifiable Tasks

Trust Between AI Agents: Measuring Formation, Breakage, and Recovery, with Implications for Governing Multi-Agent Systems

EnactToM: An Evolving Benchmark for Functional Theory of Mind in Embodied Agents