What and Where in Identifying and Quantifying Conventions in Humans and AI Agents

Makoto Fukushima; Tim Puphal

cs.HCOct 14, 2026

What and Where in Identifying and Quantifying Conventions in Humans and AI Agents

Authors: Makoto Fukushima, Tim Puphal

Organizations: Honda Research Institute Europe

Abstract

Cooperative success rests in part on implicit communication. Partners converge on conventions: regularities that give an action meaning beyond its direct effect. Yet conventions are usually observed only indirectly: a score drop when partners are swapped shows that a convention existed, but not what it was or where it was used. In the card game Hanabi, the exact probability that a play fails can be computed from the hints received and the cards in view. Players who use conventions succeed more often than this prediction allows; the resulting margin is the convention gap. Off-belief learning provides a hierarchy of agents (levels 1-5) whose first level uses no conventions by construction and whose later levels use progressively more. For humans and these agents on one scale, we ask what the conventions are and where they are used. On what: in two independently collected corpora of human games, humans show a larger gap than every agent level. On the conventions both populations use, however, humans do no better than level 2. The human advantage is concentrated in a situation that no agent level learns to use: hints that point to a card other than the ones they touch. On where: conventions depend on what the player already knows about the card, not on how recently it was hinted, and they are used most in the mid-game; humans use them earlier and later in the game than the agents. In interventions on the agents, blocking a single high-gap situation costs little, because meaning reroutes to other situations, but blocking several together costs far more than the sum of the single costs. Thus, the convention gap turns conventions from an inferred cause of score loss into a measurable, comparable, and causally testable object.

Figures & tables

Figure 1: Per-category convention gap for all six populations on the shared taxonomy. Rows are the cells of the shared taxonomy (§ 2.4 ); a row exists only if every population has enough plays in it. Markers are estimates on the estimation halves; bars are 95% bootstrap intervals (human pairs or agent games resampled). The right column gives each row’s human value. The top row is the corpus-level aggregate over all plays (

+26.16~[+24.05,+28.28]\,\text{pp}

humans;

+1.57~[+1.33,+1.81]

+21.68~[+21.34,+22.02]\,\text{pp}

for levels 1–5).

\dagger

marks the one cell where the mix of plays differs between humans and level 5 (Appendix Fig. 10 );

n

is the number of human plays in a merged cell. Row shading: indirect reference (orange), color known (blue), rank known (gray), merged (light gray), fully identified (unshaded); the strip left of zero marks gaps below literal chance. Reading the row labels: each row is one play context, described left to right by (i) the partner’s action just before the play (“After partner color-hinted it”: a color hint that touched the played card; “…color-hinted a DIFFERENT card”: the hint touched another card, i.e., indirect reference); (ii) what hints have revealed about the played card (“knows color only”, “fully identified”; “last hint was color” when only the most recent hint type is kept); (iii) timing (“played immediately” after that hint, or “held

{\geq}1

turn”); (iv) hand position (newest or older card); and, where it splits the data, (v) the hint tokens available. “Merged cell” rows pool rarer contexts that share the listed parts.

Figure 2: Mean per-cell distance between each level’s convention-gap profile and the human profile. For each level

k

, the distance

D_{k}

: the mean of

|

human

-

level

|

over the

21

category rows of Fig. 1 (pp), with 95% joint-bootstrap intervals. Blue: all

21

cells. Orange: the

11

cells where the human value exceeds every level’s (identified by the nearest-level bootstrap of Appendix F ) are excluded, leaving the conventions both populations share. Intervals are bootstrap percentiles around the full-sample value and are asymmetric by construction: resampling noise can only inflate a near-zero per-cell difference. The contrasts quoted in the text cancel this bias within draws. This per-cell distance is not the difference of the aggregates in the top row of Fig. 1 .

Situation	OBL1	OBL5	Humans (hanab.live)	Humans (AH2AC2)
Fresh color hint	$18.1$ ( $+1.0$ )	$40.4$ ( $+40.3$ )	$29.0$ ( $+40.9$ )	$31.5$ ( $+42.7$ )
Fresh rank hint	$36.9$ ( $+3.7$ )	$24.3$ ( $+11.9$ )	$24.5$ ( $+20.1$ )	$21.6$ ( $+22.1$ )
Hinted earlier	$40.6$ ( $+1.3$ )	$33.7$ ( $+6.4$ )	$41.1$ ( $+16.8$ )	$45.2$ ( $+19.4$ )
Unhinted, newest	$2.0$ ( $-13.9$ )	$0.8$ ( $+23.5$ )	$2.9$ ( $+48.8$ )	$1.1$ ( $+37.2$ )
Unhinted, older	$2.3$ ( $-9.8$ )	$0.7$ ( $+18.3$ )	$2.5$ ( $+43.2$ )	$0.7$ ( $+35.3$ )
Indirect ref. (color)	$0.40$	$0.28$ ( $+13.7$ )	$2.64$ ( $+61.3$ )	$1.08$ ( $+48.9$ )

Table 1: How often each convention situation is entered (share of plays, %), with the situation’s convention gap (pp) in parentheses. Human columns: the hanab.live corpus (all skill levels) and the independently collected AH2AC2 corpus (score-selected). Indirect-reference rows use the frozen level-5 categories of § 2.4 ; for OBL1 they give shares only. Gaps are estimates; intervals for the values discussed are in the text.

Figure 3: Convention gap by turn within the game. (a) Convention gap by 5-turn bin for levels 1–5 in self-play (both players’ plays; bins with

\geq 500

plays; bands are 95% game-clustered bootstrap intervals). The inverted U rises with level, peaking at

+21.7~[+20.3,+23.1]\,\text{pp}

(level 2) to

+36.7~[+35.2,+38.2]\,\text{pp}

(level 5) in the mid-game. (b) Level 5 decomposed: the mean literal failure probability of the chosen plays against the realized life-loss rate; the shaded area is the gap. The realized rate is flat; the literal price of the plays is what moves. (c) Humans against levels 1 and 5 by 5-turn bin (bins with

\geq 200

human plays); lines are estimates without intervals.

Figure 4: Gap and score effects of the masking interventions. (a) Gap time course under the whole-channel masks: level 5 baseline, immediacy mask and persistent mask; level 2 baseline and mask. The immediacy mask scales the mid-game peak down while preserving its shape (

+36.6\rightarrow+26.4\,\text{pp}

; the blocked plays reappear later), whereas the persistent mask removes the peak (

+9.4\,\text{pp}

). (b) Excess score cost of each single-category mask (the paired score cost of masking the category minus the cost of a frequency-matched random mask at the same trigger rate) against the category’s contribution to the aggregate gap (its share of plays times its gap, in pp), with low-gap controls in a second color. (c) Raw paired score cost of masking the top-

k

categories jointly, against the additivity reference (the sum of the

k

single-mask costs); raw costs rather than excess, so the super-additivity does not depend on the null-curve fit; the second line of each x-axis label gives the total gap under that mask (pp; baseline

21.3

). In (b) and (c), bars are 95% paired-deck bootstrap intervals. One category costs little; several together cost more than the sum of their single costs.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Level	Discovery	Estimation	Total
OBL1	11,309	11,482	22,791
OBL2	60,573	60,534	121,107
OBL3	61,228	61,487	122,715
OBL4	61,761	61,842	123,603
OBL5	62,038	61,996	124,034
All	256,909	257,341	514,250

Appendix

Table 2: Play actions entering the category induction, by level and split half. Level 1 has one seed model; levels 2–5 pool five seed models each.

Figure 5: Induced-category gap ladders for level 1 (a) and level 5 (b). Categories holding

\geq 1\%

of plays (share of plays in parentheses). Markers are gaps on the estimation half; bars are 95% game-clustered bootstrap intervals; blue markers pass Benjamini–Hochberg control at

q=0.05

, gray ones do not. Row shading gives the information family: color known (blue), rank known (gray), fully identified (unshaded). Reading the row labels, left to right: the hint type and timing (“Fresh color/rank”: played right after a hint of that type touched the card; “Delayed color/rank”: hinted earlier with that type and held before playing; “Fully identified”: color and rank both known, with the order of the hints, e.g. R

\to

C for rank then color, or “3+ hints”; “Finesse-like (color/rank)”, used in later figures: played right after a hint of that type touched a different card, i.e., indirect reference); the partner’s action just before the play where it defines the category (“after partner play”, “after partner discard”, “after C-hint on it”); how long the card was held (“held 1”, “held

{\geq}1

”, “held 2+” own turns; “lag 1”: played immediately); hand position (newest or older card); and the hint tokens available (“0”, “1-2”, “3+ tok”).

Figure 6: Out-of-sample variance explained, per level. Share of per-play residual variance explained on the estimation half (out-of-sample

R^{2}

, y axis) by the hand-crafted five-situation map, the backoff taxonomy, and a depth-4 CART tree. All three explain little at level 1 (

R^{2}\leq 0.069

); the backoff taxonomy explains

R^{2}=0.570

at level 5.

Level	Root split
1	Hint history: unhinted vs. hinted
2	Hand position: older vs. newest card
3	Hand position: older vs. newest card
4	Hint history: one color hint only vs. any other
5	Hint history: color hints only vs. any other

Appendix

Table 3: First split of the depth-4 CART tree at each level (discovery half). Hint history lists the hint types that touched the played card, in order (C: color, R: rank).

Figure 7: Per-agent convention gap and score in the ten mixed-level pairings. (a) Convention gap of each acting agent per pairing; circles are the lower-level agent (always in seat 0), squares the higher-level one, with 95% game-clustered bootstrap intervals. In pairings involving level 1 the two sides separate; among levels 2–5 they nearly coincide. (b) Mean score of each pairing (no intervals shown) against level distance

j-i

, for levels

i<j

. Pairs including level 1 fall steeply with distance; pairs among levels 2–5 do not.

Figure 8: Gap time course of a fixed agent under different partners. Convention gap of one agent’s own plays by 5-turn bin (bins with

\geq 250

plays): level 5 in self-play, with level 3, and with level 1, and level 1 with level 5 (in mixed games the lower level held seat 0; self-play pools both seats). Point estimates; no intervals are shown. The whole time course, not one phase of it, flattens with level distance.

Pair	Score	$\Delta$ , lower	$\Delta$ , higher	$\delta$ [95% CI]	Seed share (%)
$2\times 3$	$23.32\pm 0.20$	$+13.1\pm 0.7$	$+13.4\pm 0.8$	$+0.28~[-0.12,+0.67]$	$42$
$2\times 4$	$23.15\pm 0.31$	$+13.2\pm 1.1$	$+13.9\pm 0.8$	$+0.70~[+0.26,+1.14]$	$52$
$2\times 5$	$22.84\pm 0.30$	$+12.9\pm 1.1$	$+13.6\pm 0.8$	$+0.64~[+0.22,+1.07]$	$48$
$3\times 4$	$23.74\pm 0.26$	$+17.0\pm 1.2$	$+16.9\pm 1.0$	$-0.05~[-0.37,+0.27]$	$10$
$3\times 5$	$23.67\pm 0.35$	$+17.3\pm 1.2$	$+17.0\pm 1.3$	$-0.25~[-0.67,+0.16]$	$45$
$4\times 5$	$23.80\pm 0.30$	$+19.2\pm 1.5$	$+18.7\pm 1.6$	$-0.45~[-0.90,-0.00]$	$48$

Appendix

Table 4: Seed sweep: all

25

seed combinations per level pair,

200

games each. Gaps in pp (mean

\pm

sd across combinations);

\delta

is the across-combination mean of

\Delta_{\text{higher}}-\Delta_{\text{lower}}

with a

t

-based

95\%

interval treating combinations as the unit (they share seed models on each side, so the interval may be too narrow); the last column is the estimated share of across-combination variance in

\delta

beyond game-sampling noise (approximate).

Condition	Contribution	Cost	Excess [95% CI]	Total gap
single #1	$7.1$	$-0.39$	$-0.12~[-0.25,+0.01]$	$18.9$
single #2	$5.3$	$-0.36$	$-0.09~[-0.23,+0.03]$	$19.0$
single #3	$1.2$	$-0.07$	$-0.02~[-0.12,+0.09]$	$21.1$
single #4	$1.1$	$-0.01$	$+0.03~[-0.05,+0.12]$	$21.1$
single #5	$1.0$	$-0.01$	$+0.02~[-0.04,+0.09]$	$20.9$
single #6	$0.6$	$+0.10$	$+0.25~[+0.12,+0.38]$	$20.7$

Appendix

Table 5: All category-mask conditions (

1000

paired games each). Contribution is the observational share

\times

gap of the target set (estimation half, pp); cost is the paired per-deck score change against the shared baseline (points; negative values are losses); excess is the cost minus the frequency-matched random-mask cost at the same realized trigger rate, with the control curve re-fit inside every bootstrap draw, so a positive excess means the mask costs less than a random mask; total gap is the corpus gap under the mask (pp). Frequency-matched random controls define the zero line and are omitted (excess

=0

by construction). † The excess relies on extrapolating the control curve beyond the largest control rate.

Figure 9: Shared-taxonomy cells ranked by the absolute human -- level-5 difference. Same cells and basis as Fig. 1 , whose caption gives the key to the row labels,

\dagger

and

n

;

\Delta

is the human gap minus the level-5 gap (pp). Diamonds: humans, with 95% pair-clustered bootstrap intervals; dots: levels 1–5 (darker is higher); the tan bar spans the human and level-5 values; the strip left of zero marks gaps below literal chance. Row shading: indirect reference (orange), color known (blue), rank known (gray), merged (light gray), fully identified (unshaded). The three largest differences are the two indirect-reference cells and a delayed color cell.

Partner’s preceding action	Humans (hanab.live)	Humans (AH2AC2)	Level 5
Hint to another card	$+57.9~[+50.6,+62.7]$ ( $286$ )	$+43.0~[+35.2,+49.4]$ ( $229$ )	$+2.5~[-9.0,+14.2]$ ( $55$ )
Other	$+29.4~[+17.3,+40.1]$ ( $198$ )	$+33.5~[+27.7,+39.1]$ ( $507$ )	$+24.0~[+19.4,+28.7]$ ( $339$ )
All unhinted plays	$+46.2~[+35.9,+52.6]$ ( $484$ )	$+36.5~[+31.5,+40.9]$ ( $736$ )	$+21.0~[+16.8,+25.4]$ ( $394$ )
Difference from level 5, other	$+5.3~[-7.7,+17.1]$	$+9.5~[+2.0,+16.8]$

Appendix

Table 6: Unhinted plays by the partner’s preceding action. Gap (pp) with 95% interval; number of plays in parentheses. “Other” is a partner play or discard, or the first move. Humans: pair-clustered; AH2AC2 and level 5 (seed a): game-clustered. In the hanab.live corpus, the

193

unhinted plays that follow a color hint to another card (the finesse configuration) have a mean literal failure probability of

69.6\%

and a failure rate of

5.7\%

Figure 10: The two indirect-reference residual cells resolved by the played card’s own hint history. Left: sub-cell occupancy per population (shared taxonomy, estimation halves): human entries are dominated by unhinted finesse targets, agent entries by rank-known and fully-identified cards. The value right of each bar is that population’s gap in the whole cell, with its number of plays. Right: human and level-5 gaps per sub-cell (frozen level-5 taxonomy, full corpora, a narrower cell than on the left, so the two panels’ shares differ by construction; humans pair-clustered, level 5 game-clustered; sub-cells with

n<20

are marked not estimable rather than zero). The highlighted finesse rows show the largest differences: humans

+63.9

and

+45.4\,\text{pp}

where level 5 reads

+6.1

and

+1.6\,\text{pp}

. On every sub-cell estimable for both populations, the human value exceeds level 5’s, with disjoint intervals.

Figure 11: Human and agent gaps on the hand-crafted situations and the frozen level-5 categories. (a) The five hand-crafted situations for humans against levels 1 and 5 (point estimates; the values of Table 1 , where “Stale hinted” is called “Hinted earlier”). “Fresh color” and “Fresh rank”: played right after a color or rank hint that touched the card; “Stale hinted”: hinted earlier; “Unhinted (newest)” and “Unhinted (other)”: no hint touched the card, newest or older card; “hum. share”: share of human plays. (b) The frozen level-5 categories ranked by human gap: dots are humans with 95% pair-clustered intervals, ticks are level 5. The

r

in the panel title includes the

6

categories that are zero for both populations by construction (§ E.11 ). Row labels follow the key of Fig. 5 ;

n

is the number of human plays. Background shading groups rows by information family: indirect reference (orange), color known (blue), rank known (gray); fully identified rows are unshaded.

Figure 12: Per-category convention gap across levels 1–5 with the human value overlaid. Rows are the

20

non-zero frozen level-5 categories with at least

100

human plays; the row labels follow the key of Fig. 5 and

n

is the number of human plays, and the shading groups rows by information family as there. Markers give each level’s gap and the human value. The right column gives the level nearest to the human value and its bootstrap probability; “

{>}5

” means the human value exceeds every level. Where humans and agents share a convention, the agent profile plateaus by level 2–3 and humans land at that plateau; where humans diverge they exceed every level.

Multi-Agent Communication

Multi-agent communication studies how autonomous agents exchange information to coordinate decisions, share knowledge, and act together under constraints.

See all 380 papers

Sep 10, 2026

The Convention Gap: Beyond Literal Content in Human-AI Cooperation

Cooperative AI agents are often evaluated against other AIs, yet human cooperation is thought to rely on implicit conventions: shared rules for reading meaning beyond the literal message. We propose the convention gap as a diagnostic of such implicit communication. In the card game Hanabi, where hints are the only messages about one's own hand, the convention gap is the failure probability predicted exactly from the literal content of the hints and the visible cards, minus the observed failure rate of the cards played. We prove that the population gap (the value the gap approaches over many games) is zero when the partner's choices convey nothing beyond their literal content, given a fair shuffle and a player who uses only what was said and seen. A nonzero population gap thus indicates that partner choices conveyed more; it does not measure how much. The gap was +26.2 percentage points (pp) in public human-human games (+46 pp on plays of cards that no hint had pointed to), +16.4 pp for human plays with AI partners, and -0.6 pp in AI-AI games we simulated among rule-based agents. With three AI agents of a published human-AI study, the literal failure probability of the cards humans played was similar across partners (38-41%), yet human failure ranged from 14.4% to 34.4% and the gap from +24.1 to +6.2 pp, and the contrast between partners persisted within participants. Across five levels of Off-Belief Learning agents, trained so that the first level reads past actions only for their literal content and each higher level interprets more, the gap rose monotonically from +1.6 to +21.7 pp. These results argue for reporting the convention gap on human-partnered play, not only AI-AI performance, when evaluating agents intended for human partners.

Makoto Fukushima, Hua-Dong Xiong, Ehsan Moradi Pari

Honda Research Institute Japan · Honda Research Institute USA

Game-Playing Agents AI Agent Evaluation Human-AI Collaboration Emergent Communication

Ask PDF

Mar 17, 2025

A Generalist Hanabi Agent

Traditional multi-agent reinforcement learning (MARL) systems can develop cooperative strategies through repeated interactions. However, these systems are unable to perform well on any other setting than the one they have been trained on, and struggle to successfully cooperate with unfamiliar collaborators. This is particularly visible in the Hanabi benchmark, a popular 2-to-5 player cooperative card-game which requires complex reasoning and precise assistance to other agents. Current MARL agents for Hanabi can only learn one specific game-setting (e.g., 2-player games), and play with the same algorithmic agents. This is in stark contrast to humans, who can quickly adjust their strategies to work with unfamiliar partners or situations. In this paper, we introduce Recurrent Replay Relevance Distributed DQN (R3D2), a generalist agent for Hanabi, designed to overcome these limitations. We reformulate the task using text, as language has been shown to improve transfer. We then propose a distributed MARL algorithm that copes with the resulting dynamic observation- and action-space. In doing so, our agent is the first that can play all game settings concurrently, and extend strategies learned from one setting to other ones. As a consequence, our agent also demonstrates the ability to collaborate with different algorithmic agents -- agents that are themselves unable to do so. The implementation code is available at: R3D2-A-Generalist-Hanabi-Agent

Arjun V Sudhakar, Hadi Nekoei, Mathieu Reymond +3

Chandar Research Lab · Mila – Quebec AI Institute · Polytechnique Montréal +4

RL for Game AI Cooperative MARL Generalization in RL Ad Hoc Teamwork Distributed RL

Ask PDF

Oct 6, 2026

Do LLMs Act on What They Know? From Partner Representations to Cooperative Actions

Cooperation with unfamiliar partners requires adapting to communication conventions that are not known in advance. We study this problem in a controlled Hanabi-derived environment with scripted hint generation, LLM-controlled receiving decisions, and frozen model weights. Across eight LLMs, linear probes recover intent conventions substantially more accurately than target conventions, yet receiving choices do not consistently agree with the sender's convention. We compare probe-predicted and ground-truth conventions presented either as general rules or as externally computed action recommendations. Rule statements yield modest and model-dependent changes in cooperation, whereas action translation produces larger gains on average. In a Qwen3-8B case study, matched-state statement reversals reveal much greater sensitivity to action recommendations than to rule statements. Activation transfers from oracle-action and non-oracle hint-restatement donors improve intent accuracy on both action classes, but the tested alternatives do not reliably reproduce these benefits. Together, these results distinguish convention decodability, sensitivity to convention information, and cooperative performance, and highlight limitations in turning available partner information into receiving decisions.

Yuhwan Jeong, Jinnyeong Yang, Kuk-Jin Yoon

KAIST, Visual Intelligence Lab

Multi-Agent LLM Systems LLM Interpretability LLM Decision-Making Multi-Agent Systems

Ask PDF

Abstract

Multi-Agent Communication

Explore similar work

The Convention Gap: Beyond Literal Content in Human-AI Cooperation

A Generalist Hanabi Agent

Do LLMs Act on What They Know? From Partner Representations to Cooperative Actions