Interchangeable in Role, Not in History: Replacing Members of Formed LLM Teams

Jianxin Gao; Runze Li; Tianyi Yu; Liangwei Ren; Bohan Chen; Zining Wang

cs.AISep 4, 2026

Interchangeable in Role, Not in History: Replacing Members of Formed LLM Teams

Authors: Jianxin Gao, Runze Li, Tianyi Yu, Liangwei Ren, Bohan Chen, Zining Wang

Organizations: China Agricultural University · Jilin University · Tianjin University of Finance and Economics · Tianjin University of Science and Technology

Abstract

Multiagent LLM systems often treat a role as a slot: when a member is replaced, any agent competent at the role is expected to fill it. We test this view on teams whose members keep short written notebooks. A one-seat swap replays a formed, frozen team with its own member, with a same-role newcomer carrying another team's notebook for that role, or with a newcomer carrying none. In cooperative cooking, wildfire response and medical question answering, the experienced newcomer, averaged over seats, outscores the one without a notebook in every setting, so the usual comparison would approve it. Against the original team, the same replacement falls short in every setting, and the team spends more messages per unit of progress. On the hardest cooking level, the shortfall appears only after the team has worked together, persists across model families and decoding temperatures, and narrows by two thirds with added planning-only members. Because the team's history is written down, we can take it apart. In the default configuration on that level, notebooks from the same donor teams leave a newcomer that leads, holding the recipe, worse off than no notebook and one that follows better off. The notebook lines about the partner reproduce this whole difference between the seats. Whether an agent can replace a teammate depends on the seat and the team it enters.

Figures & tables

Figure 1. The one-seat swap. Receiver R and donor D form separately and are frozen. R’s seat $j$ is then replayed with its own notebook $R_{j}$ (Original, O), with D’s notebook $D_{j}$ (Experienced, E) or with none (Fresh, F). Red markers show the notice. Bars and notebook lines are illustrative. Schematic of the one-seat swap protocol comparing Original, Experienced and Fresh branches after receiver and donor teams form independently.

	Task score					Messages per unit of progress			Tokens per unit of progress
Setting	Orig.	Exp.	Fresh	Loss	Gain	Orig.	Exp.	Prem. (%)	Orig.	Exp.	Prem. (%)
Collab-Overcooked L1	93.91	92.66	89.84	1.25	2.82	2.94	3.39	15.45	46.35	53.66	15.76
Collab-Overcooked L2	84.52	81.41	77.56	3.11	3.85	5.20	6.24	19.87	88.04	104.85	19.09
Collab-Overcooked L3	71.18	64.12	57.96	7.05	6.16	8.34	11.21	34.33	163.73	218.03	33.17
CREW-Wildfire	88.13	85.50	83.09	2.63	2.41	3.43	3.75	9.08	82.89	89.98	8.55
MedQA	90.71	88.42	86.46	2.29	1.96	8.83	9.29	5.31	503.10	524.02	4.16

Table 1. Breadth screen. Averaged over seats, the experienced newcomer beats a fresh one in every setting yet falls short of the original team. Branch means over ten repetitions. Loss is O

-

E, gain is E

-

F, and premiums compare Experienced with Original.

Figure 2. The argument in three steps. (a) Task score against formation length on L3, pooled over the replaced seats, with the loss shaded. (b) Gain E

-

F by level and replaced seat in the default configuration, with 95% intervals over ten repetitions. (c) Seat asymmetry per repetition: the partner half (filled) tracks the whole notebook and the task half (open) does not. The dotted line marks equality. Three panels. (a) Line plot of task score against formation episodes (0, 1, 5, 10) on L3 for the Original, Experienced and Fresh branches, with the continuity loss shaded. (b) Experience gain with 95% intervals for L1, L2 and L3 when the initiator or the responder is replaced. (c) Scatter plot of the per-repetition seat asymmetry of the whole notebook against that of the partner half and of the task half.

		Task score (PC)					Premium (%)
Level	Replaced seat	Orig.	Exp.	Fresh	Loss	Gain	Messages	Tokens
L1	Initiator	91.35	88.60	86.49	2.75	2.10	23.25	28.85
	Responder	91.35	89.24	86.64	2.11	2.61	24.34	29.32
L2	Initiator	85.08	81.32	76.88	3.76	4.44	19.40	31.77
	Responder	85.08	80.57	74.09	4.51	6.48	22.71	34.96
L3	Initiator	70.30	54.04	62.02	16.26	$-$ 7.98	70.45	96.08

Table 3. Seat study in the default configuration. On L3 the gain changes sign with the replaced seat. Each row is a mean over ten repetitions with the named seat replaced, and both seats share one Original.

Contrast	Mean	95% interval	$k$ / $n$
$\Delta_{\mathrm{seat}}$ , whole notebook	15.03	$[10.41,19.65]$	10/10
$\Delta_{\mathrm{seat}}$ , partner half	15.10	$[10.79,19.42]$	10/10
$\Delta_{\mathrm{seat}}$ , task half	$-0.59$	$[-2.47,1.28]$	5/10
Initiator gain, whole	$-7.98$	$[-11.44,-4.51]$	9/10
Initiator gain, partner half	$-7.91$	$[-11.49,-4.33]$	10/10
Initiator gain, task half	1.47	$[0.57,2.37]$	8/10

Table 5. Contrasts of Sections 4.4 and 4.5 on L3, default configuration: mean, 95% interval and

k

n

. The last row repeats

\Delta_{\mathrm{seat}}

in the replication with twenty further pairs.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Seat	Orig.	Exp.	Fresh	Loss	Gain	95% interval	$k$
Initiator	70.90	60.80	63.50	10.10	$-2.70$	$[-8.32,2.92]$	4/5
Responder	71.46	67.44	52.42	4.02	15.02	$[8.61,21.43]$	5/5
Pooled	71.18	64.12	57.96	7.05	6.16

Appendix

Table S1. Breadth screen on L3, split by the replaced seat: branch means over the five repetitions that replaced each seat, and the gain E

-

F with its 95% interval and the number of repetitions with the sign of its mean. The pooled row is the L3 row of Table 1 of the paper.

Factor	Setting	Loss	95% interval
Formation length	0 episodes	0.02	$[-2.57,2.61]$
	1 episode	2.61	$[0.48,4.74]$
	5 episodes (default)	12.49	$[-0.42,25.39]$
	10 episodes	10.41	$[8.76,12.06]$
Model family	Gemini $\rightarrow$ Gemini	9.19	$[7.31,11.07]$
	DeepSeek $\rightarrow$ DeepSeek	10.87	$[9.10,12.65]$

Appendix

Table S2. Continuity loss O

-

E in the configuration study on L3 (Table 2 of the paper): mean over ten repetitions and 95% interval. The planning rows are compared through their paired reduction in Table S3 .

Contrast	Mean	SD	95% interval	$k$ /10
Seat asymmetry and its split, L3
Seat asymmetry $\Delta_{\mathrm{seat}}$ , whole notebook	15.03	6.45	$[10.41,19.65]$	10/10
Seat asymmetry $\Delta_{\mathrm{seat}}$ , partner half only	15.10	6.03	$[10.79,19.42]$	10/10
Seat asymmetry $\Delta_{\mathrm{seat}}$ , task half only	$-0.59$	2.62	$[-2.47,1.28]$	5/10
Seat asymmetry $\Delta_{\mathrm{seat}}$ , partner half $-$ whole notebook	0.07	2.13	$[-1.45,1.60]$	5/10
Experienced term: responder branch $-$ initiator branch	10.02	5.53	$[6.06,13.98]$	9/10

Appendix

Table S3. Repetition-level summaries of the contrasts of the seat study and the content and planning studies: mean of the ten repetition-level values, their standard deviation (SD), the 95% interval on the mean and the number of repetitions with the sign of the mean. Contrasts are in points of PC unless marked.

Contrast	1	2	3	4	5	6	7	8	9	10
$\Delta_{\mathrm{seat}}$ , whole notebook	26.04	24.31	18.24	18.42	8.26	11.72	10.44	11.03	7.92	13.91
$\Delta_{\mathrm{seat}}$ , partner half	24.56	25.46	16.48	16.91	8.68	10.86	12.12	8.20	12.03	15.72
$\Delta_{\mathrm{seat}}$ , task half	$-2.46$	1.28	2.76	0.87	$-6.28$	$-1.20$	$-1.86$	$-1.10$	1.92	0.16
Rewording cost, initiator	1.15	0.10	$-0.02$	0.97	2.20	0.20	0.13	1.10	$-0.88$	0.28
Rewording cost, responder	$-0.29$	$-0.38$	0.57	$-1.65$	1.33	0.11	$-1.16$	0.44	0.15	0.39

Appendix

Table S4. Every repetition of the seat asymmetries and rewording costs of Sections 4.4 and 4.5 of the paper, in points.

Contrast	Mean	95% interval
Continuity loss O $-$ E, initiator	15.60	$[12.42,18.78]$
Continuity loss O $-$ E, responder	6.90	$[4.61,9.19]$
Continuity loss, both seats averaged	11.25	$[8.87,13.63]$
Experience gain E $-$ F, initiator	$-7.10$	$[-9.64,-4.56]$
Experience gain E $-$ F, responder	6.40	$[3.71,9.09]$
Seat asymmetry $\Delta_{\mathrm{seat}}$	13.50	$[10.05,16.95]$

Appendix

Table S5. Replication of the L3 seat study: twenty further receiver–donor pairs in the default configuration, with both seats replayed from each snapshot. Means over the twenty pairs and 95% intervals; the loss averaged over both seats is the mean over pairs of the two seats’ losses, and

\Delta_{\mathrm{seat}}

is positive in 19 of the 20 pairs.

	Initiator				Responder
Rep.	All	Task	Partner	Drop	All	Task	Partner	Drop
1	7	3	4	0	7	4	3	0
2	9	5	4	0	7	4	3	0
3	8	3	4	1	5	3	2	0
4	8	4	4	0	10	5	4	1
5	6	4	2	0	7	3	4	0

Appendix

Table S6. Composition of the twenty donor notebooks on L3: lines per notebook by label.

Multi-Agent LLM Systems

Multi-agent large language model systems coordinate multiple AI agents to solve tasks through communication, specialization, shared tools, and memory.

See all 1,050 papers

The Interaction Tax: When Communication Erases Diversity in Multi-Agent Teams
Aug 24, 2026Summer Eunhyung Ann, Haokun Liu, Chenhao TanMulti-Agent LLM Systems Multi-Agent Collaboration
Department of Computer Science, University of Chicago, Chicago, IL, USA.
When Does Personality Composition Matter for Multi-Agent LLM Teams?
Jun 25, 2026Aryan Keluskar, Amrita Bhattacharjee, Huan LiuMulti-Agent LLM Systems Multi-Agent Negotiation
School of Computing & AI Arizona State University Tempe, AZ 85281, USA
TeamBench: Evaluating Agent Coordination under Enforced Role Separation
May 8, 2026Yubin Kim, Chanwoo Park, Taehan Kim +9Multi-Agent Coordination LLM Agent Evaluation
MIT · Google Research · Independent Researcher +1

Aug 24, 2026J/K move · Enter open · S save

The Interaction Tax: When Communication Erases Diversity in Multi-Agent Teams

Summer Eunhyung Ann, Haokun Liu, Chenhao Tan

Department of Computer Science, University of Chicago, Chicago, IL, USA.

Multi-Agent LLM Systems Multi-Agent Collaboration Multi-Agent Systems Multi-Agent Communication

Open paper AskPDF arXiv

Does multi-agent LLM interaction help or hurt? Some work reports gains from debate (Du et al., 2024), critique loops (Chen et al., 2025), and mixture-of-agents synthesis (Wang et al., 2025), while other work finds that interaction adds cost without improving quality under equal budgets (Tran & Kiela, 2026; Xu et al., 2026; Jarrett et al., 2025), or that independent sampling already captures multi-agent gains (Li et al., 2024). We argue this contradiction partly reflects a missing distinction, because not all multi-agent communication is equal. Different model families find structurally different solutions, but when agents read each other's complete outputs, their proposals converge within one round, erasing the diversity that motivates using multiple models. We call this the interaction tax. We test 11 verifier-scored optimization tasks under matched budgets and find that full-solution interaction is a weak default. Independent proposal generation avoids this collapse. Full-solution interaction mainly makes agents stay close to the first solution they see instead of trying different approaches, and critique helps only if the violated rule is easy for the LLM to find and fix. These results suggest that multi-agent performance depends less on the number of agents than on the information they exchange, and interaction helps only when agents share the right information at the right time.

Abstract

Multi-Agent LLM Systems

Explore similar work

The Interaction Tax: When Communication Erases Diversity in Multi-Agent Teams

When Does Personality Composition Matter for Multi-Agent LLM Teams?

TeamBench: Evaluating Agent Coordination under Enforced Role Separation