Consistent but Biased: Chair Personas Trade Noise for Bias in Multi-Agent Clinical Decisions

Yitong Bai; Chengyi Zhang; Ziyang Wang

cs.MAOct 22, 2026

Consistent but Biased: Chair Personas Trade Noise for Bias in Multi-Agent Clinical Decisions

Authors: Yitong Bai, Chengyi Zhang, Ziyang Wang

Organizations: Columbia University · Swansea University · Aston University

Abstract

In clinical systems built from several large language model (LLM) agents, one agent often chairs the discussion and makes the final decision. Whether the chair's persona prompt changes that decision, or the agreement between repeated runs that some systems treat as reliability, has not been tested with the case held constant. We ask whether the persona changes the decision or only its timing, how it affects agreement between runs, and which behaviors appear when the prompt describes only personality traits. A simulated department of seven agents discussed 20 published cases of degenerative cervical myelopathy, with the case reports replayed unchanged in every run. Only the prompts of the chair and its deputy changed (no persona, or a consultative, autocratic or avoidant one), written either as rules of conduct or as personality traits. With personas written as rules, the chair chose to operate now in 93% of runs when autocratic and in 2% when avoidant, against 57% without a persona. Surgery was still planned in nearly all decisions, so the personas mainly changed whether to operate now or after more tests. These effects held with a second model and reworded prompts. The autocratic and avoidant personas made the three runs of a case agree more often, but only about as often as chance predicts, and Fleiss $κ$ did not rise. Prompts that described only personality traits kept more than half of the autocratic and avoidant effects on operating now. They reproduced quick decisions, delays and extra tests, but not ignoring red flags, dropping backup plans or passing the decision to the family. Studies should report and vary the chair's persona as they do the model, and correct agreement between runs for chance before calling it reliability.

Figures & tables

Figure 1: Three chair personas and the components that psychology predicts for each (Section 3.3 ).

Figure 2: The simulated department (a) and the gist of each persona prefix (b), given in full in Appendix A .

ID	Hypothesis (against the neutral arm)	Direction	Test	Section
H1	Autocratic chair rules to operate now more often	$>$	Sign-flip, one-sided; Holm over H1–H3	4.1
H2	Avoidant chair rules to operate now less often	$<$	Sign-flip, one-sided	4.1
H3	Consultative chair differs in rulings to operate now	$\neq$	Sign-flip, two-sided	4.1
H4	A persona moves surgery intended less than operating now	–	Descriptive; bootstrap CI of the gap	4.2
H5	Share of cases whose three runs agree, per arm	–	Descriptive	4.3
H6a–c	H1–H3 for the trait-only version	$>$ , $<$ , $\neq$	As H1–H3; Holm over H6a–c	4.4

Table 1: Prespecified hypotheses

Figure 3: Every ruling of the chair (360 runs).

		Rules version			Trait-only version
	Neutral	Consultative	Autocratic	Avoidant	Consultative	Autocratic	Avoidant
Runs	60	60	60	60	40	40	40
Share of runs
Operate now	57%	52%	93%	2%	40%	77.5%	15%
Work up first	42%	48%	7%	95%	60%	22.5%	82.5%
Conservative care	2%	0	0	3%	0	0	0

Table 2: Rulings by arm (three runs per case in the rules version, two in the trait-only version)

Component	Measure	Predicted	Neutral	Rules	Trait-only	Reproduced by the trait-only version?
Autocratic
Seizing	Operate now	$\uparrow$	57%	93%	77.5%	Yes
Reduced information search	Tests ordered	$\downarrow$	2.8	0.7	2.4	Weakly
Freezing	All red flags not addressed	$\uparrow$	50.0%	71.8%	50.1%	No
	Plan has a contingency	$\downarrow$	98%	40% *	100%	No
Subordinate silence	Deputy objections (per run / first turn)	$\downarrow$	3.4 / 3.2	1.0 / 1.0 *	3.0 / 2.9	No (deputy’s prefix only)

Table 3: Predicted direction of change from the neutral chair and observed behavior for each component. Asterisks mark behaviors that the rules prefix required.

Figure 4: How much of each effect of the rules version the trait-only version reproduced.

		Rules version			Trait-only version
	Neutral	Consultative	Autocratic	Avoidant	Consultative	Autocratic	Avoidant
Borderline or not operated (6)	56%	28%	100%	6%	42%	83%	17%
Difficult differential diagnosis (8)	50%	54%	88%	0	38%	69%	13%
Clear indication (6)	67%	72%	94%	0	42%	83%	17%

Table 4: Rulings to operate now by case group (exploratory)

	Operate now				Difference from neutral in points (cases higher/lower)
	Neutral	Consultative	Autocratic	Avoidant	Autocratic	Avoidant	Consultative
Original setting (3 runs per cell)	60%	60%	97%	3%	+37 (6/0)	$-57$ (0/9)	0 (3/4)
Decision model swapped (1 run per cell)	50%	40%	80%	0	+30 (3/0)	$-50$ (0/5)	$-10$ (2/3)
Version b (1 run per cell)	50%	30%	100%	0	+50 (5/0)	$-50$ (0/5)	$-20$ (2/4)

Table 5: Swapping the decision model and the wording (10-case subset)

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Stages I–II of case C7, the same in every run of the case.

Figure 6: Stages III–IV of case C7 under the autocratic (left) and the avoidant (right) chair, run 1 of each.

Figure 7: The neutral chair’s ruling in each of the three runs of case C7 (negotiation not shown).

Figure 8: Stages III–IV of case C7 under the consultative chair, run 1: round 1 (left) and round 2 (right).

LLM Agents

Large language model agents use language models to plan, use tools, and act across multi-step tasks and environments.

See all 705 papers

Agents Catching Agents: Shortcut Cascades and Benchmark Gaming in Clinical Multi-Agent Systems
Aug 4, 2026Sebastián Andrés Cajas Ordóñez, Agastya Munnangi, Aldo Marzullo +13Emergent Behavior in Multi-Agent Systems Multi-Agent LLM Systems
MIT Critical Data, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States · Georgia State University, Atlanta, Georgia, United States · Politecnico di Milano, Milan, Italy +12
Persona Following Is Not Selective Control: The Neutrality Gap in LLM User Simulation
Sep 28, 2026Jiashen Ren, Wenlin Zhang, Bohan Zhang +5Persona-Conditioned Generation User Simulation
City University of Hong Kong · Stanford University
Multi-agent discussion gains less when dissent is withheld
Sep 29, 2026Chand Sahil Mansuri, Xin Wang, Mengying Li +4Multi-Agent LLM Systems Opinion Dynamics
School of Systems Science and Industrial Engineering, Binghamton University, Binghamton, NY, USA · School of Management, Binghamton University, Binghamton, NY, USA · IBM T. J. Watson Research Center, Yorktown Heights, NY, USA

Aug 4, 2026J/K move · Enter open · S save

Agents Catching Agents: Shortcut Cascades and Benchmark Gaming in Clinical Multi-Agent Systems

Sebastián Andrés Cajas Ordóñez, Agastya Munnangi, Aldo Marzullo, Felipe Ocampo Osorio+12

MIT Critical Data, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States · Georgia State University, Atlanta, Georgia, United States · Politecnico di Milano, Milan, Italy+12

Emergent Behavior in Multi-Agent Systems Multi-Agent LLM Systems AI Agent Safety Benchmarks AI Agent Reliability Clinical Decision Support LLM Agent Evaluation+3

Open paper AskPDF arXiv

Clinical decision support is moving toward committees of language-model agents deliberating on a shared workspace. We ask whether such committees can be gamed by shortcuts, cues a benchmark rewards but a clinician would ignore. Across seven cohorts on six public datasets spanning text (MedQA-USMLE, MedMCQA, MIMIC-CXR reports), imaging (NIH ChestX-ray14, MIMIC-CXR-JPG, CheXpert) and tabular ICU records (SUPPORT2), Gemini committees resist these cues in isolation (flip 5-16%), yet a socially plausible shortcut spreads: when two peers assert the same wrong answer, the holdout under test adopts it in 38% of cases, as does a false "pre-screen" system flag, on both capability tiers. Of three oversight agents, a gate cannot separate adoption from honest agreement (false-positive rate 100%); a same-lineage judge reading only the transcript flags adoption on text (precision 100%, recall 93%) but collapses onto the gate in imaging; a referee that privately re-queries the holdout transfers to imaging (77-88% precision, 13-21% false-positive rate). Tripling a cue's visual salience does not move contagion, whereas a second peer voice raises it by half again. Gaming a hidden rubric is near-silent: only 1/10 text and 1/134 imaging drifters name the rubric they moved toward. What games a committee is social plausibility, and only a referee independent of self-report catches it. Code: https://github.com/criticaldata/benchmaxxing

Abstract

LLM Agents

Explore similar work

Agents Catching Agents: Shortcut Cascades and Benchmark Gaming in Clinical Multi-Agent Systems

Persona Following Is Not Selective Control: The Neutrality Gap in LLM User Simulation

Multi-agent discussion gains less when dissent is withheld