Multi-agent LLM systems improve performance on clinical tasks, and that improvement is often read as evidence of a more trustworthy system. However, the two are not the same claim. A clinician acts on the stated reasoning and not only on the conclusion, so whether collaboration reduces hallucination, and what decides when it does, has to be measured in the reasoning too. Yet most evaluations score only the final answer. We evaluate four systems (a single-LLM baseline and three multi-agent topologies) across four models and three clinical tasks, scoring hallucination, under a factuality and faithfulness taxonomy, inside the rationale each system generates and reporting it alongside the task metric. We report that collaboration buys task performance but not lower hallucination as a rule, and no topology is reliably better than another. The effect differs by model: of the 36 model--system--task comparisons, 12 are large enough to resolve --- four decreases and eight increases, with no model on both sides. What sets the level, and the direction of any change, is the model rather than the topology or the number of agents. A self-correction experiment is consistent with this: shown its own flagged errors, every model except Llama-3.2-3B-Instruct reduces hallucination; the one model for which collaboration lowers hallucination gains the most, and Llama-3.2-3B-Instruct, the model whose hallucination it amplifies most, does not reduce its hallucination at all. We find that collaboration redistributes what its models can already do; it does not supply what they cannot.
Figures & tables
Figure 1: The four systems. Every seat holds the same model, so the systems differ only in the number of agents, their prompts and how they are wired; orange marks the agent that commits to the final answer. Each produces a rationale alongside its answer, and it is that rationale the hallucination measures score.
Figure 2: Hallucination taxonomy used for our analysis: factuality hallucination and faithfulness hallucination.
Figure 3: How a rationale becomes a hallucination rate: decompose, judge each claim, judge each pair. Grey marks a label the judge assigns but the paper does not report.
Figure 4: Task performance (top) and hallucination rate (bottom), by model and system.
Figure 5: Factuality hallucination on MedExpQA, by factual contradiction and factual fabrication.
Figure 6: Faithfulness hallucination, by sub-type and task.
Topology
Model
SLS
N=3
N=7
ΔN7−N3
MedExpQA (factuality)
Panel
GPT-5.4-mini
0.153
0.131
0.136
+0.005
Panel
Llama-3.2-3B-Instruct
0.297
0.326
0.336
+0.010
Debate
GPT-5.4-mini
0.153
0.143
0.148
+0.005
Debate
Llama-3.2-3B-Instruct
0.297
0.351
0.337
-0.014
Hier
GPT-5.4-mini
0.153
0.136
0.151
+0.015
Table 1: Hallucination rate at three and seven agents, against the single-agent baseline. ΔN7−N3 is the change from three agents to seven.
Task and category
own
→ GPT
Δ
Qwen3-32B
MedExpQA contradiction
0.093
0.043
−0.049
MedExpQA fabrication
0.167
0.112
−0.055
EHRNoteQA contextual
0.042
0.023
−0.019
EHRNoteQA logical
0.010
0.003
−0.007
MIMIC-DI contextual
0.096
0.102
+0.006
Table 2: Hallucination by sub-type when the aggregator alone is replaced with GPT-5.4-mini, the proposal pool held fixed. Hierarchical system; rows are each task’s sub-types, means over 200 items. own : the pool’s own model as aggregator, re-run under the same settings (temperature =0 ).
Figure 7: Net change in hallucination after each model is shown its own flagged claims (hallucination rate of the rewrite minus that of the single agent, with 95% bootstrap intervals). Green: interval excludes zero. Levels in Appendix Table 6 .
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
MedExpQA, EHRNoteQA
MIMIC-DI detection
Appendix
Figure 8: The prompt given to every role of every topology, for the question-answering tasks (left) and for MIMIC-DI detection (right). The systems differ only in the number of agents, these prompts and how the roles are wired; all decoding and context settings are identical and are given in Appendix B .
Figure 9: The guideline card of the annotation interface, shown at sign-in and reachable at any time, with its wording unchanged; the shaded panel is shown with the logical task. Lines about navigating the interface and about handling the credentialed data are omitted.
Figure 10: One item screen per task in the annotation interface, with its wording unchanged. The banner is shown on every screen; the reference is on the left, and the question and the answer buttons on the right. Reference text is drawn as grey bars.
Human–human
Judge–human
Hallucination sub-type
Scale
n
Agreement
κ
AC1
κ
AC1
Factual contradiction / fabrication
3-way
150
0.640
0.407
0.486
0.552
0.615
Contextual inconsistency
3-way
100
0.640
0.435
0.483
0.494
0.500
Logical inconsistency
binary
100
0.720
0.406
0.475
0.440
0.453
Appendix
Table 4: Judge validation against blind human annotation. Agreement is raw observed agreement between the two annotators; κ is unweighted Cohen’s κ , since the three-way categories are not ordinal, and AC1 is Gwet’s first-order agreement coefficient, which is less sensitive to unbalanced label marginals. Judge–human is the mean of the judge’s agreement with each annotator. For factuality, the judge’s label is its verdict against the supplied reference alone, before retrieval, since that is what the annotators saw. Logical inconsistency is scored binary because the reported measure counts contradiction only.
Aggregator
Task
Pool
GPT-5.4-mini
DeepSeek-V4.1-flash
Qwen3-32B
Llama-3.2-3B-Instruct
MedExpQA
GPT-5.4-mini
0.034 † ± 0.007
0.036 ± 0.007
0.050 ± 0.008
0.031 ± 0.007
Qwen3-32B
0.043 ± 0.010
0.078 ± 0.012
0.093 † ± 0.013
0.087 ± 0.014
Llama-3.2-3B-Instruct
0.044 ± 0.010
0.060 ± 0.012
0.111 ± 0.016
0.148 † ± 0.017
EHRNoteQA
GPT-5.4-mini
0.021 † ± 0.006
0.021 ± 0.007
0.025 ± 0.007
0.019 ± 0.006
Qwen3-32B
0.023 ± 0.008
0.031 ± 0.008
0.042 † ± 0.009
0.039 ± 0.008
Appendix
Table 5: Every pool–aggregator pair. Proposals come from the saved Hierarchical runs and only the aggregator is re-run, so within a row the content aggregated is identical. Cells are the factual contradiction rate on MedExpQA and the contextual inconsistency rate on EHRNoteQA and MIMIC-DI detection (lower is better), ± half the width of each cell’s 95% bootstrap CI; green cells are lower and red cells higher than the row’s diagonal, where the pool’s own model is re-run as aggregator under the same settings (temperature =0 ); † marks the diagonal, where pool and aggregator share a model. DeepSeek-V4.1-flash appears as an aggregator but not as a pool, so that column has no diagonal.
before
after
net change
95% CI
relative
factual contradiction
GPT-5.4-mini
0.042
0.020
− 0.023
[ − 0.032, − 0.014]
− 53%
DeepSeek-V4.1-flash
0.056
0.038
− 0.018
[ − 0.029, − 0.006]
− 31%
Qwen3-32B
0.090
0.072
− 0.017
[ − 0.032, − 0.003]
− 19%
Llama-3.2-3B-Instruct
0.140
0.126
− 0.014
[ − 0.035, +0.006]
− 10%
factual fabrication
Appendix
Table 6: The self-correction experiment in numbers (hallucination rate). before and after are the hallucination rates of the single-agent rationale and of the rewrite; net change is their difference, in bold where its 95% bootstrap interval excludes zero, and relative is the net change as a share of before . Items with no flag are not re-run and count as unchanged.
shown a contradiction
re-run for other errors
pairs, all items
hallucination rate
model
task
items
before
after
items
new
before
after
before
after
GPT-5.4-mini
EHRNoteQA
8
10
1
42
2
10
3
0.007
0.002
MIMIC-DI
6
6
1
116
7
6
8
0.009
0.009
DeepSeek-V4.1-flash
EHRNoteQA
6
10
3
58
5
10
8
0.005
0.005
MIMIC-DI
4
6
2
56
4
6
6
0.010
0.009
Qwen3-32B
EHRNoteQA
13
19
8
69
8
19
16
0.011
0.007
Appendix
Table 7: Logical inconsistency in the self-correction experiment, in contradicting pairs. shown a contradiction : items whose prompt listed at least one contradicting pair, with the pairs in the original and in the rewrite. re-run for other errors : items re-run for a contextual flag only, whose rewrites’ pairs are all new. Items not re-run have no pairs and are omitted from the counts, as are rewrites with no claim. hallucination rate : the share of claims in a contradicting pair, over all items, as in Table 6 .