BLUEmed: Retrieval-Augmented Multi-Agent Debate for Clinical Error Detection
Authors: Saukun Thika You, Nguyen Anh Khoa Tran, Wesley K. Marizane, Hanshu Rao, Qiunan Zhang, Xiaolei Huang
Organizations: Department of Computer Science, University of Memphis, Memphis, United States · Department of Computer Information Systems, Alabama State University, Montgomery, United States
Terminology substitution errors in clinical notes, where one medical term is replaced by a linguistically valid but clinically different term, pose a persistent challenge for automated error detection in healthcare. We introduce BLUEmed, a multi-agent debate framework augmented with hybrid Retrieval-Augmented Generation (RAG) that combines evidence-grounded reasoning with multi-perspective verification for clinical error detection. BLUEmed decomposes each clinical note into focused sub-queries, retrieves source-partitioned evidence through dense, sparse, and online retrieval, and assigns two domain expert agents distinct knowledge bases to produce independent analyses; when the experts disagree, a structured counter-argumentation round and cross-source adjudication resolve the conflict, followed by a cascading safety layer that filters common false-positive patterns. We evaluate BLUEmed on a clinical terminology substitution detection benchmark under both zero-shot and few-shot prompting with multiple backbone models spanning proprietary and open-source families. Experimental results show that BLUEmed achieves the best accuracy (69.13%), ROC-AUC (74.45%), and PR-AUC (72.44%) under few-shot prompting, outperforming both single-agent RAG and debate-only baselines. Further analyses across six backbone models and two prompting strategies confirm that retrieval augmentation and structured debate are complementary, and that the framework benefits most from models with sufficient instruction-following and clinical language understanding.
Figures & tables
Error Type
Count
Percentage
Avg Tokens
Diagnosis Error
116
37.3%
161.4
Management Error
97
31.2%
160.9
Treatment Error
51
16.4%
155.2
Pharmacotherapy Error
36
11.6%
149.8
Causal Organism Error
11
3.5%
128.5
Total Error Cases
311
100.0%
158.6
TABLE I: Error Type Distribution in MEDEC Test Collection
Fig. 1: The BLUEmed framework. The pipeline consists of a Hybrid RAG (combining dense, sparse, and online search) with a multi-agent debate structure in which experts present their respective arguments and a judge model validates the final output, with an integrated hybrid safety layer to ensure medical accuracy.
Setting
Method
Accuracy
F1-score
Precision
Recall
ROC-AUC
PR-AUC
Zero-Shot
RAG-Single (MayoClinic)
58.63
59.44
60.74
58.20
50.00
50.00
RAG-Single (WebMD)
57.79
60.50
59.02
62.06
62.93
50.00
LLM-Debate
55.28
66.91
54.44
86.82
59.51
59.47
BLUEmed (Ours)
67.67
66.67
72.01
62.05
70.24
68.88
Few-Shot
RAG-Single (MayoClinic)
57.29
40.56
73.73
27.97
50.00
50.00
RAG-Single (WebMD)
58.96
43.16
77.50
29.90
50.00
50.00
TABLE II: Results of baselines and the proposed BLUEmed (percentages).
Methods
Model
Accuracy
F1-score
Precision
Recall
ROC-AUC
PR-AUC
Zero-Shot
GPT-4o
67.67
66.67
72.01
62.05
70.24
68.88
GPT-5.2
59.57
57.77
62.28
53.89
62.54
60.63
Gemini-2.0-Flash
69.18
68.17
73.78
63.34
71.44
70.36
Qwen3-4B-Instruct
59.97
52.10
69.15
41.80
52.54
54.08
Qwen3-8B
54.70
50.73
58.65
44.70
51.65
53.21
Llama3.2-3B-Instruct
48.58
13.03
54.76
7.40
49.74
52.04
TABLE III: Performance of BLUEmed with different backbone models (percentages).
Retrieval-Augmented Generation (RAG) is widely employed to mitigate risks such as hallucinations and knowledge obsolescence in medical question answering, yet its predominantly single-round, static retrieval paradigm misaligns with the multi-stage process of clinical reasoning. This compressed workflow induces two structural deficiencies: question-to-query translation often lacks clinically grounded semantic interpretation, and retrieval lacks iterative sufficiency feedback, making it difficult to form reliable evidence chains. We argue that both issues stem from a deeper cause: overloading a single reasoning chain with heterogeneous tasks of interpretation, exploration, and adjudication. The remedy is to reconstruct the workflow via task decoupling and dynamic multi-round exploration. To this end, we propose SEMA-RAG, a Self-Evolving Multi-Agent RAG framework for medical question answering, which assigns these roles to three specialist agents: the Interpreter Agent for clinical schema interpretation, the Explorer Agent for sufficiency-driven self-evolving retrieval, and the Arbiter Agent for evidence adjudication and answer selection. Across five benchmarks and five LLM backbones, SEMA-RAG improves the strongest baseline by +6.46 accuracy points on average, measured per backbone.
Yongfeng Huang, Ruiying Chen, James Cheng
CSE, The Chinese University of Hong Kong · Wuhan University of Technology
As Large Language Models (LLMs) are increasingly deployed in healthcare settings, accurate error detection and correction in generated or existing text becomes critical, as even minor mistakes can pose risks to patient safety. Existing methods for error detection and correction, including automated checks and heuristic-based approaches, do not generalize well across unseen datasets. In this paper, we propose MedGuards as a medical safety guardrail, which is a new framework that treats medical error detection and correction as a multi-agent in-context learning task. Specialized agents separately detect, localize, and correct errors, while a confidence-guided arbitration mechanism resolves disagreements using reasoning traces and confidence scores. This design enhances interpretability, robustness, and adaptability, without requiring additional training of the base LLMs. Additionally, we introduce the Keyword-Prioritized Correction Score (KPCS), a new evaluation metric that considers whether critical keywords within the reference text are generated correctly, providing a more comprehensive assessment than conventional metrics. Experiments across four multilingual medical datasets consisting of clinical notes demonstrate significant improvements by the proposed framework across several metrics and models. Our aim is to enable safer deployment of LLMs in real-world healthcare applications. For reproducibility, we make our code publicly available at https://github.com/congboma/MedGuards.
Congbo Ma, Hu Wang, Yichun Zhang +1
New York University Abu Dhabi · Khalifa University · New York University
While interpretable prototype networks offer compelling case-based reasoning for clinical diagnostics, their raw continuous outputs lack the semantic structure required for medical documentation. Bridging this gap via standard Retrieval-Augmented Generation (RAG) routinely triggers ``retrieval sycophancy,'' where Large Language Models (LLMs) hallucinate post-hoc rationalizations to align with visual predictions. We introduce ProtoMedAgent, a framework that formalizes multimodal clinical reporting as an iterative, zero-gradient test-time optimization problem over a strict neuro-symbolic bottleneck. Operating on a frozen prototype backbone, we distill latent visual and tabular features into a discrete semantic memory. Online generation is strictly constrained by exact set-theoretic differentials and a reflective Scribe-Critic loop, mathematically precluding unsupported narrative claims. To safely bound data disclosure, we introduce a semantic privacy gate governed by k-anonymity and ℓ-diversity. Evaluated on a 4,160-patient clinical cohort, ProtoMedAgent achieves 91.2% Comparison Set Faithfulness where it fundamentally outperforms standard RAG (46.2%). ProtoMedAgent additionally leverages a binding ℓ-diversity phase transition to systematically reduce artifact-level membership inference risks by an absolute 9.8%.