cs.AISep 28, 2026
SaveBaRe-Mem: Bayesian Reliability Memory for Robust and Adaptive Agent Consultation
Organizations: DeCLaRe Lab, Nanyang Technological University · Peking University
Abstract
In multi-agent systems, reliable consultation is challenging because advisor capabilities vary across tasks, and misleading information can make consultation worse than autonomous reasoning. We introduce BaRe-Mem, an online Bayesian reliability memory for multi-agent consultation. It estimates advisor reliability based on the central model's internal belief representations and updates these estimates from historical interactions. These estimates modulate the influence of advisor responses and guide the choice between consultation and autonomous reasoning. Across nine benchmarks and six central models, BaRe-Mem is more robust to misleading advisor information than debate and majority voting. On the more challenging tasks, it remains above autonomous reasoning across all tested misleading levels. Moreover, we extend the BaRe-Mem mechanism to worker allocation in agent teams. On the MuSiQue benchmark, BaRe-Mem improves task completion over routing by historical success counts and identifies capable workers earlier.
Figures & tables
Figure 1: Illustration of the BaRe-Mem reliability memory update process. (a) Without verified evidence, all advisors are assigned equal reliability. (b) Updating only candidate 1 with two correct outcomes shifts its estimated reliability to the red posterior, while a subsequent incorrect outcome yields the blue posterior. The annotated gains denote the corresponding Kalman gains.
Figure 2: Accuracy under increasing misleading-advice ratios for Qwen3-14B and Phi-4. Left: capability-supported regime; right: capability-challenging regime. Across both regimes, BaRe-Mem remains above the no-consultation baseline as misleading information increases, while other consultation methods degrade substantially in the capability-challenging regime.
| Capability-supported | Capability-challenging | ||||||||||
| Model | Method | 0% | 25% | 50% | 75% | 100% | 0% | 25% | 50% | 75% | 100% |
| Qwen3-14B | No consultation | 72.9 | 72.9 | 72.9 | 72.9 | 72.9 | 66.5 | 66.5 | 66.5 | 66.5 | 66.5 |
| Question + Peers | 76.6 | 75.9 | 75.3 | 73.9 | 73.2 | 69.8 | 65.6 | 60.4 | 55.2 | 50.6 | |
| BaRe-Mem | 77.7 | 77.3 | 76.0 | 75.1 | 74.5 | 76.1 | 72.8 | 71.6 | 69.9 | 68.6 | |
| Consultation ratio | 87% | 86% | 85% | 85% | 85% | 90% | 83% | 76% | 61% | 21% | |
| Phi-4 | No consultation | 70.2 | 70.2 | 70.2 | 70.2 | 70.2 | 63.4 | 63.4 | 63.4 | 63.4 | 63.4 |
Table 1: Accuracy (%) and consultation ratio under increasing misleading information ratios for Qwen3-14B and Phi-4. No consultation denotes autonomous reasoning without peer information, while Question + Peers denotes consultation with peer responses without reliability memory.
Figure 3: Estimated autonomous ability versus empirical autonomous accuracy for Qwen3-14B and Phi-4. Left: Evolution of mean and empirical autonomous accuracy along the question stream. Right: Empirical autonomous accuracy for groups of questions with similar values. The dashed diagonal indicates perfect calibration.
Figure 4: Predicted versus real consultation gain for Qwen3-14B and Phi-4 under increasing misleading-information ratios. Panels (a)–(e) correspond to misleading-information ratios of , , , , and , respectively. The -axis shows the mean predicted gain within each group, while the -axis shows the corresponding mean real gain .
Figure 5: BaRe-Mem accuracy under sparse verified feedback for Qwen3-14B and Phi-4. The fraction of questions whose verified outcomes are written to memory varies from to . Insets magnify the low-feedback regime below .
Figure 6: Agent-team task completion with Qwen3-14B and Phi-4 as lead agents. We compare random routing, routing by historical success counts, and BaRe-Mem under three verification settings: no verification, verification by the lead agent, and exact verification by the dataset evaluator.
Figure 7: Task completion as the lead agent is allowed to try an increasing number of workers for each sub-task. Subfigures (a) and (b) use Qwen3-14B and Phi-4 as the lead agent, respectively. The dashed horizontal line denotes the ceiling at which at least one worker can solve the sub-task.
Appendix figures & tables33 assets
Supplementary material from the paper’s appendix.
Appendix
| Condition | Behavior of | Decision |
| increases across zero | consult iff | |
| non-positive throughout | never consult | |
| non-negative throughout | always consult | |
| decreases across zero | consult iff |
| Capability-supported | GSM8K 1,319 | SQuAD 2,000 | APPS 1,000 |
| Advisor | |||
| Gemma-3-4B | 67.4% | 17.4% | 16.8% |
| Phi-4-mini | 56.9% | 85.9% | 19.4% |
| Qwen2.5-Coder-7B | 53.6% | 75.8% | 17.3% |
| Llama-3.1-8B | 81.3% | 68.7% | 12.1% |
| DeepSeek-Coder-V2-Lite | 81.4% | 20.1% | 26.6% |
Table 2: Accuracy (%) of the six advisor models and six central models in the capability-supported regime. Numbers below each dataset name indicate the number of evaluation examples. At least one right reports the fraction of examples for which at least one advisor provides a correct answer.
| Capability-challenging | PIQA 1,838 | MMLU 1,531 | OpenBookQA 500 | SciQ 1,000 | BBH 6,511 | SuperGLUE 6,023 |
| Advisor | ||||||
| Gemma-3-4B | 77.0% | 54.1% | 72.8% | 87.1% | 28.7% | 77.5% |
| Phi-4-mini | 80.1% | 64.1% | 76.4% | 90.1% | 10.7% | 79.1% |
| Qwen2.5-Coder-7B | 84.4% | 56.2% | 79.8% | 92.1% | 12.8% | 80.7% |
| Llama-3.1-8B | 82.2% | 61.2% | 81.6% | 93.9% | 27.6% | 77.5% |
| DeepSeek-Coder-V2-Lite | 73.2% | 23.4% | 38.2% | 32.7% | 3.7% | 65.1% |
Table 3: Accuracy (%) of the six advisor models and six central models in the capability-challenging regime. Numbers below each dataset name indicate the number of evaluation examples. At least one right reports the fraction of examples for which at least one advisor produces a correct answer.
Figure 8: Direct accuracy and gains from BaRe-Mem across central models. Left: capability-supported regime; right: capability-challenging regime. Blue bars show autonomous accuracy, red segments show the additional gain from BaRe-Mem, and green bars show individual advisor accuracy for reference. The dashed line denotes the fraction of questions for which at least one advisor is correct.
Figure 9: Distribution of autonomous reasoning and advisor attribution across central models under 0% misleading advice ratio. Autonomous reasoning denotes questions answered without consultation; all remaining questions are attributed to the advisor with the highest reliability estimate before feedback on the current question. Advisor shares therefore reflect reliability based ranking rather than exclusive use of a single advisor’s response.
Figure 10: Online reliability estimation for advisor Gemma-3-4B. Subfigures (a)–(e) correspond to misleading information ratios of , , , , and , respectively. Solid curves show the pre-feedback reliability estimate produced by Qwen3-14B and Phi-4, while the dashed curve shows the empirical accuracy of Gemma-3-4B.
Figure 11: Online reliability estimation for advisor Phi-4-mini. Subfigures (a)–(e) correspond to misleading information ratios of , , , , and , respectively. Solid curves show the pre-feedback reliability estimate produced by Qwen3-14B and Phi-4, while the dashed curve shows the empirical accuracy of Phi-4-mini.
Figure 12: Online reliability estimation for advisor Qwen2.5-Coder-7B. Subfigures (a)–(e) correspond to misleading information ratios of , , , , and , respectively. Solid curves show the pre-feedback reliability estimate produced by Qwen3-14B and Phi-4, while the dashed curve shows the empirical accuracy of Qwen2.5-Coder-7B.
Figure 13: Online reliability estimation for advisor Llama-3.1-8B. Subfigures (a)–(e) correspond to misleading information ratios of , , , , and , respectively. Solid curves show the pre-feedback reliability estimate produced by Qwen3-14B and Phi-4, while the dashed curve shows the empirical accuracy of Llama-3.1-8B.
Figure 14: Online reliability estimation for advisor DeepSeek-Coder-V2-Lite. Subfigures (a)–(e) correspond to misleading information ratios of , , , , and , respectively. Solid curves show the pre-feedback reliability estimate produced by Qwen3-14B and Phi-4, while the dashed curve shows the empirical accuracy of DeepSeek-Coder-V2-Lite.
Figure 15: Online reliability estimation for advisor R1-Distill-Qwen-7B. Subfigures (a)–(e) correspond to misleading information ratios of , , , , and , respectively. Solid curves show the pre-feedback reliability estimate produced by Qwen3-14B and Phi-4, while the dashed curve shows the empirical accuracy of R1-Distill-Qwen-7B.
| Method | Capability-supported | Capability-challenging | ||||||||
| 0% | 25% | 50% | 75% | 100% | 0% | 25% | 50% | 75% | 100% | |
| No consultation | 69.2 | 69.2 | 69.2 | 69.2 | 69.2 | 67.8 | 67.8 | 67.8 | 67.8 | 67.8 |
| Question + Peers | 74.6 | 74.2 | 73.7 | 72.4 | 71.3 | 69.2 | 64.0 | 58.1 | 51.4 | 45.8 |
| Majority vote (advisors) | 64.3 | 53.0 | 32.5 | 14.6 | 9.4 | 64.1 | 48.7 | 24.5 | 5.9 | 1.2 |
| Majority vote (advisors + own) | 68.0 | 61.8 | 46.9 | 27.3 | 18.8 | 68.2 | 56.8 | 34.3 | 10.4 | 3.0 |
| Debate (2 rounds) | 73.5 | 73.2 | 73.0 | 72.3 | 72.1 | 71.4 | 68.6 | 64.6 | 60.0 | 56.1 |
Table 4: Accuracy (%) of consultation methods with Qwen3-4B as the central model under increasing misleading information ratios. Results are reported for the capability-supported and capability-challenging regimes. Bold numbers indicate the best performance in each column.
Figure 16: Accuracy of Qwen3-4B as the central model under increasing misleading information ratios. Left: capability-supported regime; right: capability-challenging regime. We compare BaRe-Mem with Advisors + memory , No consultation , Question + Peers , Debate (2 rounds) , and two majority voting baselines.
| Method | Capability-supported | Capability-challenging | ||||||||
| 0% | 25% | 50% | 75% | 100% | 0% | 25% | 50% | 75% | 100% | |
| No consultation | 72.9 | 72.9 | 72.9 | 72.9 | 72.9 | 65.6 | 65.6 | 65.6 | 65.6 | 65.6 |
| Question + Peers | 75.7 | 75.1 | 74.3 | 73.6 | 72.3 | 67.9 | 61.6 | 54.3 | 46.7 | 40.0 |
| Majority vote (advisors) | 64.3 | 53.0 | 32.5 | 14.6 | 9.4 | 64.1 | 48.7 | 24.5 | 5.9 | 1.2 |
| Majority vote (advisors + own) | 68.9 | 62.7 | 47.6 | 27.9 | 19.3 | 66.5 | 55.5 | 33.3 | 9.7 | 2.7 |
| Debate (2 rounds) | 74.8 | 74.8 | 74.6 | 74.4 | 73.8 | 67.0 | 63.9 | 60.3 | 55.7 | 51.3 |
Table 5: Accuracy (%) of consultation methods with Qwen3-8B as the central model under increasing misleading information ratios. Results are reported for the capability-supported and capability-challenging regimes. Bold numbers indicate the best performance in each column.
Figure 17: Accuracy of Qwen3-8B as the central model under increasing misleading information ratios. Left: capability-supported regime; right: capability-challenging regime. We compare BaRe-Mem with Advisors + memory , No consultation , Question + Peers , Debate (2 rounds) , and two majority voting baselines.
| Method | Capability-supported | Capability-challenging | ||||||||
| 0% | 25% | 50% | 75% | 100% | 0% | 25% | 50% | 75% | 100% | |
| No consultation | 72.9 | 72.9 | 72.9 | 72.9 | 72.9 | 66.5 | 66.5 | 66.5 | 66.5 | 66.5 |
| Question + Peers | 76.6 | 75.9 | 75.3 | 73.9 | 73.2 | 69.8 | 65.6 | 60.4 | 55.2 | 50.6 |
| Majority vote (advisors) | 64.3 | 53.0 | 32.5 | 14.6 | 9.4 | 64.1 | 48.7 | 24.5 | 5.9 | 1.2 |
| Majority vote (advisors + own) | 68.5 | 62.6 | 47.5 | 27.8 | 19.4 | 66.4 | 55.8 | 33.6 | 9.9 | 2.8 |
| Debate (2 rounds) | 74.9 | 74.7 | 73.9 | 74.0 | 73.4 | 69.2 | 66.7 | 62.6 | 58.9 | 55.4 |
Table 6: Accuracy (%) of consultation methods with Qwen3-14B as the central model under increasing misleading information ratios. Results are reported for the capability-supported and capability-challenging regimes. Bold numbers indicate the best performance in each column.
Figure 18: Accuracy of Qwen3-14B as the central model under increasing misleading information ratios. Left: capability-supported regime; right: capability-challenging regime. We compare BaRe-Mem with Advisors + memory , No consultation , Question + Peers , Debate (2 rounds) , and two majority voting baselines.
| Method | Capability-supported | Capability-challenging | ||||||||
| 0% | 25% | 50% | 75% | 100% | 0% | 25% | 50% | 75% | 100% | |
| No consultation | 70.6 | 70.6 | 70.6 | 70.6 | 70.6 | 59.1 | 59.1 | 59.1 | 59.1 | 59.1 |
| Question + Peers | 71.3 | 70.3 | 69.2 | 67.8 | 67.1 | 65.9 | 59.3 | 52.9 | 47.3 | 43.3 |
| Majority vote (advisors) | 64.3 | 53.0 | 32.5 | 14.6 | 9.4 | 64.1 | 48.7 | 24.5 | 5.9 | 1.2 |
| Majority vote (advisors + own) | 68.4 | 62.3 | 47.1 | 27.3 | 18.9 | 64.7 | 53.5 | 31.5 | 8.8 | 2.2 |
| Debate (2 rounds) | 70.5 | 70.3 | 69.9 | 69.2 | 68.2 | 63.1 | 58.6 | 53.7 | 49.1 | 45.5 |
Table 7: Accuracy (%) of consultation methods with Qwen2.5-7B as the central model under increasing misleading information ratios. Results are reported for the capability-supported and capability-challenging regimes. Bold numbers indicate the best performance in each column.
Figure 19: Accuracy of Qwen2.5-7B as the central model under increasing misleading information ratios. Left: capability-supported regime; right: capability-challenging regime. We compare BaRe-Mem with Advisors + memory , No consultation , Question + Peers , Debate (2 rounds) , and two majority voting baselines.
| Method | Capability-supported | Capability-challenging | ||||||||
| 0% | 25% | 50% | 75% | 100% | 0% | 25% | 50% | 75% | 100% | |
| No consultation | 56.5 | 56.5 | 56.5 | 56.5 | 56.5 | 58.6 | 58.6 | 58.6 | 58.6 | 58.6 |
| Question + Peers | 64.2 | 63.1 | 61.1 | 58.5 | 56.9 | 64.0 | 54.8 | 41.6 | 27.4 | 18.0 |
| Majority vote (advisors) | 64.3 | 53.0 | 32.5 | 14.6 | 9.4 | 64.1 | 48.7 | 24.5 | 5.9 | 1.2 |
| Majority vote (advisors + own) | 67.1 | 59.7 | 43.6 | 24.9 | 17.1 | 64.2 | 53.3 | 31.5 | 8.9 | 2.2 |
| Debate (2 rounds) | 61.8 | 60.8 | 59.0 | 57.0 | 56.7 | 61.5 | 56.5 | 49.7 | 41.3 | 34.4 |
Table 8: Accuracy (%) of consultation methods with Ministral-8B as the central model under increasing misleading information ratios. Results are reported for the capability-supported and capability-challenging regimes. Bold numbers indicate the best performance in each column.
Figure 20: Accuracy under increasing misleading-advice ratios with Ministral-8B as the central model. The left and right panels show the capability-supported and capability-challenging regimes, respectively. We compare BaRe-Mem, Advisors + memory , No consultation , Question + Peers , Debate (2 rounds) , and two majority-vote variants.
| Method | Capability-supported | Capability-challenging | ||||||||
| 0% | 25% | 50% | 75% | 100% | 0% | 25% | 50% | 75% | 100% | |
| No consultation | 70.2 | 70.2 | 70.2 | 70.2 | 70.2 | 63.4 | 63.4 | 63.4 | 63.4 | 63.4 |
| Question + Peers | 71.5 | 72.1 | 71.6 | 70.8 | 69.9 | 65.8 | 62.4 | 57.9 | 52.5 | 48.3 |
| Majority vote (advisors) | 64.3 | 53.0 | 32.5 | 14.6 | 9.4 | 64.1 | 48.7 | 24.5 | 5.9 | 1.2 |
| Majority vote (advisors + own) | 67.9 | 61.5 | 46.3 | 26.7 | 18.3 | 65.1 | 54.3 | 32.1 | 9.0 | 2.3 |
| Debate (2 rounds) | 68.0 | 68.8 | 69.4 | 69.6 | 68.6 | 64.8 | 62.6 | 59.3 | 55.2 | 51.4 |
Table 9: Accuracy (%) of consultation methods with Phi-4 as the central model under increasing misleading information ratios. Results are reported for the capability-supported and capability-challenging regimes. Bold numbers indicate the best performance in each column.
Figure 21: Accuracy of Phi-4 as the central model under increasing misleading information ratios. Left: capability-supported regime; right: capability-challenging regime. We compare BaRe-Mem with Advisors + memory , No consultation , Question + Peers , Debate (2 rounds) , and two majority voting baselines.
Figure 22: Agent team pipeline with BaRe-Mem. The lead agent decomposes a task into sub-tasks and queries the reliability memory to rank candidate workers. Each returned report is verified by the lead agent or an external verifier: accepted reports are committed, while rejected reports trigger the next candidate. Verification outcomes are then written back to update advisor reliability.
Figure 23: Agent team performance on MuSiQue with Qwen3-4B as the lead agent. (a) Task completion; (b) sub-task completion. We compare random routing, routing by historical success counts, and BaRe-Mem under three report checking settings: no check, lead agent check, and dataset verifier check. All settings provide gold feedback for queried reports after each sub-task is committed.
Figure 24: Agent team performance on MuSiQue with Qwen3-8B as the lead agent. (a) Task completion; (b) sub-task completion. We compare random routing, routing by historical success counts, and BaRe-Mem under three report checking settings: no check, lead agent check, and dataset verifier check. All settings provide gold feedback for queried reports after each sub-task is committed.
Figure 25: Agent team performance on MuSiQue with Qwen3-14B as the lead agent. (a) Task completion; (b) sub-task completion. We compare random routing, routing by historical success counts, and BaRe-Mem under three report checking settings: no check, lead agent check, and dataset verifier check. All settings provide gold feedback for queried reports after each sub-task is committed.
Figure 26: Agent team performance on MuSiQue with Qwen2.5-7B as the lead agent. (a) Task completion; (b) sub-task completion. We compare random routing, routing by historical success counts, and BaRe-Mem under three report checking settings: no check, lead agent check, and dataset verifier check. All settings provide gold feedback for queried reports after each sub-task is committed.
Figure 27: Agent team performance on MuSiQue with Phi-4 as the lead agent. (a) Task completion; (b) sub-task completion. We compare random routing, routing by historical success counts, and BaRe-Mem under three report checking settings: no check, lead agent check, and dataset verifier check. All settings provide gold feedback for queried reports after each sub-task is committed.
Figure 28: Agent team performance on MuSiQue with Ministral-8B as the lead agent. (a) Task completion; (b) sub-task completion. We compare random routing, routing by historical success counts, and BaRe-Mem under three report checking settings: no check, lead agent check, and dataset verifier check. All settings provide gold feedback for queried reports after each sub-task is committed.
Figure 29: Task completion on MuSiQue as the maximum number of workers tried per sub-task increases. We compare BaRe-Mem, routing by historical success counts, and random routing. The dashed line marks the ceiling obtained when at least one worker can solve the sub-task. Panels (a)–(f) use Qwen3-4B, Qwen3-8B, Qwen3-14B, Qwen2.5-7B, Ministral-8B, and Phi-4 as the lead agent, respectively.
Figure 30: Sub-task completion on MuSiQue as the maximum number of workers tried per sub-task increases. We compare BaRe-Mem, routing by historical success counts, and random routing. The dashed line marks the ceiling given by the fraction of sub-tasks for which at least one worker is correct. Panels (a)–(f) use Qwen3-4B, Qwen3-8B, Qwen3-14B, Qwen2.5-7B, Ministral-8B, and Phi-4 as the lead agent, respectively.
Figure 31: Initial worker accuracy over the MuSiQue sub-task stream. The x-axis denotes position in the sub-task stream, grouped into eight consecutive blocks of approximately 800 sub-tasks each. We compare BaRe-Mem, routing by historical success counts, and random routing. Panels (a)–(f) use Qwen3-4B, Qwen3-8B, Qwen3-14B, Qwen2.5-7B, Ministral-8B, and Phi-4 as the lead agent, respectively.