Organizations: Artificial Intelligence Innovation and Incubation Institute, Fudan University · Shanghai Innovation Institute · Shanghai Academy of AI for Science
Large Language Models (LLMs) demonstrate potentials for automating scientific code generation but face challenges in reliability, error propagation in multi-agent workflows, and evaluation in domains with ill-defined success metrics. We present a Bayesian adversarial multi-agent framework specifically designed for AI for Science (AI4S) tasks in the form of a Low-code Platform (LCP). Three LLM-based agents are coordinated under the Bayesian framework: a Task Manager that structures user inputs into actionable plans and adaptive test cases, a Code Generator that produces candidate solutions, and an Evaluator providing comprehensive feedback. The framework employs an adversarial loop where the Task Manager iteratively refines test cases to challenge the Code Generator, while prompt distributions are dynamically updated using Bayesian principles by integrating code quality metrics: functional correctness, structural alignment, and static analysis. This co-optimization of tests and code reduces dependence on LLM reliability and addresses evaluation uncertainty inherent to scientific tasks. LCP also streamlines human-AI collaboration by translating non-expert prompts into domain-specific requirements, bypassing the need for manual prompt engineering by practitioners without coding backgrounds. Benchmark evaluations demonstrate LCP's effectiveness in generating robust code while minimizing error propagation. The proposed platform is also tested on an Earth Science cross-disciplinary task and demonstrates strong reliability, outperforming competing models.
Figures & tables
Figure 1: Comparison between three code generation paradigms: Single LLM generator, multi-agent role playing and the proposed Bayesian adversarial multi-agent framework.
Figure 2: Overview of the Bayesian adversarial multi-agent framework. The three red arrows indicate fusion of the user-approved plan, test cases, and codes into prompts, the distribution of which is recursively updated under the Bayesian framework. S1 , S2 and S3 are the scores computed in equation 2 , equation 3 , and equation 4 . Loop 1-3 indicate three iterative updating processes for plan, test cases, and codes, respectively. The dashed arrows indicate latent relationships (e.g., S3 likelihood score) or steps conducted before or after the main algorithm execution.
Without Knowledge
With Knowledge
Without Knowledge
With Knowledge
Model
Method
Sub (%)
Main (%)
Sub (%)
Main (%)
Model
Method
Sub (%)
Main (%)
Sub (%)
Main (%)
Qwen3-8b
Baseline
13.2
0
19.8
1.5
GPT-4o
Baseline
24.1
1.5
33.7
7.7
Ours
24.7(87.1%)
4.6
27.4(38.4%)
4.6
Ours
37.2(54.3%)
7.7
40.6(20.4%)
10.8
Qwen3-14b
Baseline
17.7
1.5
25.0
6.2
Deepseek-v3
Baseline
27.8
3.1
38.8
10.8
Ours
30.6(72.9%)
6.2
32.6(30.4%)
6.2
Ours
40.3(45.0%)
10.8
42.4(9.28%)
12.3
Qwen3-32b
Baseline
18.4
0
27.4
7.7
Deepseek-R1
Baseline
29.6
4.6
37.8
10.8
Table 1: Model Performance Comparison on SciCode with various backbone models
Method
SR(w/o)
CBS(w/o)
VER(w/o)
SR(w/)
CBS(w/)
VER(w/)
Direct
11.8
82.6
52.9
10.8
83.8
41.2
OpenHands CodeAct
19.6
83.1
78.4
27.5
86.3
73.5
Self-Debug
22.6
84.4
83.3
23.5
85.6
71.6
LCP(Ours)
26.5
85.1
90.2
27.5
86.4
87.3
Table 2: Results on ScienceAgentBench using GPT-4o as base model with/without prior knowledge, compare with two different agent frameworks and baseline.
Figure 3: Illustration of LCP performance over: (a) different iteration number with and without ATC component on general code benchmark; (b) difficulty iteration number on the SciCode benchmark
Figure 4: Model performance with basic vs. expert-crafted prompts. Our framework (blue/green lines) is significantly more robust to prompt quality than the baseline (red lines), showing a much smaller performance gap (shaded area) and achieving superior results even without expert knowledge.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Models
HumanEval
HumanEval-ET
MBPP
MBPP-ET
Foundation Models (Zero-Shot)
Incoder (6.7B)
15.2
11.6
17.6
14.3
CodeLlama (34B)
51.8
-
69.3
-
GPT-3.5-turbo
57.3
42.7
52.2
36.8
Claude-instant-1
31.1
28.1
26.9
19.9
GPT-4-turbo
57.9
48.8
63.4
47.5
Appendix
Table 3: Pass@1 score comparison of various competing methods
Difficulty Level
Baseline
LDB
LPW
LCP (Ours)
Introductory
63.8
78.7
87.2
92.1
Interview
43.5
52.2
65.2
77.5
Competition
17.4
28.3
34.8
38.0
Appendix
Table 4: Pass@1 accuracy of multi-agent framework (LPW) compared with baseline and LDB on APPS benchmark across different difficulty levels using GPT-4o as the LLM backbone.