HypoForge: A Self-Improving Multi-Agent Framework for Automated Hypothesis Generation and Testing via Scientific Skill Learning
Authors: Ziqing Qian, Jiaying Lei, Yifang Wang, Nan Cao
Organizations: Intelligent Big Data Visualization Lab, Tongji University, Shanghai, China · Shanghai Research Institute for Intelligent Autonomous System, Tongji University, Shanghai, China · Shanghai Innovation Institute, Shanghai, China · Florida State University Tallahassee, Florida, USA
Large language models (LLMs) have enabled AI scientist systems to automate scientific discovery, yet existing approaches most rely on static prompting or fixed workflows and fail to accumulate experience for continual improvement. We propose HypoForge, an experience-guided multi-agent framework that learns reusable scientific skills for automated hypothesis generation and hypothesis testing. HypoForge is built on the observation that these two stages involve different supervision signals. For hypothesis generation, where explicit feedback is unavailable, HypoForge adopts an adversarial generator--discriminator mechanism to improve reasoning through comparative critique. For hypothesis testing, where empirical feedback is available, HypoForge learns testing skills from execution outcomes and ground-truth results. By matching skill learning strategies with stage-specific supervision, HypoForge enables continual improvement without fine-tuning foundation models. Experiments on hypothesis generation and testing benchmarks show that HypoForge consistently outperforms existing AI scientist frameworks and skill-level variants. Further analysis demonstrates the effectiveness of the proposed stage-specific skill learning paradigms.
Figures & tables
Figure 1: Multi-agent framework that supports hypothesis generation and testing with skill learning.
Task
Type
#Data Size
Deceptive Reviews Detection
Real
800
Dreddit Mental Stress Detection
Real
200
AI-Generated Content Detection
Real
200
News Headline Engagement
Real
200
Persuasive Argument Prediction
Real
200
Retweet Prediction
Real
200
Table 1: HypoBench tasks used in our experiments.
Method
Q(Ht)↑
Hit@K↑
HypoGeniC
0.719 ± 0.014
0.472 ± 0.050
HypotheSAEs
0.582 ± 0.043
0.208 ± 0.038
LLM-ZeroShot
0.701 ± 0.006
0.516 ± 0.029
LLM-FewShot
0.719 ± 0.007
0.522 ± 0.022
No-Skill
0.724 ± 0.040
0.409 ± 0.079
AI-Generated-Skill
0.738 ± 0.050
0.579 ± 0.022
Table 2: Hypothesis generation performance.
Method
Th↑
Eh↑
POPPER
0.417 ± 0.141
0.984 ± 0.000
ReAct
0.589 ± 0.078
0.859 ± 0.051
LLM-ZeroShot
0.516 ± 0.014
0.870 ± 0.030
LLM-FewShot
0.539 ± 0.008
0.966 ± 0.005
No-Skill
0.562 ± 0.055
0.885 ± 0.099
AI-Generated-Skill
0.555 ± 0.090
0.891 ± 0.131
Table 3: Hypothesis testing performance on test tasks.
Figure 2: Skill learning progress across refinement iterations.