LG-Bench: A Graph-Structured Evaluation Benchmark for Life Sciences
Authors: Lu Sun, Xiangyi Zhang, Xiangyang Zhu, Kaiyuan Ji, Zijian Chen, Qiang Hu, Zicheng Zhang, Yuan Tian, Guangtao Zhai
Organizations: Shanghai Artificial Intelligence Laboratory · SJTU-Paris Elite Institute of Technology, Shanghai Jiao Tong University · School of Information and Electronic Engineering, East China Normal University · Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University · Cooperative Medianet Innovation Center, Shanghai Jiao Tong University
Traditional evaluation benchmarks reduce inherently interconnected scientific knowledge in life sciences into flat lists of questions, disregarding the underlying topological structure of the knowledge. We introduce, the first graph-structured benchmark for life sciences, featuring over 10,000 high-quality multiple-choice questions across medicine, biology, and chemistry. Our approach constructs a weighted evaluation graph using bidirectional matching and semantic similarity algorithms, where nodes represent questions and edge weights capture their semantic relationships. Leveraging this graph topology, we design two novel evaluation metrics. The Global Coherence Score (GCS) measures a model's consistency within semantically related neighborhoods, while Knowledge Balance Score (KBS) analyzes how model errors are distributed across the graph to reveal conceptual blind spots. LG-Bench facilitates fine-grained comparison of LLMs by surfacing differences in conceptual coherence and patterns of knowledge organization across models. Our framework shifts the evaluation paradigm from flat accuracy metrics to structure-aware analysis, offering a new lens for diagnosing and improving LLM performance in the life sciences domain.
Figures & tables
Figure 1: Case study comparing LG-Bench with traditional benchmarks. Unlike traditional list-based approaches that overlook semantic connections, LG-Bench utilizes a graph structure to capture these links, effectively revealing LLM inconsistencies on related questions.
Figure 2: Overview diagram of LG-Bench. Our LG-Bench extracts over 10,000 questions from top-tier journals in the life sciences, covering three subfields: Medicine, Biology, and Chemistry. It innovatively introduces graph structures into the benchmark to better capture the nuanced capabilities of large language models.
Figure 3: Two-stage Pipeline for Construction of LG-Bench.
Figure 4: Qwen2.5 results within a local graph structure. Qwen2.5-7B failed on many questions in the neighborhood-level evaluation. Red keywords indicate areas of severe knowledge weakness, while blue keywords denote relatively weak points. In contrast, Qwen2.5-72B correctly answered all relevant questions in the neighborhood test, demonstrating a relatively good grasp of the associated knowledge.
Model
Medicine
Biology
Chemistry
Overall
Acc
GCS
KBS
Acc
GCS
KBS
Acc
GCS
KBS
Acc
GCS
KBS
Open-Weight LLM (Scale<10B)
Qwen2.5-0.5B
37.07
13.96
22.19
38.10
14.83
26.34
37.44
14.35
36.52
37.56
14.39
16.11
Gemma-3-1b
51.56
26.73
29.42
55.45
30.71
21.97
52.65
28.20
39.34
53.30
28.62
17.94
BioMedLM-2.7B
24.14
5.80
12.85
23.80
5.65
9.03
23.87
5.56
13.69
23.93
5.67
5.28
Llama-7B
30.06
9.19
13.62
30.91
9.60
14.41
32.20
10.47
19.14
31.07
9.76
6.80
Table 2: Evaluation Results on LG-Bench: Acc denotes traditional accuracy (in %), GCS measures global coherence (in %), with higher scores indicating stronger capabilities. We use bold to highlight the best-performing model in each domain, and italics to indicate the second-best. KBS quantifies knowledge balance, and higher scores indicate a more uneven distribution. For hyperparameter settings, we set γ=0.6 and θ=0.4 .
Figure 7
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Stage
Description
Count
Removed
Stage 1
Initially generated questions
13,250
-
Stage 2
Multi-Model Committee Selection
10,597
2,653
Stage 3
Expert Review
10,200
397
Appendix
Table 4: Statistics of the Data Construction Pipeline. The table tracks the number of questions retained and removed at each stage of the quality control process.
Error Type
Number
Percentage
Low relevance to the domain
214
53.9%
Topics not suitable for evaluation
136
34.3%
Logical or factual errors
47
11.8%
Total Rejected
397
100.0%
Appendix
Table 5: Distribution of Error Types in Expert Validation. Analysis of the 397 questions rejected during the final human review stage.
Figure 7: Visualization of a subset of LG-Bench, where red nodes represent sampled questions and their top-5 semantic neighbors (orange) based on edge weights. All other nodes and edges are displayed in gray to show overall connectivity. The blue edges indicate the strongest semantic connections (top-5 weighted edges) for each sampled node.
State Key Laboratory for Multimedia Information Processing, School of Computer Science, PKU-Anker LLM Lab, Peking University · IDEA · Zhongguancun Academy +5