BioAlchemy: Distilling Biological Literature into Reasoning-Ready Reinforcement Learning Training Data

Brian Hsu; Ozan Gökdemir; Carlo Siebenschuh; Bruce Parrello; Neil Getty; Thomas S. Brettin; Rick L. Stevens; Ian T. Foster

cs.AIApr 3, 2026

BioAlchemy: Distilling Biological Literature into Reasoning-Ready Reinforcement Learning Training Data

Authors: Brian Hsu, Ozan Gökdemir, Carlo Siebenschuh, Bruce Parrello, Neil Getty, Thomas S. Brettin, Rick L. Stevens, Ian T. Foster, +2 more

Organizations: Department of Computer Science, University of Chicago, Chicago, IL 60637 · Data Science and Learning Division, Argonne National Laboratory, Lemont, IL 60439

Abstract

Despite the large corpus of biology training text, the impact of reasoning models on biological research generally lags behind math and coding. In this work, we show that biology questions from current large-scale reasoning datasets do not align well with modern research topic distributions in biology, and that this topic imbalance may negatively affect performance. In addition, we find that methods for extracting challenging and verifiable research problems from biology research text are a critical yet underdeveloped ingredient in applying reinforcement learning for better performance on biology research tasks. We introduce BioAlchemy, a pipeline for sourcing a diverse set of verifiable question-and-answer pairs from a scientific corpus of biology research text. We curate BioAlchemy-345K, a training dataset containing over 345K scientific reasoning problems in biology. Then, we demonstrate how aligning our dataset to the topic distribution of modern scientific biology can be used with reinforcement learning to improve reasoning performance. Finally, we present BioAlchemist-8B, which improves over its base reasoning model by 9.12% on biology benchmarks. These results demonstrate the efficacy of our approach for developing stronger scientific reasoning capabilities in biology. The BioAlchemist-8B model is available at: https://huggingface.co/BioAlchemy.

Figures & tables

Figure 1: Content distribution differences between reasoning problems in biology. Comparison is based on MeSH topics representing PubMed research articles from 2020–2024.

Data Source	# Research Articles	# QA Pairs
ASM	35.8K	173.6K
PLoS CompBio	10.5K	76.1K
PLoS Genetics	8.9K	56.9K
Semantic Scholar	22.5K	38.4K
Total	77.7K	345K

Table 1: Composition of dataset for BioAlchemy.

Dataset	Source	Size (Biology)	Training	MeSH Coverage
BioAlchemy	Scientific Research Text	345K	RL (GRPO)	95.6%
NaturalReasoning	Pretraining Corpora	52.9K	SFT	34.7%
TextbookReasoning	University Textbooks	51.2K	SFT	66.6%
Nemotron-Science-v1	N/A	29.1K	RL (GRPO)	91.4%

Table 2: Comparison of reasoning datasets, where dataset sizes are based on questions classified as biology questions. BioAlchemy is the largest and has the strongest coverage.

Figure 2: BioAlchemy has more MeSH topics per biology question. Comparison is based on 50K subsamples.

Rank	MeSH Biology Subcategory	PubMed 2020–2024	Textbook Reasoning	Natural Reasoning	Nemotron Science-v1	BioAlchemy
1	Computational Biology	45.32%	0.57%	12.09%	7.44%	24.95%
2	Genetics	36.18%	11.37%	18.30%	34.67%	36.89%
3	Microbiology	10.90%	14.39%	7.19%	3.78%	16.75%
4	Ecology	2.56%	2.51%	9.24%	0.99%	2.80%
5	Molecular Biology	1.88%	12.59%	8.45%	40.41%	6.63%
6	Synthetic Biology	0.97%	0.04%	0.63%	1.76%	0.50%

Table 3: BioAlchemy has better distributional alignment at the subtopic level compared to the biology subsets of TextbookReasoning, NaturalReasoning, and Nemotron-Science-v1. Percentages are not mutually exclusive as publications/QA pairs can be indexed under multiple MeSH terms.

Data Source	Mean MeSH Topics	% Non-Zero MeSH Topics	% With $\geq 2$ MeSH Topics
NaturalReasoning	0.41	34.7%	6.2%
TextbookReasoning	0.77	66.6%	10.1%
Nemotron-Science-v1	1.19	91.4%	28.0%
BioAlchemy	1.37	95.6%	40.8%

Table 4: BioAlchemy has higher MeSH topic coverage and higher average number of topics. The average number of MeSH subcategories for PubMed publications is 1.07.

Dataset	Size	Topic Max/Min Ratio	Avg. Upsample †	Sample Yield	TVD
Full Dataset $\mathcal{D}$	345K	9,301 $\times$	11.8 $\times$	100%	0.52
Dataset $\mathcal{S}_{\alpha=1.0}$	126.7K	7,932 $\times$	1.0 $\times$	36.73%	0.21
Dataset $\mathcal{S}_{\alpha=0.7}$	146.2K	536 $\times$	9.7 $\times$	42.38%	0.25
Dataset $\mathcal{S}_{\alpha=0.5}$	152.0K	89 $\times$	39.6 $\times$	44.06%	0.27
Dataset $\mathcal{S}_{\alpha=0.3}$	83.2K	15 $\times$	134.3 $\times$	24.10%	0.39

Table 5: Sub-sampling BioAlchemy-345K with increasing representation of low-frequency topics from exponential smoothing.

\mathcal{S}_{\alpha=0.5}

provides the best tradeoff in terms of dataset size, distributional alignment, and upsampling of rare topics.

Benchmark	SFT			RL
	Natural Reasoning	Textbook Reasoning	BioAlchemy	Nemotron- Science-v1	BioAlchemy
ProtocolQA	43.61 $\pm$ 2.55%	43.33 $\pm$ 2.89%	43.80 $\pm$ 2.10%	44.44 $\pm$ 2.09%	45.83 $\pm$ 2.32%
SeqQA	10.43 $\pm$ 0.75%	15.20 $\pm$ 0.73%	14.03 $\pm$ 0.69%	20.42 $\pm$ 0.66%	18.73 $\pm$ 0.85%
Cloning Scenarios	8.48 $\pm$ 1.99%	8.18 $\pm$ 2.30%	9.70 $\pm$ 2.85%	13.94 $\pm$ 3.85%	11.52 $\pm$ 3.03%
PubMedQA	67.89 $\pm$ 0.60%	72.47 $\pm$ 0.15%	69.44 $\pm$ 0.24%	69.21 $\pm$ 0.38%	68.09 $\pm$ 0.28%
GPQA-Bio	36.84 $\pm$ 0.00%	42.11 $\pm$ 0.00%	44.21 $\pm$ 3.64%	47.89 $\pm$ 1.19%	57.89 $\pm$ 0.00%

Table 6: Comparison of reasoning datasets across relevant training strategies. BioAlchemy with GRPO performs best overall.

Dataset	Alignment	Upsampling	ProtocolQA	SeqQA	Cloning Scenarios	PubMedQA	GPQA-Bio	General Avg.
Baseline	N/A	N/A	42.69 $\pm$ 2.06%	8.42 $\pm$ 0.43%	5.76 $\pm$ 2.16%	69.00 $\pm$ 0.35%	42.63 $\pm$ 1.19%	33.70 $\pm$ 0.65%
$\bm{\mathcal{S}_{\alpha=0.3}}$	Weakest	Strongest	46.30 $\pm$ 1.25%	19.63 $\pm$ 0.98%	12.12 $\pm$ 3.96%	67.75 $\pm$ 0.28%	53.16 $\pm$ 2.78%	39.79 $\pm$ 1.02%
$\bm{\mathcal{S}_{\alpha=0.5}}$	Weak	Strong	45.83 $\pm$ 2.32%	18.73 $\pm$ 0.85%	11.52 $\pm$ 3.03%	68.09 $\pm$ 0.28%	57.89 $\pm$ 0.00%	40.41 $\pm$ 0.78%
$\bm{\mathcal{S}_{\alpha=0.7}}$	Strong	Weak	44.63 $\pm$ 1.97%	20.13 $\pm$ 0.75%	11.82 $\pm$ 3.14%	68.65 $\pm$ 0.51%	51.58 $\pm$ 2.38%	39.36 $\pm$ 0.90%
$\bm{\mathcal{S}_{\alpha=1.0}}$	Strongest	Weakest	45.93 $\pm$ 1.60%	19.98 $\pm$ 0.73%	10.91 $\pm$ 3.57%	68.83 $\pm$ 0.30%	63.16 $\pm$ 0.00%	41.76 $\pm$ 0.80%
Best	$\alpha=1.0$	$\alpha=0.3$	46.30 $\pm$ 1.25%	20.13 $\pm$ 0.75%	12.12 $\pm$ 3.96%	69.00 $\pm$ 0.35%	63.16 $\pm$ 0.00%	41.76 $\pm$ 0.80%

Table 7: Accuracy from training on datasets with stronger tail categories. Strongest alignment with

P_{\text{Bio}}

leads to best performance. Baseline represents base Qwen3-8B results.

Model	ProtocolQA	SeqQA	Cloning Scenarios	PubMedQA	GPQA-Bio	Overall Avg.
BioAlchemist-8B 150K	46.20 $\pm$ 1.89%	22.32 $\pm$ 0.91%	15.15 $\pm$ 4.57%	68.32 $\pm$ 0.48%	62.11 $\pm$ 2.38%	42.82 $\pm$ 1.12%
BioAlchemist-8B 50K	45.83 $\pm$ 2.32%	18.73 $\pm$ 0.85%	11.52 $\pm$ 3.03%	68.09 $\pm$ 0.28%	57.89 $\pm$ 0.00%	40.41 $\pm$ 0.78%
Qwen3-8B	42.69 $\pm$ 2.06%	8.42 $\pm$ 0.43%	5.76 $\pm$ 2.16%	69.00 $\pm$ 0.35%	42.63 $\pm$ 1.19%	33.70 $\pm$ 0.65%
DeepSeek-R1-Llama-8B	33.61 $\pm$ 2.58%	4.97 $\pm$ 0.41%	10.91 $\pm$ 2.09%	26.89 $\pm$ 0.58%	5.26 $\pm$ 0.00%	16.33 $\pm$ 0.68%
GPT-OSS-20B	52.78 $\pm$ 2.25%	18.12 $\pm$ 0.79%	7.27 $\pm$ 1.83%	72.79 $\pm$ 0.24%	55.79 $\pm$ 3.17%	41.35 $\pm$ 0.88%
Best	52.78 $\pm$ 2.25%	22.32 $\pm$ 0.91%	15.15 $\pm$ 4.57%	72.79 $\pm$ 0.24%	62.11 $\pm$ 2.38%	42.82 $\pm$ 1.12%

Table 8: Final RL model performance across benchmarks. BioAlchemy-8B 150K performs best, and scales with more data.

Appendix figures & tables23 assets

Supplementary material from the paper’s appendix.

Appendix

Training	Rollouts	Max Response Len	Temperature	Clip Ratio Range	Normalize Adv.	Entropy Coef.	Loss Aggr.
GRPO	5	4096	0.6	[0.2, 0.28]	False	0.0005	Token-mean

Appendix

Table 9: GRPO-specific training configurations used across all experiments. Settings are applied using the VeRL framework.

Dataset	LR	LR-Scheduler	Batch Size	Epochs	Max Tokens	Loss	Dataset Size	Teacher Model
BioAlchemy RL	4e-6	cosine	128	1	5120	GRPO	50K	N/A
BioAlchemy SFT	4e-6	cosine	128	1	5120	SFT	50K	Llama-3.3 70B Instruct
Nemotron-Science-v1	4e-6	cosine	128	1	5120	GRPO	29K	N/A
NaturalReasoning	4e-6	cosine	128	1	5120	SFT	50K	DeepSeek-R1
TextbookReasoning	4e-6	cosine	128	1	5120	SFT	50K	DeepSeek-V3

Appendix

Table 10: Table containing training configurations for GRPO and SFT experiments.

Dataset	LR	LR-Scheduler	Batch Size	Epochs	Max Tokens	Max Response Len	Loss	Dataset Size
$\mathcal{S}_{\alpha=0.3}$	4e-6	cosine	128	1	5120	4096	GRPO	60K
$\mathcal{S}_{\alpha=0.5}$	4e-6	cosine	128	1	5120	4096	GRPO	50K
$\mathcal{S}_{\alpha=0.7}$	4e-6	cosine	128	1	5120	4096	GRPO	60K
$\mathcal{S}_{\alpha=1.0}$	4e-6	cosine	128	1	5120	4096	GRPO	60K
$\mathcal{S}^{\text{Final}}_{\alpha=0.5}$	4e-6	cosine	256	1	5120	4096	GRPO	150K

Appendix

Table 11: Training configurations for distribution optimization experiments and final RL experiment.

Benchmark	Max Response Len	Seed	Temperature	Top-p	Metric	N-runs
ProtocolQA	4096	42	0.0	1.0	EM	10
SeqQA	4096	42	0.0	1.0	EM	10
Cloning Scenarios	4096	42	0.0	1.0	EM	10
PubMedQA	4096	42	0.0	1.0	EM	10
GPQA-Bio	4096	42	0.0	1.0	EM	10

Appendix

Table 12: Inference settings for benchmarking. Models were served via vLLM, using tensor parallelism and data parallelism for throughput.

		PubMed Biology		Smoothed % (Upsample $\times$ )
Rank	MeSH Biology Subcategory	Count	%	$\alpha=1.0$	$\alpha=0.7$	$\alpha=0.5$	$\alpha=0.3$
1	Computational Biology	95,179	45.32%	45.32% (1.0 $\times$ )	36.05% (0.8 $\times$ )	26.97% (0.6 $\times$ )	16.51% (0.4 $\times$ )
2	Genetics	75,973	36.18%	36.18% (1.0 $\times$ )	30.79% (0.9 $\times$ )	24.10% (0.7 $\times$ )	15.43% (0.4 $\times$ )
3	Microbiology	22,900	10.90%	10.90% (1.0 $\times$ )	13.30% (1.2 $\times$ )	13.23% (1.2 $\times$ )	10.77% (1.0 $\times$ )
4	Ecology	5,385	2.56%	2.56% (1.0 $\times$ )	4.83% (1.9 $\times$ )	6.42% (2.5 $\times$ )	6.98% (2.7 $\times$ )
5	Molecular Biology	3,944	1.88%	1.88% (1.0 $\times$ )	3.88% (2.1 $\times$ )	5.49% (2.9 $\times$ )	6.35% (3.4 $\times$ )

Appendix

Table 13: Effect of exponential smoothing (

p^{\alpha}_{c_{j}}\propto n_{c_{j}}^{\alpha}

) on the PubMed Biology subcategory sampling distribution, where

c_{j}

for

j\in[21]

represents the 21 MeSH Biology subcategories. We adopt

\alpha=0.5

as a moderate compromise between the values used in multilingual language model pretraining:

\alpha=0.7

( Devlin et al., 2019 ; Liu et al., 2020 ) and

\alpha=0.3

( Conneau et al., 2020 ; Xue et al., 2021 ) . Note that

\alpha=1.0

recovers the original unsmoothed distribution. Total count for PubMed Biology is 210,011 articles. Upsample factor for each category is

\nicefrac{{p_{c_{j}}^{\alpha}}}{{p_{c_{j}}^{\mathrm{orig}}}}

, showing stronger upsampling as

\alpha

decreases.

Rank	MeSH Biology Subcategory	PubMed Biology	Textbook Reasoning	Natural Reasoning	Nemotron Science-v1	BioAlchemy
1	Computational Biology	45.32%	0.57%	12.09%	7.44%	24.95%
2	Genetics	36.18%	11.37%	18.30%	34.67%	36.89%
3	Microbiology	10.90%	14.39%	7.19%	3.78%	16.75%
4	Ecology	2.56%	2.51%	9.24%	0.99%	2.80%
5	Molecular Biology	1.88%	12.59%	8.45%	40.41%	6.63%
6	Synthetic Biology	0.97%	0.04%	0.63%	1.76%	0.50%

Appendix

Table 14: MeSH Biology subcategory distributions across biological reasoning datasets compared to PubMed Biology (2020–2024). TVD = total variation distance. This BioAlchemy dataset version uses the strongest alignment method (exponential smoothing of

\alpha=1.0

detailed in Section 3.2 ), yielding the lowest TVD with PubMed Biology.

Configuration	Optimization F1	Validation F1
Zero-shot baseline	N/A	37.0%
Single-label training	52.7%	49.3%
Multi-label training	61.6%	72.7%

Appendix

Table 15: MeSH Biology classification performance before and after prompt optimization.

Metric	Value
Unique articles	1,474
Total category labels	1,579
Mean labels per article	1.07
Number of categories	21
Training examples	714
Validation examples	180

Appendix

Table 16: Training data statistics for MeSH Biology classification.

Panel of Figure 3 — Figure 3: Prompt template for biology domain classification

Panel of Figure 4 — Figure 4: Few-shot examples for MeSH Biology classification (part 1)

Panel of Figure 5 — Figure 5: Few-shot examples for MeSH Biology classification (part 2)

Panel of Figure 6 — Figure 6: Few-shot examples for MeSH Biology classification (part 3)

Panel of Figure 7 — Figure 7: Prompt template for QA MeSH classification

Panel of Figure 8 — Figure 8: Prompt template for content relevance evaluation

Panel of Figure 9 — Figure 9: Prompt template for generating multiple-choice questions

Panel of Figure 10 — Figure 10: Prompt template for evaluating multiple-choice question quality

Panel of Figure 11 — Figure 11: Prompt template for extracting free-form QA (part 1)

Panel of Figure 12 — Figure 12: Prompt template for extracting free-form QA (part 2)

Panel of Figure 13 — Figure 13: Prompt template for grading free-form QA

Panel of Figure 14 — Figure 14: Prompt for LAB-Bench evaluation

Panel of Figure 15 — Figure 15: Prompt for evaluating GPQA-Diamond

Panel of Figure 16 — Figure 16: Prompt for evaluating PubMedQA

Panel of Figure 17 — Figure 17: Example reasoning question

Explore similar work

How Post-Training Shapes Biological Reasoning Models
Jun 15, 2026Lukas Fesser, Hanlin Zhang, Michelle M. Li +5Neural Network Generalization Supervised Fine-Tuning
Harvard University · Google DeepMind · Google Research
BioPhys-Bridge: A Benchmark for Interdisciplinary Scientific Reasoning in Physics-Grounded Biological Research
Sep 15, 2026Qingyang XuEvidence-Grounded Reasoning Scientific Reasoning in Language Models
Independent Researcher Shanghai, China
When Do Biological Reasoning Models Use Their Biological Inputs?
Oct 1, 2026Ada Fang, Nikitha Thoduguli, Lukas Fesser +3Faithfulness of Language Model Explanations Causal Interventions in Language Models
Harvard University · Massachusetts Institute of Technology

Jun 15, 2026J/K move · Enter open · S save

How Post-Training Shapes Biological Reasoning Models

Lukas Fesser, Hanlin Zhang, Michelle M. Li, Eric Wang+4

Harvard University · Google DeepMind · Google Research

Neural Network Generalization Supervised Fine-Tuning RL Post-Training Language Model Post-Training

Open paper AskPDF arXiv

Scientific reasoning models for biology combine language models with foundation models trained on multimodal biological data, including DNA, RNA, and proteins. These models are built through post-training, yet how each stage shapes reasoning and generalization remains poorly understood. We study when post-training improves performance and when it induces over-specialization. Across genomics, transcriptomics, and proteins, we train and evaluate more than 100 biological reasoning models under controlled variation in backbone, continued pre-training (CPT), supervised fine-tuning (SFT), and reinforcement learning (RL), measuring both in-domain (ID) and out-of-domain (OOD) performance. We find that each post-training stage reshapes generalization in a distinct way rather than contributing uniform gains. CPT improves downstream performance by aligning models with biological language. SFT consistently increases ID performance but causes OOD performance to peak early and decline as models fit the training distribution. RL, when applied to strong SFT checkpoints with aligned rewards, improves OOD performance and partially recovers generalization. These results show that biological reasoning does not improve monotonically with additional supervision or compute. Instead, performance depends on how training stages are composed. Under fixed post-training budgets, the strongest ID-OOD trade-off comes from brief SFT, larger RL allocations, and asymmetric adaptation capacity across stages.

Abstract

Explore similar work

How Post-Training Shapes Biological Reasoning Models

BioPhys-Bridge: A Benchmark for Interdisciplinary Scientific Reasoning in Physics-Grounded Biological Research

When Do Biological Reasoning Models Use Their Biological Inputs?