BioAlchemy: Distilling Biological Literature into Reasoning-Ready Reinforcement Learning Training Data

Brian Hsu; Ozan Gökdemir; Carlo Siebenschuh; Bruce Parrello; Neil Getty; Thomas S. Brettin; Rick L. Stevens; Ian T. Foster

cs.AIApr 3, 2026

BioAlchemy: Distilling Biological Literature into Reasoning-Ready Reinforcement Learning Training Data

Authors: Brian Hsu, Ozan Gökdemir, Carlo Siebenschuh, Bruce Parrello, Neil Getty, Thomas S. Brettin, Rick L. Stevens, Ian T. Foster, +2 more

Organizations: Department of Computer Science, University of Chicago, Chicago, IL 60637 · Data Science and Learning Division, Argonne National Laboratory, Lemont, IL 60439

Abstract

Despite the large corpus of biology training text, the impact of reasoning models on biological research generally lags behind math and coding. In this work, we show that biology questions from current large-scale reasoning datasets do not align well with modern research topic distributions in biology, and that this topic imbalance may negatively affect performance. In addition, we find that methods for extracting challenging and verifiable research problems from biology research text are a critical yet underdeveloped ingredient in applying reinforcement learning for better performance on biology research tasks. We introduce BioAlchemy, a pipeline for sourcing a diverse set of verifiable question-and-answer pairs from a scientific corpus of biology research text. We curate BioAlchemy-345K, a training dataset containing over 345K scientific reasoning problems in biology. Then, we demonstrate how aligning our dataset to the topic distribution of modern scientific biology can be used with reinforcement learning to improve reasoning performance. Finally, we present BioAlchemist-8B, which improves over its base reasoning model by 9.12% on biology benchmarks. These results demonstrate the efficacy of our approach for developing stronger scientific reasoning capabilities in biology. The BioAlchemist-8B model is available at: https://huggingface.co/BioAlchemy.

Figures & tables

Figure 1: Content distribution differences between reasoning problems in biology. Comparison is based on MeSH topics representing PubMed research articles from 2020–2024.

Data Source	# Research Articles	# QA Pairs
ASM	35.8K	173.6K
PLoS CompBio	10.5K	76.1K
PLoS Genetics	8.9K	56.9K
Semantic Scholar	22.5K	38.4K
Total	77.7K	345K

Table 1: Composition of dataset for BioAlchemy.

Dataset	Source	Size (Biology)	Training	MeSH Coverage
BioAlchemy	Scientific Research Text	345K	RL (GRPO)	95.6%
NaturalReasoning	Pretraining Corpora	52.9K	SFT	34.7%
TextbookReasoning	University Textbooks	51.2K	SFT	66.6%
Nemotron-Science-v1	N/A	29.1K	RL (GRPO)	91.4%

Table 2: Comparison of reasoning datasets, where dataset sizes are based on questions classified as biology questions. BioAlchemy is the largest and has the strongest coverage.

Figure 2: BioAlchemy has more MeSH topics per biology question. Comparison is based on 50K subsamples.

Rank	MeSH Biology Subcategory	PubMed 2020–2024	Textbook Reasoning	Natural Reasoning	Nemotron Science-v1	BioAlchemy
1	Computational Biology	45.32%	0.57%	12.09%	7.44%	24.95%
2	Genetics	36.18%	11.37%	18.30%	34.67%	36.89%
3	Microbiology	10.90%	14.39%	7.19%	3.78%	16.75%
4	Ecology	2.56%	2.51%	9.24%	0.99%	2.80%
5	Molecular Biology	1.88%	12.59%	8.45%	40.41%	6.63%
6	Synthetic Biology	0.97%	0.04%	0.63%	1.76%	0.50%

Table 3: BioAlchemy has better distributional alignment at the subtopic level compared to the biology subsets of TextbookReasoning, NaturalReasoning, and Nemotron-Science-v1. Percentages are not mutually exclusive as publications/QA pairs can be indexed under multiple MeSH terms.

Data Source	Mean MeSH Topics	% Non-Zero MeSH Topics	% With $\geq 2$ MeSH Topics
NaturalReasoning	0.41	34.7%	6.2%
TextbookReasoning	0.77	66.6%	10.1%
Nemotron-Science-v1	1.19	91.4%	28.0%
BioAlchemy	1.37	95.6%	40.8%

Table 4: BioAlchemy has higher MeSH topic coverage and higher average number of topics. The average number of MeSH subcategories for PubMed publications is 1.07.

Dataset	Size	Topic Max/Min Ratio	Avg. Upsample †	Sample Yield	TVD
Full Dataset $\mathcal{D}$	345K	9,301 $\times$	11.8 $\times$	100%	0.52
Dataset $\mathcal{S}_{\alpha=1.0}$	126.7K	7,932 $\times$	1.0 $\times$	36.73%	0.21
Dataset $\mathcal{S}_{\alpha=0.7}$	146.2K	536 $\times$	9.7 $\times$	42.38%	0.25
Dataset $\mathcal{S}_{\alpha=0.5}$	152.0K	89 $\times$	39.6 $\times$	44.06%	0.27
Dataset $\mathcal{S}_{\alpha=0.3}$	83.2K	15 $\times$	134.3 $\times$	24.10%	0.39

Table 5: Sub-sampling BioAlchemy-345K with increasing representation of low-frequency topics from exponential smoothing.

\mathcal{S}_{\alpha=0.5}

provides the best tradeoff in terms of dataset size, distributional alignment, and upsampling of rare topics.

Benchmark	SFT			RL
	Natural Reasoning	Textbook Reasoning	BioAlchemy	Nemotron- Science-v1	BioAlchemy
ProtocolQA	43.61 $\pm$ 2.55%	43.33 $\pm$ 2.89%	43.80 $\pm$ 2.10%	44.44 $\pm$ 2.09%	45.83 $\pm$ 2.32%
SeqQA	10.43 $\pm$ 0.75%	15.20 $\pm$ 0.73%	14.03 $\pm$ 0.69%	20.42 $\pm$ 0.66%	18.73 $\pm$ 0.85%
Cloning Scenarios	8.48 $\pm$ 1.99%	8.18 $\pm$ 2.30%	9.70 $\pm$ 2.85%	13.94 $\pm$ 3.85%	11.52 $\pm$ 3.03%
PubMedQA	67.89 $\pm$ 0.60%	72.47 $\pm$ 0.15%	69.44 $\pm$ 0.24%	69.21 $\pm$ 0.38%	68.09 $\pm$ 0.28%
GPQA-Bio	36.84 $\pm$ 0.00%	42.11 $\pm$ 0.00%	44.21 $\pm$ 3.64%	47.89 $\pm$ 1.19%	57.89 $\pm$ 0.00%

Table 6: Comparison of reasoning datasets across relevant training strategies. BioAlchemy with GRPO performs best overall.

Dataset	Alignment	Upsampling	ProtocolQA	SeqQA	Cloning Scenarios	PubMedQA	GPQA-Bio	General Avg.
Baseline	N/A	N/A	42.69 $\pm$ 2.06%	8.42 $\pm$ 0.43%	5.76 $\pm$ 2.16%	69.00 $\pm$ 0.35%	42.63 $\pm$ 1.19%	33.70 $\pm$ 0.65%
$\bm{\mathcal{S}_{\alpha=0.3}}$	Weakest	Strongest	46.30 $\pm$ 1.25%	19.63 $\pm$ 0.98%	12.12 $\pm$ 3.96%	67.75 $\pm$ 0.28%	53.16 $\pm$ 2.78%	39.79 $\pm$ 1.02%
$\bm{\mathcal{S}_{\alpha=0.5}}$	Weak	Strong	45.83 $\pm$ 2.32%	18.73 $\pm$ 0.85%	11.52 $\pm$ 3.03%	68.09 $\pm$ 0.28%	57.89 $\pm$ 0.00%	40.41 $\pm$ 0.78%
$\bm{\mathcal{S}_{\alpha=0.7}}$	Strong	Weak	44.63 $\pm$ 1.97%	20.13 $\pm$ 0.75%	11.82 $\pm$ 3.14%	68.65 $\pm$ 0.51%	51.58 $\pm$ 2.38%	39.36 $\pm$ 0.90%
$\bm{\mathcal{S}_{\alpha=1.0}}$	Strongest	Weakest	45.93 $\pm$ 1.60%	19.98 $\pm$ 0.73%	10.91 $\pm$ 3.57%	68.83 $\pm$ 0.30%	63.16 $\pm$ 0.00%	41.76 $\pm$ 0.80%
Best	$\alpha=1.0$	$\alpha=0.3$	46.30 $\pm$ 1.25%	20.13 $\pm$ 0.75%	12.12 $\pm$ 3.96%	69.00 $\pm$ 0.35%	63.16 $\pm$ 0.00%	41.76 $\pm$ 0.80%

Table 7: Accuracy from training on datasets with stronger tail categories. Strongest alignment with

P_{\text{Bio}}

leads to best performance. Baseline represents base Qwen3-8B results.

Model	ProtocolQA	SeqQA	Cloning Scenarios	PubMedQA	GPQA-Bio	Overall Avg.
BioAlchemist-8B 150K	46.20 $\pm$ 1.89%	22.32 $\pm$ 0.91%	15.15 $\pm$ 4.57%	68.32 $\pm$ 0.48%	62.11 $\pm$ 2.38%	42.82 $\pm$ 1.12%
BioAlchemist-8B 50K	45.83 $\pm$ 2.32%	18.73 $\pm$ 0.85%	11.52 $\pm$ 3.03%	68.09 $\pm$ 0.28%	57.89 $\pm$ 0.00%	40.41 $\pm$ 0.78%
Qwen3-8B	42.69 $\pm$ 2.06%	8.42 $\pm$ 0.43%	5.76 $\pm$ 2.16%	69.00 $\pm$ 0.35%	42.63 $\pm$ 1.19%	33.70 $\pm$ 0.65%
DeepSeek-R1-Llama-8B	33.61 $\pm$ 2.58%	4.97 $\pm$ 0.41%	10.91 $\pm$ 2.09%	26.89 $\pm$ 0.58%	5.26 $\pm$ 0.00%	16.33 $\pm$ 0.68%
GPT-OSS-20B	52.78 $\pm$ 2.25%	18.12 $\pm$ 0.79%	7.27 $\pm$ 1.83%	72.79 $\pm$ 0.24%	55.79 $\pm$ 3.17%	41.35 $\pm$ 0.88%
Best	52.78 $\pm$ 2.25%	22.32 $\pm$ 0.91%	15.15 $\pm$ 4.57%	72.79 $\pm$ 0.24%	62.11 $\pm$ 2.38%	42.82 $\pm$ 1.12%

Table 8: Final RL model performance across benchmarks. BioAlchemy-8B 150K performs best, and scales with more data.

Appendix figures & tables23 assets

Supplementary material from the paper’s appendix.

Appendix

Training	Rollouts	Max Response Len	Temperature	Clip Ratio Range	Normalize Adv.	Entropy Coef.	Loss Aggr.
GRPO	5	4096	0.6	[0.2, 0.28]	False	0.0005	Token-mean

Appendix

Table 9: GRPO-specific training configurations used across all experiments. Settings are applied using the VeRL framework.

Dataset	LR	LR-Scheduler	Batch Size	Epochs	Max Tokens	Loss	Dataset Size	Teacher Model
BioAlchemy RL	4e-6	cosine	128	1	5120	GRPO	50K	N/A
BioAlchemy SFT	4e-6	cosine	128	1	5120	SFT	50K	Llama-3.3 70B Instruct
Nemotron-Science-v1	4e-6	cosine	128	1	5120	GRPO	29K	N/A
NaturalReasoning	4e-6	cosine	128	1	5120	SFT	50K	DeepSeek-R1
TextbookReasoning	4e-6	cosine	128	1	5120	SFT	50K	DeepSeek-V3

Appendix

Table 10: Table containing training configurations for GRPO and SFT experiments.

Dataset	LR	LR-Scheduler	Batch Size	Epochs	Max Tokens	Max Response Len	Loss	Dataset Size
$\mathcal{S}_{\alpha=0.3}$	4e-6	cosine	128	1	5120	4096	GRPO	60K
$\mathcal{S}_{\alpha=0.5}$	4e-6	cosine	128	1	5120	4096	GRPO	50K
$\mathcal{S}_{\alpha=0.7}$	4e-6	cosine	128	1	5120	4096	GRPO	60K
$\mathcal{S}_{\alpha=1.0}$	4e-6	cosine	128	1	5120	4096	GRPO	60K
$\mathcal{S}^{\text{Final}}_{\alpha=0.5}$	4e-6	cosine	256	1	5120	4096	GRPO	150K

Appendix

Table 11: Training configurations for distribution optimization experiments and final RL experiment.

Benchmark	Max Response Len	Seed	Temperature	Top-p	Metric	N-runs
ProtocolQA	4096	42	0.0	1.0	EM	10
SeqQA	4096	42	0.0	1.0	EM	10
Cloning Scenarios	4096	42	0.0	1.0	EM	10
PubMedQA	4096	42	0.0	1.0	EM	10
GPQA-Bio	4096	42	0.0	1.0	EM	10

Appendix

Table 12: Inference settings for benchmarking. Models were served via vLLM, using tensor parallelism and data parallelism for throughput.

		PubMed Biology		Smoothed % (Upsample $\times$ )
Rank	MeSH Biology Subcategory	Count	%	$\alpha=1.0$	$\alpha=0.7$	$\alpha=0.5$	$\alpha=0.3$
1	Computational Biology	95,179	45.32%	45.32% (1.0 $\times$ )	36.05% (0.8 $\times$ )	26.97% (0.6 $\times$ )	16.51% (0.4 $\times$ )
2	Genetics	75,973	36.18%	36.18% (1.0 $\times$ )	30.79% (0.9 $\times$ )	24.10% (0.7 $\times$ )	15.43% (0.4 $\times$ )
3	Microbiology	22,900	10.90%	10.90% (1.0 $\times$ )	13.30% (1.2 $\times$ )	13.23% (1.2 $\times$ )	10.77% (1.0 $\times$ )
4	Ecology	5,385	2.56%	2.56% (1.0 $\times$ )	4.83% (1.9 $\times$ )	6.42% (2.5 $\times$ )	6.98% (2.7 $\times$ )
5	Molecular Biology	3,944	1.88%	1.88% (1.0 $\times$ )	3.88% (2.1 $\times$ )	5.49% (2.9 $\times$ )	6.35% (3.4 $\times$ )

Appendix

Table 13: Effect of exponential smoothing (

p^{\alpha}_{c_{j}}\propto n_{c_{j}}^{\alpha}

) on the PubMed Biology subcategory sampling distribution, where

c_{j}

for

j\in[21]

represents the 21 MeSH Biology subcategories. We adopt

\alpha=0.5

as a moderate compromise between the values used in multilingual language model pretraining:

\alpha=0.7

( Devlin et al., 2019 ; Liu et al., 2020 ) and

\alpha=0.3

( Conneau et al., 2020 ; Xue et al., 2021 ) . Note that

\alpha=1.0

recovers the original unsmoothed distribution. Total count for PubMed Biology is 210,011 articles. Upsample factor for each category is

\nicefrac{{p_{c_{j}}^{\alpha}}}{{p_{c_{j}}^{\mathrm{orig}}}}

, showing stronger upsampling as

\alpha

decreases.

Rank	MeSH Biology Subcategory	PubMed Biology	Textbook Reasoning	Natural Reasoning	Nemotron Science-v1	BioAlchemy
1	Computational Biology	45.32%	0.57%	12.09%	7.44%	24.95%
2	Genetics	36.18%	11.37%	18.30%	34.67%	36.89%
3	Microbiology	10.90%	14.39%	7.19%	3.78%	16.75%
4	Ecology	2.56%	2.51%	9.24%	0.99%	2.80%
5	Molecular Biology	1.88%	12.59%	8.45%	40.41%	6.63%
6	Synthetic Biology	0.97%	0.04%	0.63%	1.76%	0.50%

Appendix

Table 14: MeSH Biology subcategory distributions across biological reasoning datasets compared to PubMed Biology (2020–2024). TVD = total variation distance. This BioAlchemy dataset version uses the strongest alignment method (exponential smoothing of

\alpha=1.0

detailed in Section 3.2 ), yielding the lowest TVD with PubMed Biology.

Configuration	Optimization F1	Validation F1
Zero-shot baseline	N/A	37.0%
Single-label training	52.7%	49.3%
Multi-label training	61.6%	72.7%

Appendix

Table 15: MeSH Biology classification performance before and after prompt optimization.

Metric	Value
Unique articles	1,474
Total category labels	1,579
Mean labels per article	1.07
Number of categories	21
Training examples	714
Validation examples	180

Appendix

Table 16: Training data statistics for MeSH Biology classification.

Panel of Figure 3 — Figure 3: Prompt template for biology domain classification

Panel of Figure 4 — Figure 4: Few-shot examples for MeSH Biology classification (part 1)

Panel of Figure 5 — Figure 5: Few-shot examples for MeSH Biology classification (part 2)

Panel of Figure 6 — Figure 6: Few-shot examples for MeSH Biology classification (part 3)

Panel of Figure 7 — Figure 7: Prompt template for QA MeSH classification

Panel of Figure 8 — Figure 8: Prompt template for content relevance evaluation

Panel of Figure 9 — Figure 9: Prompt template for generating multiple-choice questions

Panel of Figure 10 — Figure 10: Prompt template for evaluating multiple-choice question quality

Panel of Figure 11 — Figure 11: Prompt template for extracting free-form QA (part 1)

Panel of Figure 12 — Figure 12: Prompt template for extracting free-form QA (part 2)

Panel of Figure 13 — Figure 13: Prompt template for grading free-form QA

Panel of Figure 14 — Figure 14: Prompt for LAB-Bench evaluation

Panel of Figure 15 — Figure 15: Prompt for evaluating GPQA-Diamond

Panel of Figure 16 — Figure 16: Prompt for evaluating PubMedQA

Panel of Figure 17 — Figure 17: Example reasoning question

Explore similar work

Jun 15, 2026

How Post-Training Shapes Biological Reasoning Models

Scientific reasoning models for biology combine language models with foundation models trained on multimodal biological data, including DNA, RNA, and proteins. These models are built through post-training, yet how each stage shapes reasoning and generalization remains poorly understood. We study when post-training improves performance and when it induces over-specialization. Across genomics, transcriptomics, and proteins, we train and evaluate more than 100 biological reasoning models under controlled variation in backbone, continued pre-training (CPT), supervised fine-tuning (SFT), and reinforcement learning (RL), measuring both in-domain (ID) and out-of-domain (OOD) performance. We find that each post-training stage reshapes generalization in a distinct way rather than contributing uniform gains. CPT improves downstream performance by aligning models with biological language. SFT consistently increases ID performance but causes OOD performance to peak early and decline as models fit the training distribution. RL, when applied to strong SFT checkpoints with aligned rewards, improves OOD performance and partially recovers generalization. These results show that biological reasoning does not improve monotonically with additional supervision or compute. Instead, performance depends on how training stages are composed. Under fixed post-training budgets, the strongest ID-OOD trade-off comes from brief SFT, larger RL allocations, and asymmetric adaptation capacity across stages.

Lukas Fesser, Hanlin Zhang, Michelle M. Li +5

Harvard University · Google DeepMind · Google Research

Neural Network Generalization Supervised Fine-Tuning RL Post-Training Language Model Post-Training

Ask PDF

Sep 15, 2026

BioPhys-Bridge: A Benchmark for Interdisciplinary Scientific Reasoning in Physics-Grounded Biological Research

Language models face unique challenges in analyzing interdisciplinary scientific research literature. In biophysics research, faithful answers require grounding observed data in source evidence, interpreting it through a quantitative physics model, and linking it to a biological mechanism. To address this challenge, we introduce BioPhys-Bridge, a novel benchmark dataset for evidence-grounded scientific reasoning over biophysical literature. Each case contains evidence blocks, stable evidence IDs, quantitative values, units, equations, assumptions, mechanisms, and next decisions as grounding targets for question answering (QA) and retrieval-augmented generation (RAG). The initial release contains 500 cases, 1,517 agent-facing tasks, and covers six biological domains and nine physical model families, including three sparse families reserved for future expansion. We enforce strict quality gates for all cases in schema, evidence-integrity, quantitative-grounding, source-license, duplicate, unit-normalization, with domain expert review and annotation for 81 cases. Preliminary evaluations show that DeepSeek-V4-Flash obtain the highest evidence-ID

F_1

score (0.360), followed by Qwen3.7-Max (0.316) and GPT-4o-mini (0.294). BioPhys-Bridge is an interdisciplinary benchmark for evaluating attribution, faithfulness, hallucination reduction, and biological experiment design with complex, multi-step scientific reasoning. Future works will increase the size and complexity of the dataset and perform comprehensive evaluations. Code and data are available in the GitHub repository and on Hugging Face.

Qingyang Xu

Independent Researcher Shanghai, China

Evidence-Grounded Reasoning Scientific Reasoning in Language Models Scientific QA

Ask PDF

Oct 1, 2026

When Do Biological Reasoning Models Use Their Biological Inputs?

Biological reasoning models use post-training to connect LLMs to biological foundation model representations and biological text. Their benchmark accuracy is taken as evidence that LLMs reason over these inputs. We test this assumption in six biological reasoning models across DNA, protein, and single-cell tasks. We perturb one biological input while holding the query and other inputs fixed, construct evidence conflicts that pair the foundation model representation of one genome, protein, or cell with the text of another, fit linear probes to the representations the language model receives, and analyze reasoning traces against the biological inputs. Evo2 and ESM3 contribute little to BioReason and BioReason-Pro performance on the evaluated tasks. Shuffling the DNA sequence barely changes BioReason disease prediction accuracy, and in evidence conflicts the two models follow the text in 97.9% and 99.7% of cases. Linear probes trained on the Evo2 and ESM3 representations predict the task targets, so these foundation models encode information relevant to the task, but provide limited overall performance improvement to BioReason and BioReason-Pro. In contrast, foundation model inputs contribute to ChatNT, Prot2Text-V2, and CellWhisperer performance, and differentially expressed genes in the gene sentence contribute to Cell2Sentence-Scale performance. Across SFT and RL checkpoints of BioReason-Pro and 42 BioReason checkpoints, increases in accuracy do not imply greater performance contributions from biological inputs. BioReason traces misstate nucleotide changes, while BioReason-Pro traces describe functions omitted from final predictions under evidence conflicts. We find that current post-training strategies do not ensure that foundation model representations contribute to task performance.

Ada Fang, Nikitha Thoduguli, Lukas Fesser +3

Harvard University · Massachusetts Institute of Technology

Faithfulness of Language Model Explanations Causal Interventions in Language Models

Ask PDF