q-bio.GNSep 13, 2025

Genome-Factory: A Library for Tuning, Deploying, and Interpreting Genomic Foundation Models

Authors: Weimin Wu, Xuefeng Song, Yibo Wen, Qinjie Lin, Zhihan Zhou, Jerry Yao-Chieh Hu, Zhong Wang, Han Liu

Organizations: Center for Foundation Models and Generative AI & Department of Computer Science, Northwestern University, USA · School of Natural Sciences, University of California at Merced, USA · Department of Energy Joint Genome Institute, Lawrence Berkeley National Laboratory, USA · Environmental Genomics and Systems Biology Division, Lawrence Berkeley National Laboratory, USA · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We introduce Genome-Factory, the first integrated Python library for tuning, deploying, and interpreting genomic foundation models. Our core contribution is to simplify and unify the workflow for genomic model development: data collection, model tuning, inference, benchmarking, and interpretability. For data collection, Genome-Factory offers an automated pipeline to download genomic sequences and preprocess them. For model tuning, Genome-Factory supports both full and parameter-efficient fine-tuning across diverse genomic models. For inference, Genome-Factory enables both embedding extraction and DNA sequence generation. For benchmarking, we include two existing benchmarks and provide a flexible interface to incorporate additional benchmarks. For interpretability, Genome-Factory introduces an open-source biological interpreter based on a sparse auto-encoder. We validate the utility of Genome-Factory across three dimensions: (i) Compatibility with diverse models and fine-tuning methods; (ii) Benchmarking downstream performance using two open-source benchmarks; (iii) Biological interpretation of learned representations with DNABERT-2. These results highlight its practical value for real-world genomic analysis. GitHub: https://github.com/WeiminWu2000/Genome_Factory.

Figures & tables

Figure 1: Overview of G enome -F actory . The framework consists of six components. Genome Collector acquires genomic sequences from public repositories and performs preprocessing (e.g., GC normalization). Model Loader supports major genomic models (e.g., HyenaDNA, DNABERT-2, Caduceus, Nucleotide Transformer, EVO and GenomeOcean) and their tokenizers. Model Trainer configures workflows, adapts models to classification or regression tasks, and executes training with full fine-tuning or parameter-efficient methods (LoRA, adapters). Inference Engine enables embedding extraction and sequence generation. Benchmarker provides standard benchmarks and allows integration of custom tasks. Biological Interpreter enhances interpretability through sparse auto-encoders.
 FullLoRAAdapter
Mixed precision✓\checkmark✓\checkmark✓\checkmark
Flash attention✓\checkmark✓\checkmark✓\checkmark
Gradient accumulation✓\checkmark✓\checkmark✓\checkmark
Table 1: Featured tuning techniques.
 HyenaDNA-160kDNABERT2Caduceus-131k
MethodTrainableMemThroughputTrainableMemThroughputTrainableMemThroughput
 params(GB)(KTok/s)params(GB)(KTok/s)params(GB)(KTok/s)
Full6.55M5.75381.48117.08M7.3040.707.73M6.89143.55
LoRA0.10M4.45446.051.49M6.1844.070.81M1.85394.30
Adapter0.07M1.841113.320.20M2.09124.300.07M1.85467.72
Table 2: Fine-tuning efficiency with different methods in G enome -F actory . We report the number of trainable parameters, peak GPU memory usage, and throughput (thousands of tokens per second). “—” marks settings we did not evaluate due to computational constraints (e.g., EVO with full fine-tuning). We conduct all experiments on a single NVIDIA H100 (80GB). Due to the large size of EVO (7B) and its computational demands, we exclude full-parameter fine-tuning for this model. ”Full” denotes full-parameter fine-tuning, ”LoRA” denotes low-rank adaptation, and ”Adapter” denotes adapter-based tuning.
ModelMethodTrainable ParamsPeak Mem (GB)Throughput (KTok/s)
Nucleotide Transformer-500MFull480.45 M18.9412.08
LoRA4.46 M15.9514.49
Nucleotide Transformer-2.5BFull2.54 B63.952.40
LoRA11.86 M43.333.34
GenomeOcean-100MFull116.42 M7.3945.28
LoRA1.70 M7.0045.88
Table 3: Fine-tuning efficiency across different model scales. We report experimental results to clearly illustrate the scalability of LoRA’s efficiency with Nucleotide Transformer (500M/2.5B) and GenomeOcean (100M/500M) as representative examples. We conduct all experiments on a single NVIDIA H100 (80GB). ”Full” denotes full-parameter fine-tuning, and ”LoRA” denotes low-rank adaptation.
ModelSizeGUEGenomic Benchmarks
  FullLoRAAdapterFullLoRAAdapter
HyenaDNA-160k7M59.91 ±\pm0.2250.95 ±\pm0.2325.00 ±\pm0.3766.71 ±\pm0.1861.26 ±\pm0.2936.90 ±\pm0.42
Caduceus-131k8M50.07 ±\pm1.6134.70 ±\pm0.1938.61 ±\pm0.3365.38 ±\pm0.2242.10 ±\pm0.3647.74 ±\pm0.27
DNABERT-2117M65.25 ±\pm0.2548.39 ±\pm0.1140.19 ±\pm0.1771.97 ±\pm0.2064.58 ±\pm0.2548.96 ±\pm0.31
GenomeOcean-100M117M65.52 ±\pm0.5759.35 ±\pm0.2346.65 ±\pm0.0468.26 ±\pm0.2666.28 ±\pm0.1953.60 ±\pm0.21
Nucleotide Transformer-500M480M57.63 ±\pm0.2652.96 ±\pm0.1332.01 ±\pm0.4867.99 ±\pm0.2464.68 ±\pm0.3043.95 ±\pm0.28
Table 4: Benchmark across models and tuning methods. We report results on both the GUE benchmark and Genomic Benchmarks. We include traditional baselines to establish a performance floor for foundation-model evaluation: a Convolutional Neural Network (CNN), Random Forest, and Support Vector Machine (SVM) classifiers trained on 4-mer frequency features, Position Weight Matrix (PWM)-based motif-scanning pipelines paired with Random Forest or linear SVM classifiers, and Hidden Markov Model (HMM)-based motif-finding methods. “—” marks settings we did not evaluate due to computational constraints. We exclude EVO (7B) from full fine-tuning due to the high computation cost. Three baselines are trained directly on downstream tasks and thus do not utilize LoRA or Adapter modules. “Full” denotes full-parameter, “LoRA” denotes low-rank adaptation, and “Adapter” denotes adapter-based tuning.
Figure 2: Trade-off between tuning efficiency and performance. The figure shows memory usage in gigabytes (GB), throughput in kilotokens per second (KTok/s), and averaged scores on the GUE benchmark for three models: DNABERT-2, HyenaDNA-160k, and Nucleotide Transformer-500M. We report results for full-tuning (Full), low-rank adaptation (LoRA), and adapter-based fine-tuning (Adapter). The results highlight the trade-offs between resource efficiency and predictive performance.
Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

ModelInput featuresSearch spaceSelected configuration
SVM4-mer frequency; stride =1=1 ; reverse complements treated separately; normalized by total k-mer countKernel ∈{linear,rbf}\in\{\text{linear},\text{rbf}\} ; C∈{0.1,1,10,100}C\in\{0.1,1,10,100\} ; for RBF, γ∈{scale,10−3,10−2,10−1}\gamma\in\{\text{scale},10^{-3},10^{-2},10^{-1}\}Kernel =rbf=\text{rbf} ; C=10C=10 ; γ=scale\gamma=\text{scale}
RFSame 4-mer frequency features as SVMTrees ∈{100,300,500}\in\{100,300,500\} ; max depth ∈{None,10,20,40}\in\{\text{None},10,20,40\} ; min samples split ∈{2,5,10}\in\{2,5,10\} ; min samples leaf ∈{1,2,4}\in\{1,2,4\} ; max features ∈{sqrt,log2,0.5}\in\{\text{sqrt},\text{log2},0.5\}Trees =500=500 ; max depth =20=20 ; min samples split =2=2 ; min samples leaf =1=1 ; max features =sqrt=\text{sqrt}
CNNOne-hot encoded DNA sequence3 convolutional layers; channels (128,256,256)(128,256,256) ; kernel sizes (7,5,3)(7,5,3) ; ReLU; global max pooling; FC hidden dim =256=256 ; dropout ∈{0.3,0.5}\in\{0.3,0.5\} ; learning rate ∈{10−4,5×10−4,10−3}\in\{10^{-4},5\times 10^{-4},10^{-3}\} ; batch size ∈{32,64,128}\in\{32,64,128\} ; weight decay ∈{0,10−5,10−4}\in\{0,10^{-5},10^{-4}\}Adam; learning rate =10−3=10^{-3} ; batch size =64=64 ; dropout =0.5=0.5 ; weight decay =10−5=10^{-5} ; max epochs =30=30 ; early stopping patience =5=5
PWM + RFPWM motif-scanning scoresSame RF search space as aboveTrees =500=500 ; max depth =20=20 ; min samples split =2=2 ; min samples leaf =1=1 ; max features =sqrt=\text{sqrt}
PWM + SVMPWM motif-scanning scoresLinear SVM with C∈{0.1,1,10,100}C\in\{0.1,1,10,100\}Linear kernel; C=10C=10
HMMHMM-derived sequence likelihoodNumber of hidden states ∈{2,4,8}\in\{2,4,8\} ; covariance/transition smoothing ∈{10−4,10−3,10−2}\in\{10^{-4},10^{-3},10^{-2}\}Hidden states =4=4 ; smoothing =10−3=10^{-3}
Appendix
Table 5: Hyperparameter settings for traditional baseline models. Hyperparameters are selected on the validation set, and the test set is evaluated once using the best validation configuration.
ModelVariant TypeVariants  
HyenadnaSequence Length1K/16K/32K/160K/450K/1M  
DNABERT-2Parameter Size117M  
CaduceusSequence Length1K/131K  
Nucleotide TransformerParameter Size50M/100M/250M/500M/1B/2.5B  
EVOSequence Length8K/131K  
GenomeOceanParameter Size100M/500M/4B  
Appendix
Table 6: Supported models in G enome -F actory with their available variants. Models vary by either parameter size (e.g., GenomeOcean: 100M/500M/4B) or input sequence length (e.g., Hyenadna: 1K to 1M). ”Variant Type” specifies the axis of variation, and ”Variants” lists the available options.
ModelDNABERT-2HyenaDNA-160kEVO 2-7BGenomeOcean-4B
ARI0.1670.1460.0820.321
Appendix
Table 7: Unsupervised representation quality on the Genomic Benchmarks species classification task. We extract mean-pooled embeddings, apply kk -means clustering, and report the adjusted Rand index (ARI) against ground-truth species labels.
ModelGUEGBGUANinELRB
GROVER61.38±0.33{\bf 61.38\pm 0.33}65.42±0.2765.42\pm 0.2758.15±0.4158.15\pm 0.4142.73±0.5542.73\pm 0.55
DNABERT-248.39±0.1148.39\pm 0.1164.58±0.2564.58\pm 0.2562.80±0.3562.80\pm 0.3545.19±0.4845.19\pm 0.48
HyenaDNA-160k50.95±0.2350.95\pm 0.2361.26±0.2961.26\pm 0.2955.47±0.3955.47\pm 0.3951.36±0.62{\bf 51.36\pm 0.62}
GenomeOcean-100M59.35±0.2359.35\pm 0.2366.28±0.19{\bf 66.28\pm 0.19}63.14±0.28{\bf 63.14\pm 0.28}48.92±0.5148.92\pm 0.51
Appendix
Table 8: Additional benchmark results with broader benchmark suites. We report MCC under LoRA fine-tuning. GUE denotes the Genome Understanding Evaluation benchmark, GB denotes Genomic Benchmarks, GUANinE denotes GUANinE v1.0, and LRB denotes the Genomics Long-Range Benchmark.
More in this topic

Genomic Foundation Models

See all 21 papers

Explore similar work

CardsList

  1. GENEB: Why Genomic Models Are Hard to Compare

    Jun 3, 2026Daria Ledneva, Mikhail Nuridinov, Denis KuznetsovBenchmark DesignGenomic Foundation Models

    Moscow Independent Research Institute of Artificial Intelligence, Moscow, Russia · Moscow State Institute of Steel and Alloys, Moscow, Russia

  2. GenomeQA: Benchmarking General Large Language Models for Genome Sequence Understanding

    Apr 7, 2026Weicai Long, Yusen Hou, Junning Feng +4LLM EvaluationGenomics

    Hong Kong University of Science and Technology (Guangzhou) · The University of Hong Kong · Peking University

  3. JEPA-DNA: Grounding Genomic Foundation Models through Joint-Embedding Predictive Architectures

    Feb 19, 2026Ariel Larey, Elay Dahan, Amit Bleiweiss +15Genomic Foundation ModelsMolecular Representation Learning

    Applied AI Architecture, NVIDIA, Israel. · Worldwide Field Ops, NVIDIA, Israel. · Developer Programs, NVIDIA, Israel. +2