Organizations: Electrical Engineering and Computer Science, University of Kansas, USA · Cell Biology and Physiology, University of Kansas Medical Center, USA
Gene set analysis is a cornerstone of functional genomics, yet it remains labor-intensive and heavily dependent on manual curation and expert biological interpretation. While Large Language Models (LLMs) have emerged as powerful tools for genomic reasoning and annotation, most existing approaches rely on symbolic gene names and fail to capture domain-specific biological structure, particularly protein sequence information that governs molecular activity, interactions, and downstream gene function. In this work, we propose SoftGene, a novel framework for LLM-based gene set annotation that leverages the hierarchical structure of gene sets. First, we use a hierarchical attention-based encoder built on ESM, a protein language model, to represent each gene set using protein-level amino acid sequence information. Second, we construct a hybrid prompting scheme that combines soft prompts derived from gene set embeddings with hard prompts containing auxiliary context generated by an LLM, and feed the resulting prompt into a local LLM for annotation. We evaluate our framework on two benchmark datasets: Gene Ontology (GO) and the Molecular Signatures Database (MSigDB). Our results show that integrating protein-sequence representations with textual context improves gene set annotation overall, while per-domain analyses reveal that the contribution of protein embeddings varies across biological domains.
Figures & tables
Figure 1: High-level overview of the model architecture. The proposed approach combines a sequence-level pathway that encodes amino acid information into soft prompt tokens with a textual pathway that provides high-level functional context from a large language model, enabling the generation of descriptive gene set annotations optimized via supervision against ground-truth labels.
Figure 2: Overview of the hierarchical gene set encoder. Frozen pretrained protein representations are hierarchically aggregated using trainable attention-based pooling to produce a gene set embedding.
Figure 3: To obtain the gene set context, we utilize a system prompt to define a persona and stylistic guidelines alongside a user prompt for task instructions, which are fed to a commercial model.
Figure 4: Overview of the hybrid gene set soft prompting module. A gene set embedding is converted into soft prompt tokens that, together with instruction and context prompts, condition a frozen language model to generate free-text gene set annotations.
Measure
GO
MSigDB
Number of Gene Sets
18,889
2,461
Avg. Genes Per Set
21.18
50.07
Min. Number of Genes
1
5
Max. Number of Genes
13,616
1,491
Unique Genes Covered
42,597
12,021
Avg. Annotation Length
38.51
36.25
Table 1: Dataset statistics (annotation length measured in characters).
Configuration
Gene Ontology (GO)
MSigDB
METEOR
ROUGE-L
BERTScore-F1
METEOR
ROUGE-L
BERTScore-F1
Prompt Engineering Methods
Zero-shot
0.024 ± 0.001
0.133 ± 0.001
0.773 ± 0.00
0.087 ± 0.014
0.127 ± 0.009
0.762 ± 0.002
Few-shot
0.018 ± 0.001
0.164 ± 0.003
0.740 ± 0.001
0.100 ± 0.025
0.136 ± 0.017
0.771 ± 0.008
One-shot CoT
0.065 ± 0.001
0.096 ± 0.002
0.761 ± 0.001
0.086 ± 0.010
0.112 ± 0.003
0.762 ± 0.001
Prompt-tuning Methods
Table 2: Performance of SoftGene and baselines on the Gene Ontology (GO) and MSigDB datasets. First, second, and third-best results are indicated by boldface , double underline , and single underline , respectively.
Configuration
Gene Ontology (GO)
MSigDB
METEOR
ROUGE-L
BERTScore
METEOR
ROUGE-L
BERTScore
w/o Soft Prompt ( S )
0.020 ± 0.000
0.065 ± 0.001
0.707 ± 0.001
0.006 ± 0.004
0.045 ± 0.002
0.704 ± 0.001
w/o Context ( Hcont )
0.182 ± 0.024
0.232 ± 0.028
0.795 ± 0.005
0.145 ± 0.007
0.202 ± 0.020
0.790 ± 0.003
w/o Protein Embeddings
0.199 ± 0.015
0.252 ± 0.017
0.803 ± 0.002
0.097 ± 0.017
0.135 ± 0.025
0.774 ± 0.012
SoftGene
0.220 ± 0.033
0.267 ± 0.029
0.806 ± 0.006
0.157 ± 0.005
0.215 ± 0.001
0.795 ± 0.003
Table 3: Ablation study.
Figure 5: Performance comparison of the SoftGene model across different soft prompt lengths using the GO dataset.
Figure 6: Performance comparison of the SoftGene model across different soft prompt lengths using the MSigDB dataset.
Gene Ontology (GO)
MSigDB
Split
METEOR
ROUGE-L
BERTScore
METEOR
ROUGE-L
BERTScore
Gene-Disjoint
0.205 ± 0.014
0.251 ± 0.015
0.801 ± 0.009
0.136 ± 0.082
0.193 ± 0.062
0.791 ± 0.034
Random
0.220 ± 0.033
0.267 ± 0.029
0.806 ± 0.006
0.157 ± 0.005
0.215 ± 0.001
0.795 ± 0.003
Table 4: Performance comparison between random and gene-disjoint train-test splits.
PLM
Randomized
Δ
Domain
METEOR
ROUGE-L
BERTScore
METEOR
ROUGE-L
BERTScore
METEOR
ROUGE-L
BERTScore
BP
0.223
0.264
0.802
0.222
0.256
0.798
-0.001
-0.008
-0.004
CC
0.164
0.212
0.784
0.102
0.151
0.769
-0.062
-0.061
-0.015
MF
0.236
0.314
0.825
0.213
0.285
0.820
-0.023
-0.029
-0.005
Table 5: Per-domain ablation of PLM embeddings on the GO dataset. Comparison between real PLM embeddings and randomized embeddings, with differences.
Figure 7: Attention visualization for GO:0045986. The tree depicts the three-level hierarchy of the gene set encoder, with the top five protein isoforms (by attention weight) shown beneath each gene. Node size reflects learned attention weights, indicating the relative contribution of each gene and protein isoform.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Performance comparison of SoftGene across different commercial models using the GO dataset.
Figure 9: Performance comparison of SoftGene across different commercial models using the MSigDB dataset.
Figure 10: Performance comparison of SoftGene across different protein language models using the GO dataset.
Figure 11: Performance comparison of SoftGene across different protein language models using the MSigDB dataset.
Figure 12: BERTScore F1 performance across gene set characteristics in the GO dataset. Box plots display the distribution of BERTScore F1 scores stratified by four gene set properties: (A) description length in words, (B) GO domain (biological process, cellular component, molecular function), (C) mean gene frequency across all gene sets (binned into quintiles Q1–Q5, where Q1 represents the most unique genes and Q5 the least unique), and (D) number of genes per gene set. Sample sizes (n) are indicated at the bottom of each panel.
Figure 13: BERTScore F1 performance across gene set characteristics in the MSigDB dataset. Box plots display the distribution of BERTScore F1 scores stratified by four gene set properties: (A) description length in words, (B) MSigDB collection (BIOCARTA, KEGG LEGACY, PID, REACTOME), (C) mean gene frequency across all gene sets (binned into quintiles Q1–Q5, where Q1 represents the most unique genes and Q5 the least unique), and (D) number of genes per gene set. Sample sizes (n) are indicated at the bottom of each panel.
Figure 14: BERTScore F1 performance stratified by GO ontology depth. Box plots display the distribution of BERTScore F1 scores across gene sets binned by their depth within the Gene Ontology hierarchy (0–3, 4–5, 6–7, 8–9, and 10+).
Configuration
Gene Ontology (GO)
MSigDB
Semantic Similarity
Hierarchical Semantic Similarity
Semantic Similarity
Prompt Engineering Methods
Zero-shot
0.861 ± 0.002
0.871 ± 0.002
0.876 ± 0.001
Few-shot
0.860 ± 0.001
0.869 ± 0.001
0.872 ± 0.002
One-shot CoT
0.863 ± 0.010
0.885 ± 0.000
0.876 ± 0.003
Prompt-tuning Methods
Appendix
Table 6: Performance of SoftGene and baselines on the Gene Ontology (GO) and MSigDB datasets using additional performance metrics. Best results are indicated by boldface .
Group
Gene Set
Score
Reference
Predicted
Top 5
GO:0004143
0.9500
ATP-dependent diacylglycerol kinase activity
diacylglycerol kinase activity
GO:0045588
0.9500
positive regulation of gamma-delta T cell differentiation
positive regulation of T cell differentiation
GO:0042129
0.9499
regulation of T cell proliferation
positive regulation of T cell activation
GO:0140950
0.9496
histone H2A deubiquitinase activity
histone H2A deubiquitination
GO:0006563
0.9493
L-serine metabolic process
serine metabolic process
Bottom 5
GO:0030431
0.5774
sleep
positive regulation of interleukin-18 production
Appendix
Table 7: Top 5 and bottom 5 scoring gene set predictions from GO by BERTScore F1 (excluding perfect scores).
Functional annotation of newly sequenced proteins remains a bottleneck in molecular biology: the number of sequences in public repositories grows far faster than the capacity for manual curation. Most computational approaches consider annotation as multi-label classification over a fixed ontology, which constrains predictions to a predefined label set. In this work we study the the protein annotation as a sequence-to-text generation problem. We fine-tune the 3B-parameter Ministral 3 base model with QLoRA (4-bit NF4 quantization with low-rank adapters) on sequence annotation pairs. We assess predictions with an LLM-as-expert protocol: a GPT model prompted as a senior molecular-biology curator scores organism identification as binary and function annotation quality. We conclude that QLoRA-fine-tuned compact LLMs can generate curator-style annotations with genuine biological value for a substantial subset of proteins. We also discuss future directions in data quality, model scaling, and evidence grounding that are needed to make the approach sufficiently reliable for practical use.
The functional annotation of genes in non-model organisms remains a significant challenge in computational biology, with 20-70% of sequenced genes lacking characterized functions. Traditional homology-based methods are often costly and strongly dependent on high sequence similarity. This study presents Homo-RAG, a framework for large language model-based gene function prediction that integrates homology-guided multi-hop retrieval with evidence-aware ranking. The framework exploits biological relationships between zebrafish and human orthologs to guide evidence acquisition from ZFIN, UniProt, and PubMed through hybrid dense and lexical retrieval. An Evidence Confidence Score (ECS) integrates semantic relevance, entity matching, orthology information, source reliability, and literature association signals to refine the ranking of retrieved evidence. Extensive evaluation across 150 queries and 7,200 retrieved documents shows that evidence weighting parameter of lambda=0.50 improves NDCG@10 to 0.9879 and MRR to 0.99, while retrieving relevant evidence for 99.33% of queries. Furthermore, 80% of the retrieved documents are query-exclusive, indicating that evidence quality complements rather than replaces retrieval relevance. These findings establish Homo-RAG as a practical and robust framework for reliable, evidence-grounded gene function prediction in understudied organisms. The study addresses important limitations of conventional annotation pipelines while identifying opportunities for future improvements in evidence features and attribution mechanisms.
Azrin Sultana
Department of Computer Science, American International University-Bangladesh, 408/1, Kuratoli, Khilkhet, 1229, Dhaka, Bangladesh.
Gene Ontology (GO) enrichment analysis is a foundational tool for translating large-scale genomic data into biological insights, but typically yields hundreds of redundant terms that obscure overarching themes. Existing summarization tools rely on fixed similarity metrics (REVIGO, GOSemSim, clusterProfiler::simplify()), gene-overlap measures (Metascape), or static hierarchy mappings (GO-slim), and therefore cannot incorporate biological context. Manual curation provides context-aware grouping but is subjective and labor-intensive. A scalable, context-aware framework is needed to cluster GO terms into interpretable higher-order biological domains. Here we present LLMBDC (Large Language Model for Biological Domains Oriented Clustering of Gene Ontology), a training-free framework that leverages zero-shot semantic reasoning of LLMs with confidence scoring to cluster GO terms into BioDomains using only ontology information at inference time. Benchmarked across Alzheimer's disease (AD) and Fragile X syndrome (FXS) against six baseline methods including SapBERT, LLMBDC achieved substantially higher precision, recall, and clustering performance. Against ground-truth annotations, LLMBDC improved ARI from 9.7% to 73.3% (AD) and from 15.7% to 66.6% (FXS) over REVIGO, with corresponding NMI gains from 59.9% to 73.4% (AD) and 66.0% to 79.5% (FXS). A Cauchy combination test further confirmed that aggregated BioDomains retained statistically significant functional signals. LLMBDC provides a scalable, reproducible, and interpretable route to context-aware, system-level interpretation of GO enrichment results while preserving biological specificity.