q-bio.QMAug 18, 2026

Rigorous Evaluation of Large Language Models for Malaria Drug Discovery: Trade-offs in Performance, Scale, and Resource Utility

Authors: Marvellous O. Ajala, Zainab Ashimiyu-Abdusalam, Comfort Adesina

Organizations: Magami Open Sciences Initiative

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We introduce Malaria-Instruct, a curated instruction-following dataset derived from the ChEMBL Legacy Malaria corpus for Malaria virtual screening, and conduct a systematic evaluation of five open-source LLMs; Gemma-2 2B/9B, TxGemma-2B/9B, and LlaSMol-Mistral-7B, on a rigorous out-of-distribution data split. Performance was benchmarked against classical ML models (Random Forest, XGBoost) and frontier proprietary models (Gemini 2.5, OpenAI o3) under few-shot conditions. Fine-tuned LLMs substantially outperformed all baselines: TxGemma-9B achieved the highest ROC-AUC (0.731±0.0050.731 \pm 0.005) and LlaSMol-Mistral-7B the best enrichment factor (EF@1% ≈\approx 4.99). Domain-specific fine-tuning proved categorically indispensable with TxGemma-9B collapsing from ROC-AUC 0.731 to 0.499, under its best few-shot condition, and neither Gemini 2.5 (ROC-AUC ≈\approx 0.53) nor o3 (ROC-AUC ≈\approx 0.59) achieved reliable discrimination without fine-tuning. Biomedical pretraining conferred a measurable advantage at equivalent scale, while chemistry-aware pretraining yielded superior prospective enrichment. Fine-tuned open-source LLMs represent a compelling, resource-efficient paradigm for antimalarial VS, outperforming both classical pipelines and proprietary reasoning models under structurally challenging conditions.

Figures & tables

CategoryTask TypeModelSizeROC_AUCMCCEF (1%)Inf. Time (s)
ClassicalFinetunedRandomForest–0.68280.25812.57900.0004
 FinetunedXGBoost–0.66520.21641.79080.00004
Open SourceFinetunedGemma 22B0.70630.477353.660450.0040
LLMsFinetunedTxGemma 22B0.69550.532754.66460.0047
 FinetunedLlaSmol - Mistral7B0.702250.564154.98650.08015
 FinetunedGemma 29B0.63830.44084.63470.0304
Table 1: General Performance results of the experiments.
ModelShotsAccAUCMCCEF
LlaSMol 7B30.79520.51400.05451.5424
40.80110.51960.07211.6980
50.80200.51280.04551.4416
Gemma 2B30.00610.5025-0.00260.2971
40.00590.50290.00530.7198
50.00090.49930.00381.2486
Table 2: Evaluation Metrics of the Fewshot Open Source Models
ModelShotsROC_AUCMCCEF (1%)
Gemini 2.530.51820.09452.342
40.52840.16893.788
50.51750.08832.254
OpenAI o330.57510.12851.411
40.59070.15731.551
50.58670.15831.646
Table 3: Evaluation Metrics of the Fewshot Closed Source Models[cite: 39].
  FinetuningFewshot (Best)
SizeModelTrain GPUInf. TimeEFInf. GPUInf. TimeEF
2BGemma 216GB0.00403.66024GB0.04570.720
2BTxGemma16GB0.00474.66524GB0.01420.804
7BLlaSmol24GB0.08024.98724GB0.23371.698
9BGemma 224GB0.03044.63524GB0.48951.110
9BTxGemma24GB0.03714.23024GB0.08731.816
Table 4: Compute requirement table of the finetuning and fewshoting task[cite: 101].
Figure 1: Plot of the Enrichment Factor 1% against inference time (s) per molecule.
Figure 2: Plot of the AUC Score against inference time (s) per molecule
More in this topic

Drug Discovery

Drug discovery identifies, designs, and tests molecules that can safely affect disease biology.

See all 165 papers

Explore similar work

CardsList
Mar 3, 2026

MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery

General-purpose large language models (LLMs) that rely on in-context learning do not reliably deliver the scientific understanding and performance required for drug discovery tasks. Simply increasing model size or introducing reasoning tokens does not yield significant performance gains. To address this gap, we introduce the MMAI Gym for Science, a one-stop shop molecular data formats and modalities as well as task-specific reasoning, training, and benchmarking recipes designed to teach foundation models the 'language of molecules' in order to solve practical drug discovery problems. We use MMAI Gym to train an efficient Liquid Foundation Model (LFM) for these applications, demonstrating that smaller, purpose-trained foundation models can outperform substantially larger general-purpose or specialist models on molecular benchmarks. Across essential drug discovery tasks - including molecular optimization, ADMET property prediction, retrosynthesis, drug-target activity prediction, and functional group reasoning - the resulting model achieves near specialist-level performance and, in the majority of settings, surpasses larger models, while remaining more efficient and broadly applicable in the domain.

Maksim Kuznetsov, Zulfat Miftahutdinov, Rim Shayakhmetov +17

Insilico Medicine · Liquid AI

Molecular OptimizationMolecular Property PredictionMolecular Foundation ModelsRetrosynthesisMolecular Representation LearningDrug Discovery
AskPDF
Sep 4, 2026

Training Large Language Models for Small-Molecule Design with Synthetic Task Scaling

Designing viable drug candidates requires searching a combinatorially large and rugged chemical space for molecules that satisfy multiple, often competing, objectives. Large language models (LLMs) provide a useful generative prior for this problem because of their representational capacity, reasoning ability, and flexibility when incorporating information from the external environment. While reinforcement learning from verifiable rewards (RLVR) can be used to improve the capabilities of LLMs, many chemically relevant scoring functions require hours or even days per evaluation, making them prohibitively expensive to use directly during online training. Here, we investigate whether LLMs can learn molecular design strategies from cheaper synthetic tasks that generalize to expensive molecular lead optimization settings. We find that curriculum-based training recipes that gradually incorporate more challenging synthetic design tasks enable strong performance that surpasses that of much larger frontier models on structure-based lead optimization. Our results suggest that scaling post-training using synthetic tasks is an effective strategy for adapting LLMs to high-cost experimental scenarios that are too expensive to directly train on.

Frank Hu, Shriram Chennakesavalu, Zichen Wang +5

Prescient Design, Genentech, South San Francisco, CA, USA

Molecule DesignLLM TrainingDrug DiscoveryCurriculum Learning
AskPDF
Apr 17, 2026

Evaluating the Progression of Large Language Model Capabilities for Small-Molecule Drug Design

Large Language Models (LLMs) have the potential to accelerate small molecule drug design due to their ability to reason about information from diverse sources and formats. However, their practical utility remains unclear due to the lack of benchmarks that reflect real-world scenarios. In this work, we introduce a suite of chemically-grounded tasks spanning molecular property prediction, molecular representation transformations, and molecular design. Importantly, we formulate these tasks as reinforcement learning (RL) environments, enabling a unified approach for evaluation and post-training. Across three model families, we find that frontier models are increasingly proficient at chemical tasks, but that there is significant room for improvement, especially in experimental settings with low data. Critically, we show that RL-based post-training can substantially improve performance. A smaller model post-trained on our environments becomes competitive with state-of-the-art frontier models, despite a significantly weaker base model. This suggests a practical route toward employing LLMs in drug discovery; by combining carefully-designed evaluation tasks with targeted post-training, we can both elucidate and close critical capability gaps.

Shriram Chennakesavalu, Kirill Shmilovich, Hayley Weir +5

Prescient Design, Genentech, South San Francisco, CA, USA

LLM EvaluationMolecular Property PredictionLLM Post-TrainingDrug Discovery
AskPDF