On the Reliability of AI Methods in Drug Discovery: Evaluation of Boltz-2 for Structure and Binding Affinity Prediction
Authors: Shunzhou Wan, Xibei Zhang, Xiao Xue, Peter V. Coveney
Organizations: Centre for Computational Science, University College London, London, WC1H 0AJ, U.K. · Advanced Research Computing Centre, University College London, London, WC1H 0AJ, U.K.
Despite continuing hype about the role of AI in drug discovery, no "AI-discovered drugs" have so far received regulatory approval. Here we assess one of the latest AI based tools in this domain. The ability to rapidly predict protein-ligand structures and binding affinities is pivotal for accelerating drug discovery. Boltz-2, a recently developed biomolecular foundation model, aims to bridge the gap between AI efficiency and physics-based precision through a joint "co-folding" approach. In this study, we provide an extensive evaluation of Boltz-2 using two large-scale datasets: 16,780 compounds for 3CLPro and 21,702 compounds for TNKS2. We compare Boltz-2 predicted structures with traditional docking and binding affinities with binding free energies derived from the physics-based ESMACS protocol. Structural analysis reveals significant global RMSD variations, indicating that Boltz-2 predicts multiple protein conformations and ligand binding positions rather than a single converged pose. Energetic evaluations exhibit only weak to moderate correlations across the global datasets. Furthermore, a focused analysis of the top 100 compounds yields no significant correlation between the Boltz-2 predictions and the binding free energies from fine-grained ESMACS, alongside observed saturation difference in ligand structures. Our results show that while Boltz-2 offers substantial speed for initial screening, it lacks the energetic resolution required for lead identification. These findings highlight the necessity of employing physics-based methods for the reliability and refinement of AI-derived models.
Figures & tables
Figure 1: Structural comparison of Boltz-2 and docking predictions for 3CLPro (a-c) and TNKS2 (d-f). Distributions are shown for pairwise RMSDs of the protein structures (a, d) and ligand poses (b, e), alongside LDDT scores (c, f) evaluating the preservation of the local atomic environment and binding site interactions.
Figure 2: Representative binding sites predicted by Boltz-2 co-folding for a) 3CLPro and b) TNKS2. The protein (cartoon) and a bound compound (yellow sticks) are from PDB 6W63 and 4UI5, respectively. Five binding sites (i-vi) are predicted for 3CLPro, with LDDT metric values of 1.8 Å, 6.4 Å, 7.8 Å, 13.0 Å, 13.2 Å and 14.3 Å (Fig. 1 c), respectively. For TNKS2, all but one compounds bind at or near the x-ray identified binding site (i, ii), with the exception at a different site (iii), with LDDT values (Fig. 1 f) of 0.8 Å, 3.6 Å and 6.9 Å, respectively.
Figure 3: The distribution of confidence scores from the Boltz-2 predictions of both 3CLPro and TNKS2 systems. Percentages are calculated relative to the total number of ligands in each protein system.
Figure 4: Correlation between binding free energies predicted by Boltz-2 ( ΔGBoltz ) and calculated via ESMACS ( ΔGESMACS ) for a) 3CLPro and b) TNKS2. Dashed lines indicate standardised major axis (SMA) regressions. Data points are coloured by Boltz-2 predicted binding probabilities.
Figure 5: Binding affinity consistency between two independent runs of Boltz-2 for a) 3CLPro and b) TNKS2.
Figure 6: Comparison of saturation state deviations. Boltz-2 saturation is derived from hydrogen placement on predicted heavy-atom coordinates. Representative deviations of the saturation states from Boltz-2 predictions are shown for rings (a) and aliphatic chains (b), alongside their SMILES string counterparts (c, d).
Figure 7: Comparison of binding free energies from Boltz-2 prediction ( ΔGBoltz ) and ESMACS simulations. Three sets of ESMACS simulations are performed, denoted as ΔGBoltzESMACS_1 , ΔGBoltzESMACS_2 and ΔGDockedESMACS , respectively, initiated from structures derived directly from Boltz-2 predicted structures, derived from SMILES strings with positions of heavy atoms from Boltz-2 predictions, and from docked approach. Some compounds have different saturation or protonation states for ΔGBoltzESMACS_1 and ΔGBoltzESMACS_2 calculations; they are shown as red dots.
Designing small molecule ligands that bind with high affinity to specific protein pockets is a fundamental goal in drug discovery, as small molecules constitute a major fraction of approved therapeutics. Recent breakthroughs in structure prediction, such as AlphaFold-3 and Boltz-2, enable accurate biomolecular interaction prediction and show promise as foundation models for downstream tasks, including binding affinity prediction. We propose to leverage these models and introduce DBMol, a new structure predictor-guided framework for de novo small molecule design. DBMol formulates an alternating optimization and projection process. In the optimization stage, DBMol starts from an initial molecule and uses gradient-based optimization to improve pocket-specific interactions and predicted binding affinity using a structure prediction model. In the projection stage, a flow-matching model maps the optimized molecular graph to discrete and chemically valid molecules. Experiments show that DBMol effectively optimizes the Boltz-2 affinity proxy and generates molecules with strong predicted affinity and specificity under Boltz-2 evaluation. To reduce self-confirmation bias, we further evaluate generated molecules using held-out metrics, including AF3-based evaluation. DBMol substantially improves pocket coverage while maintaining molecular diversity over unconditional generation, and is competitive under held-out metrics despite the absence of reference-ligand supervision. These results support the promise of structure prediction models as effective optimization signals for de novo molecular design.
Yiming Qin, Kai Yi, Miruna Cretu +3
EPFL, Switzerland · MRC-LMB, UK · University of Cambridge, UK
Accurate protein-ligand binding affinity prediction is central to computational drug discovery, yet modern docking engines frequently disagree without indicating which prediction to trust. Consensus scoring and ensemble methods improve mean accuracy but treat all predictions identically without interpretable confidence measures or uncertainty decomposition, ignoring the chemical context of each protein-ligand pair. To address this limitation, we introduce RELIABLE-BA (RELIABiLity-aware Evidential fusion for Binding Affinity), an evidential framework for multi-engine binding affinity prediction. Our model comprises three steps: (1) modeling each engine as an evidential expert via Normal-Inverse-Gamma distributions, (2) scaling epistemic uncertainty through learned reliability from molecular context while preserving each expert's predictive mean, and (3) fusing experts through closed-form aggregation that captures both individual uncertainty and inter-engine disagreement. Experiments on the PDBBind and BDB2020+ benchmarks demonstrate competitive point prediction with substantially improved uncertainty calibration, and additional validation on the SARS-CoV-2 Mpro dataset and 5HT2A receptor demonstrates applicability to clinically relevant drug targets. Crucially, these uncertainty estimates enable reliable filtering of protein-ligand pairs, reducing prediction error by up to 25% when retaining only high-confidence pairs. To our knowledge, RELIABLE-BA is the first multi-engine binding affinity prediction framework to combine evidential fusion with context-dependent reliability, offering a principled path toward trustworthy AI-guided drug discovery. Our code is publicly available at https://github.com/yongchand/RELIABLE-BA.
Yongchan Hong, Defu Cao, Wenjin Liu +6
Department of Quantitative & Computational Biology University of Southern California Los Angeles, California, USA · Department of Computer Science University of Southern California Los Angeles, California, USA · Department of Chemistry University of California, Berkeley Berkeley, California, USA
Computational approaches to drug discovery involve multiple sub-problems, and among them, drug-target binding affinity prediction plays an important role. Despite recent advances, accurately predicting binding affinity remains an open research area. The major objective of our paper is to perform a comprehensive review and comparative analysis of recent machine learning methods for drug-target binding affinity prediction, with a focus on identifying strengths, limitations, and research gaps. We review representative recent deep learning approaches that use common benchmark datasets and evaluation metrics, covering a range of neural network architectures and representation strategies. In addition, we analyze seven widely used benchmark datasets and commonly adopted evaluation metrics for drug-target binding affinity prediction. Our analysis indicates that although many methods report strong performance on standard benchmarks, their effectiveness is often influenced by dataset bias and limited evaluation settings. Furthermore, most methods exhibit reduced performance in cold-start scenarios, highlighting challenges in generalization. We identify several limitations of current approaches, including dataset imbalance, the lack of standardized evaluation, limited real-world applicability, and challenges in cold-start scenarios. We also discuss future research directions, including better dataset design, more robust evaluation methods, improved handling of cold-start problems, and the integration of multimodal representations.
Jafin Khan, Md Hossain Shuvo
Department of Computer Science, Prairie View A&M University, Prairie View, Texas, USA.