Organizations: Georgia State University, Atlanta, GA, USA · Georgia Institute of Technology, Atlanta, GA, USA · University of Engineering and Technology, Lahore, Pakistan · Lahore University of Management Sciences, Pakistan
Computational antibody design has seen rapid methodological progress, with dozens of deep generative methods proposed in the past three years, yet the field lacks a standardized benchmark for fair comparison and model development. These methods are evaluated on different SAbDab snapshots, non-overlapping test sets, and incompatible metrics, and the literature fragments the design problem into numerous sub-tasks with no common definition. We introduce CHIMERA-Bench: (CDR Modeling with Epitope-guided Redesign), a unified benchmark built around a single canonical task: epitope-conditioned CDR sequence-structure co-design. CHIMERA-Bench provides three components. The first is a curated, deduplicated dataset of 2,922 antibody-antigen complexes with epitope and paratope annotations. The second is a set of three biologically motivated splits that test generalization to unseen epitopes, unseen antigen folds, and prospective temporal targets. The third is a comprehensive evaluation protocol with five metric groups, including novel epitope-specificity measures. We benchmark eleven methods spanning six generative paradigms and report results across all splits. CHIMERA-Bench is the largest dataset of its kind for the antibody design problem, allowing the community to develop and test novel methods and evaluate their generalizability.
Figures & tables
Chimera-Bench
RAbD
SAbDab
SKEMPI v2
AsEP
AbBiBench
ProteinGym
Domain
Ab design
Ab design
Ab structures
Affinity
Epitope pred.
Affinity mat.
Fitness
Complexes
2,922
60
7,000+
53 (Ab)
1,723
14 Ab / 9 Ag
–
Epitope labels
✓
–
–
–
✓
–
–
Paratope labels
✓
–
–
–
✓
–
–
CDR annotations
✓
–
–
–
–
–
–
Multiple splits
3
1
–
1
2
3
3
Table 1: Comparison of Chimera-Bench with prior antibody design benchmarks and datasets.
Figure 1: The Chimera-Bench data curation pipeline that collects antigen-antibody complexes from SAbDab and produces filtered and annotated complexes for the antibody design task.
Figure 2: Distributions across the Chimera-Bench dataset. (a) Heavy, light, and antigen chain lengths. (b) Interface statistics, namely epitope size, paratope size, and number of residue contacts per complex. (c) CDR loop lengths under IMGT numbering, where CDR-H3 shows the widest variability consistent with its role as the primary determinant of antigen specificity.
Figure 3: Antigen species and origin distributions. (a) Top 15 species by complex count. (b) Breakdown by biological origin category.
Figure 4: Epitope diversity in Chimera-Bench . (a) Distribution of distinct epitopes per antigen sequence (log scale); most antigens have a single epitope, but a long tail extends to 30. (b) Top 15 multi-epitope antigens ranked by number of distinct epitope sites.
Split
Train
Val
Test
Generalization target
Epitope-group
2,338
292
292
Unseen epitope patterns
Antigen-fold
2,338
292
292
Unseen antigen topologies
Temporal
2,337
292
293
Prospective (post-2023)
Table 2: Summary of Chimera-Bench evaluation splits.
Figure 5: Distribution of CDR-H3 length, epitope size, and antigen size (columns) across the train, validation, and test partitions of the three splits (rows).
Figure 6: CDR-H3 design metrics across all methods on the three test splits (EG: epitope-group, AF: antigen-fold, TP: temporal).
Method
AAR ↑
CAAR ↑
PPL ↓
RMSD ↓
TM ↑
Fnat ↑
iRMSD ↓
DockQ ↑
EpiF1 ↑
n_liab. ↓
RAAD
0.38 ± 0.13
0.21 ± 0.23
3.31 ± 0.47
1.95 ± 0.87
0.21 ± 0.13
0.49 ± 0.30
1.64 ± 0.86
0.64 ± 0.21
0.67 ± 0.27
0.71 ± 0.62
MEAN
0.42 ± 0.15
0.21 ± 0.23
3.00 ± 0.61
2.01 ± 0.89
0.19 ± 0.12
0.48 ± 0.31
1.66 ± 0.86
0.63 ± 0.21
0.67 ± 0.29
0.90 ± 0.92
dyMEAN
0.35 ± 0.12
0.17 ± 0.20
3.38 ± 0.46
2.60 ± 0.94
0.12 ± 0.07
0.41 ± 0.30
2.23 ± 0.84
0.56 ± 0.19
0.55 ± 0.29
0.83 ± 0.38
DiffAb
0.22 ± 0.12
0.11 ± 0.18
–
2.64 ± 1.19
0.12 ± 0.12
0.48 ± 0.32
2.27 ± 0.98
0.58 ± 0.23
0.56 ± 0.27
0.62 ± 0.77
AbFlowNet
0.22 ± 0.12
0.13 ± 0.18
–
2.70 ± 1.24
0.12 ± 0.12
0.49 ± 0.33
2.37 ± 1.01
0.57 ± 0.22
0.55 ± 0.28
0.46 ± 0.68
AbMEGD
0.22 ± 0.11
0.11 ± 0.18
–
2.76 ± 1.26
0.11 ± 0.11
0.50 ± 0.32
2.41 ± 1.04
0.57 ± 0.22
0.56 ± 0.27
0.47 ± 0.72
Table 3: CDR-H3 design results on the epitope-group test split. Best values are in bold , second-best are underlined . “–” indicates a metric that is not applicable (PPL for diffusion and flow models).
AAR ↑
RMSD ↓
EpiF1 ↑
Method
H1
H2
H3
L1
L2
L3
H1
H2
H3
L1
L2
L3
H1
H2
H3
L1
L2
L3
RAAD
0.72
0.67
0.38
–
–
–
0.59
0.51
1.95
–
–
–
0.92
0.89
0.67
–
–
–
MEAN
0.70
0.59
0.42
–
–
–
0.85
0.83
2.01
–
–
–
0.86
0.85
0.67
–
–
–
dyMEAN
0.67
0.65
0.35
–
–
–
1.07
0.99
2.60
–
–
–
0.80
0.79
0.55
–
–
–
DiffAb
0.57
0.29
0.22
0.55
0.50
0.43
0.87
0.67
2.64
1.03
2.03
0.96
0.83
0.84
0.56
0.80
0.93
0.66
AbFlowNet
0.56
0.31
0.22
0.54
0.51
0.40
0.79
0.65
2.70
0.99
0.49
0.96
0.84
0.84
0.55
0.82
0.92
0.65
Table 4: Per-CDR results on the epitope-group test split. Mean AAR, RMSD, and EpiF1 are shown per CDR type. “–” indicates the method does not support the given CDR. Best values are in bold , second-best are underlined .
Figure 7: CDR-H3 design quality on the epitope-group split, stratified by three covariates (rows: CDR-H3 length, antigen size, epitope size) across three metrics (columns: sequence recovery AAR, backbone accuracy RMSD, native-contact recovery Fnat). Loop length is the dominant difficulty axis, with all three metrics degrading as the loop lengthens, while antigen and epitope size leave AAR nearly flat.
Params
Train (GPU-h)
Infer (min)
Method
Type
Total
Train.
EG
AF
TP
EG
AF
TP
MEAN
Equiv. GNN
0.7M
0.7M
3.5
2.6
2.4
0.5
0.5
0.6
RAAD
Equiv. GNN
6.8M
6.8M
2.8
3.0
2.1
0.8
0.7
0.5
dyMEAN
Equiv. GNN
2.0M
2.0M
15.3
14.5
16.5
0.7
1.4
0.6
RefineGNN
Autoreg. GNN
5.8M
5.8M
3.5
3.2
3.1
2.8
3.2
3.2
dyAb
Flow matching
2.5M
2.5M
1.1
0.7
0.7
2.2
1.6
1.6
Table 5: Computational resources for all evaluated methods. Params are total and trainable parameter counts. Train is total wall-clock training time in GPU-hours summed over all CDR models a method trains, and Infer is total test-set inference time in minutes, both reported per split (epitope group: EG, antigen fold: AF, temporal: TP).
Figure 8: Native (a) and predicted CDR-H3 backbone structures for three complexes of increasing difficulty: 7JKS (H3 length 14), 4J4P (H3 length 17), and 4XMP (H3 length 25). Panels show predictions from (b) AbFlowNet, (c) AbODE, (d) DiffAb, (e) dyAb, (f) dyMEAN, (g) RAAD, (h) RefineGNN. The native CDR H3 regions are shown in cyan colors.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Parameter
Description
Value
Max resolution
Structure quality threshold
4.0 Å
Contact cutoff
Epitope/paratope definition
4.5 Å
Sequence identity
MMseqs2 clustering threshold
95%
Coverage
MMseqs2 alignment coverage
80%
Numbering schemes
Residue numbering
IMGT, Chothia
Graph k -NN
Nearest neighbors for intra-chain edges
10
Appendix
Table 6: Pipeline configuration parameters.
Scheme
H1
H2
H3
L1
L2
L3
IMGT ( Lefranc et al., 2003 )
27–38
56–65
105–117
27–38
56–65
105–117
Chothia ( Chothia and Lesk, 1987 )
26–32
52–56
95–102
24–34
50–56
89–97
Appendix
Table 7: CDR boundary definitions used in Chimera-Bench .
Method
AAR ↑
CAAR ↑
PPL ↓
RMSD ↓
TM ↑
Fnat ↑
iRMSD ↓
DockQ ↑
EpiF1 ↑
n_liab. ↓
RAAD ( Wu et al., 2025 )
0.38 ± 0.13
0.20 ± 0.21
3.06 ± 0.49
1.65 ± 0.70
0.24 ± 0.13
0.62 ± 0.31
1.42 ± 0.62
0.70 ± 0.21
0.78 ± 0.25
0.59 ± 0.60
MEAN ( Kong et al., 2022 )
0.40 ± 0.14
0.20 ± 0.23
2.79 ± 0.57
1.66 ± 0.64
0.23 ± 0.11
0.62 ± 0.32
1.42 ± 0.61
0.70 ± 0.21
0.78 ± 0.26
0.38 ± 0.70
dyMEAN ( Kong et al., 2023 )
0.37 ± 0.13
0.20 ± 0.23
3.35 ± 0.66
2.06 ± 0.83
0.16 ± 0.09
0.61 ± 0.31
1.79 ± 0.82
0.66 ± 0.20
0.75 ± 0.25
0.21 ± 0.41
DiffAb ( Luo et al., 2022 )
0.23 ± 0.12
0.10 ± 0.15
–
2.24 ± 1.02
0.15 ± 0.13
0.52 ± 0.34
2.10 ± 0.89
0.58 ± 0.25
0.65 ± 0.30
0.70 ± 0.77
AbFlowNet ( Abir et al., 2025 )
0.23 ± 0.12
0.13 ± 0.18
–
2.70 ± 5.36
0.16 ± 0.14
0.56 ± 0.34
2.46 ± 4.31
0.60 ± 0.26
0.65 ± 0.30
0.61 ± 0.81
AbMEGD ( Chen et al., 2025 )
0.24 ± 0.13
0.13 ± 0.20
–
2.25 ± 1.06
0.16 ± 0.15
0.55 ± 0.34
2.13 ± 0.94
0.59 ± 0.25
0.65 ± 0.30
0.64 ± 0.76
Appendix
Table 8: CDR-H3 design results on the antigen-fold test split. Best values are in bold , second-best are underlined .
Method
AAR ↑
CAAR ↑
PPL ↓
RMSD ↓
TM ↑
Fnat ↑
iRMSD ↓
DockQ ↑
EpiF1 ↑
n_liab. ↓
RAAD ( Wu et al., 2025 )
0.38 ± 0.13
0.20 ± 0.22
3.24 ± 0.44
1.81 ± 0.76
0.22 ± 0.12
0.61 ± 0.28
1.46 ± 0.70
0.71 ± 0.16
0.74 ± 0.23
0.77 ± 0.65
MEAN ( Kong et al., 2022 )
0.39 ± 0.13
0.22 ± 0.23
3.31 ± 0.53
1.84 ± 0.68
0.20 ± 0.09
0.60 ± 0.29
1.44 ± 0.66
0.71 ± 0.16
0.73 ± 0.23
0.66 ± 0.66
dyMEAN ( Kong et al., 2023 )
0.35 ± 0.13
0.20 ± 0.22
3.28 ± 0.42
2.32 ± 0.97
0.15 ± 0.10
0.50 ± 0.30
2.00 ± 0.92
0.63 ± 0.17
0.64 ± 0.26
0.99 ± 0.71
DiffAb ( Luo et al., 2022 )
0.23 ± 0.11
0.12 ± 0.17
–
2.46 ± 1.05
0.13 ± 0.10
0.51 ± 0.30
2.18 ± 0.95
0.61 ± 0.19
0.61 ± 0.26
0.74 ± 0.84
AbFlowNet ( Abir et al., 2025 )
0.23 ± 0.12
0.13 ± 0.19
–
2.67 ± 2.79
0.13 ± 0.11
0.52 ± 0.32
2.33 ± 1.98
0.61 ± 0.20
0.61 ± 0.26
0.45 ± 0.70
AbMEGD ( Chen et al., 2025 )
0.23 ± 0.11
0.12 ± 0.17
–
2.72 ± 2.42
0.12 ± 0.10
0.55 ± 0.31
2.35 ± 1.67
0.62 ± 0.20
0.58 ± 0.26
0.63 ± 0.83
Appendix
Table 9: CDR-H3 design results on the temporal test split. Best values are in bold , second-best are underlined .
AAR ↑
Fnat ↑
DockQ ↑
EpiF1 ↑
Method
EG
AF
EG
AF
EG
AF
EG
AF
RAAD
0.38
0.38
0.49
0.62
0.64
0.70
0.67
0.78
MEAN
0.42
0.40
0.48
0.62
0.63
0.70
0.67
0.78
DiffAb
0.22
0.23
0.48
0.52
0.58
0.58
0.56
0.65
AbFlowNet
0.22
0.23
0.49
0.56
0.57
0.60
0.55
0.65
AbMEGD
0.22
0.24
0.50
0.55
0.57
0.59
0.56
0.65
Appendix
Table 10: Generalization gap on CDR-H3. For each method, we report the epitope-group (EG, unseen epitope clusters) and antigen-fold (AF, unseen antigen folds) test splits. Sequence recovery (AAR) is nearly identical across the two splits, while the interface and epitope metrics (Fnat, DockQ, EpiF1) are consistently lower on the harder epitope-group split.
Figure 9: Per-CDR comparison of methods across all evaluation metrics for (a) CDR-H1, (b) CDR-H2, and (c) CDR-H3. Each axis is scaled so that a larger radius is better; RMSD and iRMSD (marked with an asterisk) are inverted accordingly.
Method
AAR
RMSD
TM
lDDT
DockQ
PPL
dG/IMP
CAAR
MEAN ( Kong et al., 2022 )
AAR
K
✓
–
–
✓
ddG
–
dyMEAN ( Kong et al., 2023 )
AAR
K
✓
✓
✓
–
ddG
✓
RAAD ( Wu et al., 2025 )
AAR
K
✓
–
–
✓
–
–
RefineGNN ( Jin et al., 2021 )
AAR
K
–
–
–
✓
–
–
AbODE ( Verma et al., 2023 )
AAR
K
–
–
–
✓
–
–
AbDockGen ( Jin et al., 2022 )
AAR
K
–
–
✓
✓
–
–
Appendix
Table 11: Metrics originally reported by each method (prior to Chimera-Bench standardization).
Antibody design methods condition on antigen structure to generate complementarity-determining regions (CDR), yet a systematic evaluation of baseline methods reveals that they largely ignore the antigen input. We identify three failure modes that explain this behavior. Antigen blindness arises because models derive predictions from antibody framework context rather than antigen information, producing nearly identical CDRs regardless of the target. Vocabulary collapse reduces predicted amino acids to three to five per position, far below the ground truth distribution in native sequences. Moreover, any model trained with standard per-position cross-entropy converges to the positional marginal distribution, making it provably unable to produce antigen-specific sequence predictions. We propose a novel encoder-decoder architecture called AgForce, that uses a graph neural network (GNN) as the encoder and specialized decoders for sequence-structure co-design. Specifically, we apply framework dropout, gated bottlenecks, and hyperbolic cross attention that prevent the antibody shortcut path. In the decoder, a Mixture Density Network (MDN) sequence head with Potts-like pairwise coupling and annealed Multiple Choice Learning (aMCL) replaces the cross-entropy objective with a multi-component distribution whose optimal solution differs from the positional marginal. An antigen cycle consistency head routes gradients through the sequence decoder, forcing predicted distributions to encode antigen identity. AgForce achieves the best binding quality and sequence recovery simultaneously on the CHIMERA-Bench dataset, improving amino acid recovery by 8% over the strongest sequence baseline while surpassing the baselines across all interface metrics, and nearly doubling the effective vocabulary of GNN methods. The source code is available at: https://github.com/mansoor181/ag-force.git
Mansoor Ahmed, Murray Patterson
1Georgia State University, Atlanta, GA, USA · 2Georgia Institute of Technology, Atlanta, GA, USA
Epitopes determine where antibodies bind antigens and shape downstream therapeutic properties such as functional blockade and escape resistance, making epitope understanding central to antibody drug discovery. Although large language models (LLMs) have shown strong biomedical reasoning ability, it remains unclear whether they can infer epitope information directly from antigen and antibody sequences. Existing epitope resources typically focus on isolated prediction tasks or rely on specialized structural settings, while general protein benchmarks do not evaluate epitope-centered decisions across the antibody development workflow. To address this gap, we introduce EpiBench, a closed-book, sequence-based, and automatically scorable benchmark for evaluating epitope reasoning in LLMs. EpiBench contains 1,609 curated samples grounded in structural antibody--antigen contacts, curated functional B-cell assays, and deep mutational scanning escape measurements. It covers five connected tasks: targetable region discovery, antibody-conditioned epitope identification, epitope binning, functional epitope assessment, and antibody escape assessment, with controlled sampling to reduce shortcut-based evaluation artifacts. We evaluate nine general-purpose LLMs and analyze their behavior through task-specific baselines, antigen length stratification, explicit-reasoning comparison, and failure-mode inspection. The results show that current LLMs capture partial epitope-related signals but remain limited in antibody-specific sequence grounding, long-context residue localization, and biologically grounded reasoning. Therefore, EpiBench provides a diagnostic testbed for measuring and improving sequence-aware biomedical LLMs toward reliable LLM-assisted antibody discovery.
Computational antibody CDR design methods condition on antigen structure to generate binding loops. Yet, the existing architectures conflate two fundamentally distinct sub-problems: identifying which CDR positions will contact the antigen, and selecting amino acids at those positions. This forces models to learn contact reasoning implicitly through uniform message passing, diluting antigen signal across all positions equally. We introduce ConTact, a contact-then-act architecture that explicitly decomposes CDR design into three cascaded stages: learning surface complementarity fingerprints, predicting CDR-antigen contacts, and injecting contact-gated antigen features into the prediction head. A distance-biased cross-attention module encodes geometric priors favoring spatial neighbors, while a contact-weighted cross-entropy loss concentrates gradient signal on binding-critical positions. On the CHIMERA-Bench dataset, ConTact achieves the lowest backbone RMSD on every split (a 5 to 6% improvement over the best baseline) and the best fraction of native contacts, interface RMSD, and epitope F1 on the antigen-fold and temporal splits, while remaining competitive on the harder epitope-group split. The source code is available at: https://github.com/mansoor181/ConTact.git
Mansoor Ahmed, Spencer VonBank, Nadeem Taj +3
Georgia State University, Atlanta, USA · Georgia Institute of Technology, Atlanta, USA · DePauw University, Indiana, USA +2