CHIMERA-Bench: A Benchmark Dataset for Epitope-Specific Antibody Design

Mansoor Ahmed; Nadeem Taj; Imdad Ullah Khan; Hemanth Venkateswara; Murray Patterson

cs.LGMar 13, 2026

CHIMERA-Bench: A Benchmark Dataset for Epitope-Specific Antibody Design

Authors: Mansoor Ahmed, Nadeem Taj, Imdad Ullah Khan, Hemanth Venkateswara, Murray Patterson

Organizations: Georgia State University, Atlanta, GA, USA · Georgia Institute of Technology, Atlanta, GA, USA · University of Engineering and Technology, Lahore, Pakistan · Lahore University of Management Sciences, Pakistan

Abstract

Computational antibody design has seen rapid methodological progress, with dozens of deep generative methods proposed in the past three years, yet the field lacks a standardized benchmark for fair comparison and model development. These methods are evaluated on different SAbDab snapshots, non-overlapping test sets, and incompatible metrics, and the literature fragments the design problem into numerous sub-tasks with no common definition. We introduce CHIMERA-Bench: (CDR Modeling with Epitope-guided Redesign), a unified benchmark built around a single canonical task: epitope-conditioned CDR sequence-structure co-design. CHIMERA-Bench provides three components. The first is a curated, deduplicated dataset of 2,922 antibody-antigen complexes with epitope and paratope annotations. The second is a set of three biologically motivated splits that test generalization to unseen epitopes, unseen antigen folds, and prospective temporal targets. The third is a comprehensive evaluation protocol with five metric groups, including novel epitope-specificity measures. We benchmark eleven methods spanning six generative paradigms and report results across all splits. CHIMERA-Bench is the largest dataset of its kind for the antibody design problem, allowing the community to develop and test novel methods and evaluate their generalizability.

Figures & tables

	Chimera-Bench	RAbD	SAbDab	SKEMPI v2	AsEP	AbBiBench	ProteinGym
Domain	Ab design	Ab design	Ab structures	Affinity	Epitope pred.	Affinity mat.	Fitness
Complexes	2,922	60	7,000+	53 (Ab)	1,723	14 Ab / 9 Ag	–
Epitope labels	✓	–	–	–	✓	–	–
Paratope labels	✓	–	–	–	✓	–	–
CDR annotations	✓	–	–	–	–	–	–
Multiple splits	3	1	–	1	2	3	3

Table 1: Comparison of Chimera-Bench with prior antibody design benchmarks and datasets.

Panel of Figure 1 — Figure 1: The Chimera-Bench data curation pipeline that collects antigen-antibody complexes from SAbDab and produces filtered and annotated complexes for the antibody design task.

Figure 2: Distributions across the Chimera-Bench dataset. (a) Heavy, light, and antigen chain lengths. (b) Interface statistics, namely epitope size, paratope size, and number of residue contacts per complex. (c) CDR loop lengths under IMGT numbering, where CDR-H3 shows the widest variability consistent with its role as the primary determinant of antigen specificity.

Figure 3: Antigen species and origin distributions. (a) Top 15 species by complex count. (b) Breakdown by biological origin category.

Figure 4: Epitope diversity in Chimera-Bench . (a) Distribution of distinct epitopes per antigen sequence (log scale); most antigens have a single epitope, but a long tail extends to 30. (b) Top 15 multi-epitope antigens ranked by number of distinct epitope sites.

Split	Train	Val	Test	Generalization target
Epitope-group	2,338	292	292	Unseen epitope patterns
Antigen-fold	2,338	292	292	Unseen antigen topologies
Temporal	2,337	292	293	Prospective (post-2023)

Table 2: Summary of Chimera-Bench evaluation splits.

Figure 5: Distribution of CDR-H3 length, epitope size, and antigen size (columns) across the train, validation, and test partitions of the three splits (rows).

Figure 6: CDR-H3 design metrics across all methods on the three test splits (EG: epitope-group, AF: antigen-fold, TP: temporal).

Method	AAR $\uparrow$	CAAR $\uparrow$	PPL $\downarrow$	RMSD $\downarrow$	TM $\uparrow$	Fnat $\uparrow$	iRMSD $\downarrow$	DockQ $\uparrow$	EpiF1 $\uparrow$	n_liab. $\downarrow$
RAAD	0.38 $\pm$ 0.13	0.21 $\pm$ 0.23	3.31 $\pm$ 0.47	1.95 $\pm$ 0.87	0.21 $\pm$ 0.13	0.49 $\pm$ 0.30	1.64 $\pm$ 0.86	0.64 $\pm$ 0.21	0.67 $\pm$ 0.27	0.71 $\pm$ 0.62
MEAN	0.42 $\pm$ 0.15	0.21 $\pm$ 0.23	3.00 $\pm$ 0.61	2.01 $\pm$ 0.89	0.19 $\pm$ 0.12	0.48 $\pm$ 0.31	1.66 $\pm$ 0.86	0.63 $\pm$ 0.21	0.67 $\pm$ 0.29	0.90 $\pm$ 0.92
dyMEAN	0.35 $\pm$ 0.12	0.17 $\pm$ 0.20	3.38 $\pm$ 0.46	2.60 $\pm$ 0.94	0.12 $\pm$ 0.07	0.41 $\pm$ 0.30	2.23 $\pm$ 0.84	0.56 $\pm$ 0.19	0.55 $\pm$ 0.29	0.83 $\pm$ 0.38
DiffAb	0.22 $\pm$ 0.12	0.11 $\pm$ 0.18	–	2.64 $\pm$ 1.19	0.12 $\pm$ 0.12	0.48 $\pm$ 0.32	2.27 $\pm$ 0.98	0.58 $\pm$ 0.23	0.56 $\pm$ 0.27	0.62 $\pm$ 0.77
AbFlowNet	0.22 $\pm$ 0.12	0.13 $\pm$ 0.18	–	2.70 $\pm$ 1.24	0.12 $\pm$ 0.12	0.49 $\pm$ 0.33	2.37 $\pm$ 1.01	0.57 $\pm$ 0.22	0.55 $\pm$ 0.28	0.46 $\pm$ 0.68
AbMEGD	0.22 $\pm$ 0.11	0.11 $\pm$ 0.18	–	2.76 $\pm$ 1.26	0.11 $\pm$ 0.11	0.50 $\pm$ 0.32	2.41 $\pm$ 1.04	0.57 $\pm$ 0.22	0.56 $\pm$ 0.27	0.47 $\pm$ 0.72

Table 3: CDR-H3 design results on the epitope-group test split. Best values are in bold , second-best are underlined . “–” indicates a metric that is not applicable (PPL for diffusion and flow models).

	AAR $\uparrow$						RMSD $\downarrow$						EpiF1 $\uparrow$
Method	H1	H2	H3	L1	L2	L3	H1	H2	H3	L1	L2	L3	H1	H2	H3	L1	L2	L3
RAAD	0.72	0.67	0.38	–	–	–	0.59	0.51	1.95	–	–	–	0.92	0.89	0.67	–	–	–
MEAN	0.70	0.59	0.42	–	–	–	0.85	0.83	2.01	–	–	–	0.86	0.85	0.67	–	–	–
dyMEAN	0.67	0.65	0.35	–	–	–	1.07	0.99	2.60	–	–	–	0.80	0.79	0.55	–	–	–
DiffAb	0.57	0.29	0.22	0.55	0.50	0.43	0.87	0.67	2.64	1.03	2.03	0.96	0.83	0.84	0.56	0.80	0.93	0.66
AbFlowNet	0.56	0.31	0.22	0.54	0.51	0.40	0.79	0.65	2.70	0.99	0.49	0.96	0.84	0.84	0.55	0.82	0.92	0.65

Table 4: Per-CDR results on the epitope-group test split. Mean AAR, RMSD, and EpiF1 are shown per CDR type. “–” indicates the method does not support the given CDR. Best values are in bold , second-best are underlined .

Figure 7: CDR-H3 design quality on the epitope-group split, stratified by three covariates (rows: CDR-H3 length, antigen size, epitope size) across three metrics (columns: sequence recovery AAR, backbone accuracy RMSD, native-contact recovery Fnat). Loop length is the dominant difficulty axis, with all three metrics degrading as the loop lengthens, while antigen and epitope size leave AAR nearly flat.

Method	Type	Total	Train.	EG	AF	TP	EG	AF	TP
		Params		Train (GPU-h)			Infer (min)
MEAN	Equiv. GNN	0.7M	0.7M	3.5	2.6	2.4	0.5	0.5	0.6
RAAD	Equiv. GNN	6.8M	6.8M	2.8	3.0	2.1	0.8	0.7	0.5
dyMEAN	Equiv. GNN	2.0M	2.0M	15.3	14.5	16.5	0.7	1.4	0.6
RefineGNN	Autoreg. GNN	5.8M	5.8M	3.5	3.2	3.1	2.8	3.2	3.2
dyAb	Flow matching	2.5M	2.5M	1.1	0.7	0.7	2.2	1.6	1.6

Table 5: Computational resources for all evaluated methods. Params are total and trainable parameter counts. Train is total wall-clock training time in GPU-hours summed over all CDR models a method trains, and Infer is total test-set inference time in minutes, both reported per split (epitope group: EG, antigen fold: AF, temporal: TP).

Panel of Figure 8 — Figure 8: Native (a) and predicted CDR-H3 backbone structures for three complexes of increasing difficulty: 7JKS (H3 length 14), 4J4P (H3 length 17), and 4XMP (H3 length 25). Panels show predictions from (b) AbFlowNet, (c) AbODE, (d) DiffAb, (e) dyAb, (f) dyMEAN, (g) RAAD, (h) RefineGNN. The native CDR H3 regions are shown in cyan colors.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Description	Value
Max resolution	Structure quality threshold	4.0 Å
Contact cutoff	Epitope/paratope definition	4.5 Å
Sequence identity	MMseqs2 clustering threshold	95%
Coverage	MMseqs2 alignment coverage	80%
Numbering schemes	Residue numbering	IMGT, Chothia
Graph $k$ -NN	Nearest neighbors for intra-chain edges	10

Appendix

Table 6: Pipeline configuration parameters.

Scheme	H1	H2	H3	L1	L2	L3
IMGT ( Lefranc et al., 2003 )	27–38	56–65	105–117	27–38	56–65	105–117
Chothia ( Chothia and Lesk, 1987 )	26–32	52–56	95–102	24–34	50–56	89–97

Appendix

Table 7: CDR boundary definitions used in Chimera-Bench .

Method	AAR $\uparrow$	CAAR $\uparrow$	PPL $\downarrow$	RMSD $\downarrow$	TM $\uparrow$	Fnat $\uparrow$	iRMSD $\downarrow$	DockQ $\uparrow$	EpiF1 $\uparrow$	n_liab. $\downarrow$
RAAD ( Wu et al., 2025 )	0.38 $\pm$ 0.13	0.20 $\pm$ 0.21	3.06 $\pm$ 0.49	1.65 $\pm$ 0.70	0.24 $\pm$ 0.13	0.62 $\pm$ 0.31	1.42 $\pm$ 0.62	0.70 $\pm$ 0.21	0.78 $\pm$ 0.25	0.59 $\pm$ 0.60
MEAN ( Kong et al., 2022 )	0.40 $\pm$ 0.14	0.20 $\pm$ 0.23	2.79 $\pm$ 0.57	1.66 $\pm$ 0.64	0.23 $\pm$ 0.11	0.62 $\pm$ 0.32	1.42 $\pm$ 0.61	0.70 $\pm$ 0.21	0.78 $\pm$ 0.26	0.38 $\pm$ 0.70
dyMEAN ( Kong et al., 2023 )	0.37 $\pm$ 0.13	0.20 $\pm$ 0.23	3.35 $\pm$ 0.66	2.06 $\pm$ 0.83	0.16 $\pm$ 0.09	0.61 $\pm$ 0.31	1.79 $\pm$ 0.82	0.66 $\pm$ 0.20	0.75 $\pm$ 0.25	0.21 $\pm$ 0.41
DiffAb ( Luo et al., 2022 )	0.23 $\pm$ 0.12	0.10 $\pm$ 0.15	–	2.24 $\pm$ 1.02	0.15 $\pm$ 0.13	0.52 $\pm$ 0.34	2.10 $\pm$ 0.89	0.58 $\pm$ 0.25	0.65 $\pm$ 0.30	0.70 $\pm$ 0.77
AbFlowNet ( Abir et al., 2025 )	0.23 $\pm$ 0.12	0.13 $\pm$ 0.18	–	2.70 $\pm$ 5.36	0.16 $\pm$ 0.14	0.56 $\pm$ 0.34	2.46 $\pm$ 4.31	0.60 $\pm$ 0.26	0.65 $\pm$ 0.30	0.61 $\pm$ 0.81
AbMEGD ( Chen et al., 2025 )	0.24 $\pm$ 0.13	0.13 $\pm$ 0.20	–	2.25 $\pm$ 1.06	0.16 $\pm$ 0.15	0.55 $\pm$ 0.34	2.13 $\pm$ 0.94	0.59 $\pm$ 0.25	0.65 $\pm$ 0.30	0.64 $\pm$ 0.76

Appendix

Table 8: CDR-H3 design results on the antigen-fold test split. Best values are in bold , second-best are underlined .

Method	AAR $\uparrow$	CAAR $\uparrow$	PPL $\downarrow$	RMSD $\downarrow$	TM $\uparrow$	Fnat $\uparrow$	iRMSD $\downarrow$	DockQ $\uparrow$	EpiF1 $\uparrow$	n_liab. $\downarrow$
RAAD ( Wu et al., 2025 )	0.38 $\pm$ 0.13	0.20 $\pm$ 0.22	3.24 $\pm$ 0.44	1.81 $\pm$ 0.76	0.22 $\pm$ 0.12	0.61 $\pm$ 0.28	1.46 $\pm$ 0.70	0.71 $\pm$ 0.16	0.74 $\pm$ 0.23	0.77 $\pm$ 0.65
MEAN ( Kong et al., 2022 )	0.39 $\pm$ 0.13	0.22 $\pm$ 0.23	3.31 $\pm$ 0.53	1.84 $\pm$ 0.68	0.20 $\pm$ 0.09	0.60 $\pm$ 0.29	1.44 $\pm$ 0.66	0.71 $\pm$ 0.16	0.73 $\pm$ 0.23	0.66 $\pm$ 0.66
dyMEAN ( Kong et al., 2023 )	0.35 $\pm$ 0.13	0.20 $\pm$ 0.22	3.28 $\pm$ 0.42	2.32 $\pm$ 0.97	0.15 $\pm$ 0.10	0.50 $\pm$ 0.30	2.00 $\pm$ 0.92	0.63 $\pm$ 0.17	0.64 $\pm$ 0.26	0.99 $\pm$ 0.71
DiffAb ( Luo et al., 2022 )	0.23 $\pm$ 0.11	0.12 $\pm$ 0.17	–	2.46 $\pm$ 1.05	0.13 $\pm$ 0.10	0.51 $\pm$ 0.30	2.18 $\pm$ 0.95	0.61 $\pm$ 0.19	0.61 $\pm$ 0.26	0.74 $\pm$ 0.84
AbFlowNet ( Abir et al., 2025 )	0.23 $\pm$ 0.12	0.13 $\pm$ 0.19	–	2.67 $\pm$ 2.79	0.13 $\pm$ 0.11	0.52 $\pm$ 0.32	2.33 $\pm$ 1.98	0.61 $\pm$ 0.20	0.61 $\pm$ 0.26	0.45 $\pm$ 0.70
AbMEGD ( Chen et al., 2025 )	0.23 $\pm$ 0.11	0.12 $\pm$ 0.17	–	2.72 $\pm$ 2.42	0.12 $\pm$ 0.10	0.55 $\pm$ 0.31	2.35 $\pm$ 1.67	0.62 $\pm$ 0.20	0.58 $\pm$ 0.26	0.63 $\pm$ 0.83

Appendix

Table 9: CDR-H3 design results on the temporal test split. Best values are in bold , second-best are underlined .

	AAR $\uparrow$		Fnat $\uparrow$		DockQ $\uparrow$		EpiF1 $\uparrow$
Method	EG	AF	EG	AF	EG	AF	EG	AF
RAAD	0.38	0.38	0.49	0.62	0.64	0.70	0.67	0.78
MEAN	0.42	0.40	0.48	0.62	0.63	0.70	0.67	0.78
DiffAb	0.22	0.23	0.48	0.52	0.58	0.58	0.56	0.65
AbFlowNet	0.22	0.23	0.49	0.56	0.57	0.60	0.55	0.65
AbMEGD	0.22	0.24	0.50	0.55	0.57	0.59	0.56	0.65

Appendix

Table 10: Generalization gap on CDR-H3. For each method, we report the epitope-group (EG, unseen epitope clusters) and antigen-fold (AF, unseen antigen folds) test splits. Sequence recovery (AAR) is nearly identical across the two splits, while the interface and epitope metrics (Fnat, DockQ, EpiF1) are consistently lower on the harder epitope-group split.

Figure 9: Per-CDR comparison of methods across all evaluation metrics for (a) CDR-H1, (b) CDR-H2, and (c) CDR-H3. Each axis is scaled so that a larger radius is better; RMSD and iRMSD (marked with an asterisk) are inverted accordingly.

Method	AAR	RMSD	TM	lDDT	DockQ	PPL	dG/IMP	CAAR
MEAN ( Kong et al., 2022 )	AAR	K	✓	–	–	✓	ddG	–
dyMEAN ( Kong et al., 2023 )	AAR	K	✓	✓	✓	–	ddG	✓
RAAD ( Wu et al., 2025 )	AAR	K	✓	–	–	✓	–	–
RefineGNN ( Jin et al., 2021 )	AAR	K	–	–	–	✓	–	–
AbODE ( Verma et al., 2023 )	AAR	K	–	–	–	✓	–	–
AbDockGen ( Jin et al., 2022 )	AAR	K	–	–	✓	✓	–	–

Appendix

Table 11: Metrics originally reported by each method (prior to Chimera-Bench standardization).

Benchmark Design

Benchmark design creates tasks, datasets, metrics, and protocols that make AI capabilities measurable and comparisons credible.

See all 391 papers

AgForce Enables Antigen-conditioned Generative Antibody Design
May 20, 2026Mansoor Ahmed, Murray PattersonProtein Design Conditional Generative Modeling
1Georgia State University, Atlanta, GA, USA · 2Georgia Institute of Technology, Atlanta, GA, USA
EpiBench: Can LLMs Understand Epitopes for Antibody Drug Discovery?
Aug 6, 2026Zirui Wang, Jiaqi Wang, Qinghan Wang +4LLM Evaluation Drug Discovery
1Valhalla Technology · 2Zhejiang University
ConTact: Contact-First Antibody CDR Design via Explicit Interface Reasoning
May 20, 2026Mansoor Ahmed, Spencer VonBank, Nadeem Taj +3Protein Design
Georgia State University, Atlanta, USA · Georgia Institute of Technology, Atlanta, USA · DePauw University, Indiana, USA +2

May 20, 2026J/K move · Enter open · S save

AgForce Enables Antigen-conditioned Generative Antibody Design

Mansoor Ahmed, Murray Patterson

1Georgia State University, Atlanta, GA, USA · 2Georgia Institute of Technology, Atlanta, GA, USA

Protein Design Conditional Generative Modeling

Open paper AskPDF arXiv

Antibody design methods condition on antigen structure to generate complementarity-determining regions (CDR), yet a systematic evaluation of baseline methods reveals that they largely ignore the antigen input. We identify three failure modes that explain this behavior. Antigen blindness arises because models derive predictions from antibody framework context rather than antigen information, producing nearly identical CDRs regardless of the target. Vocabulary collapse reduces predicted amino acids to three to five per position, far below the ground truth distribution in native sequences. Moreover, any model trained with standard per-position cross-entropy converges to the positional marginal distribution, making it provably unable to produce antigen-specific sequence predictions. We propose a novel encoder-decoder architecture called AgForce, that uses a graph neural network (GNN) as the encoder and specialized decoders for sequence-structure co-design. Specifically, we apply framework dropout, gated bottlenecks, and hyperbolic cross attention that prevent the antibody shortcut path. In the decoder, a Mixture Density Network (MDN) sequence head with Potts-like pairwise coupling and annealed Multiple Choice Learning (aMCL) replaces the cross-entropy objective with a multi-component distribution whose optimal solution differs from the positional marginal. An antigen cycle consistency head routes gradients through the sequence decoder, forcing predicted distributions to encode antigen identity. AgForce achieves the best binding quality and sequence recovery simultaneously on the CHIMERA-Bench dataset, improving amino acid recovery by 8% over the strongest sequence baseline while surpassing the baselines across all interface metrics, and nearly doubling the effective vocabulary of GNN methods. The source code is available at: https://github.com/mansoor181/ag-force.git

Abstract

Benchmark Design

Explore similar work

AgForce Enables Antigen-conditioned Generative Antibody Design

EpiBench: Can LLMs Understand Epitopes for Antibody Drug Discovery?

ConTact: Contact-First Antibody CDR Design via Explicit Interface Reasoning