Spectral Coherence Index: A Model-Free Metric for Protein Structural Ensemble Quality Assessment

Yuda Bi; Huaiwen Zhang; Jingnan Sun; Vince D Calhoun

q-bio.QMMar 26, 2026

Spectral Coherence Index: A Model-Free Metric for Protein Structural Ensemble Quality Assessment

Authors: Yuda Bi, Huaiwen Zhang, Jingnan Sun, Vince D Calhoun

Organizations: Tri-institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State University, Georgia Institute of Technology, and Emory University, Atlanta, GA 30303 USA · School of Computer Science and Technology, Xidian University, Xi’an 710126, China · Departments of Biomedical Engineering, Johns Hopkins University, Baltimore, MD 21287 USA · Tri-institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State University, Georgia Institute of Technology, and Emory University, Atlanta, GA 30303 USA, and also with the School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332 USA

Abstract

Protein structural ensembles from NMR spectroscopy capture biologically important conformational heterogeneity, but it remains difficult to determine whether observed variation reflects coordinated motion or noise-like artifacts. We evaluate the Spectral Coherence Index (SCI), a model-free, rotation-invariant summary derived from the participation-ratio effective rank of the inter-model pairwise distance-variance matrix. Under grouped primary analysis of a Main110 cohort of 110 NMR ensembles (30--403 residues; 10--30 models per entry), SCI separated experimental ensembles from matched synthetic incoherent controls with AUC-ROC $= 0.973$ and Cliff's $δ= -0.945$ . Relative to an internal 27-protein pilot, discrimination softened modestly, showing that pilot-era thresholds do not transfer perfectly to a larger, more heterogeneous cohort: the primary operating point $τ= 0.811$ yielded 95.5% sensitivity and 89.1% specificity. PDB-level sensitivity remained nearly unchanged (AUC $= 0.972$ ), and an independent 11-protein holdout reached AUC $= 0.983$ . Across 5-fold grouped stratified cross-validation and leave-one-function-class-out testing, SCI remained strong (AUC $= 0.968$ and $0.971$ ), although $σ_{R_g}$ was the stronger single-feature discriminator and a QC-augmented multifeature model generalized best (AUC $= 0.989$ and $0.990$ ). Residue-level validation linked SCI-derived contributions to experimental RMSF across 110 proteins and showed broad concordance with GNM-based flexibility patterns. Rescue analyses showed that Main110 softening arose mainly from size and ensemble normalization rather than from loss of spectral signal. Together, these results establish SCI as an interpretable, bounded coherence summary that is most useful when embedded in a multimetric QC workflow for heterogeneous protein ensembles.

Figures & tables

Fig. 1: Conceptual overview of the SCI workflow. Starting from an NMR ensemble of $M$ conformers, pairwise distance variation is summarized in the distance-variance matrix $V$ , eigendecomposed to obtain the normalized positive-spectrum weights $\tau_{k}$ , compressed into the bounded score $\mathrm{SCI}=1-r_{\mathrm{eff}}/d$ , and finally interpreted jointly with $\sigma_{R_{g}}$ in a multimetric QC plane. The SCI anchors shown in Stage 4 correspond to the current Main110 mean SCI for NMR ensembles (0.870) and matched synthetic incoherent controls (0.533), and the QC plane in Stage 5 uses the current Main110 data with the manuscript’s screening thresholds.

Cohort	Role	$n$	Residues	Models per entry
Main110 NMR	grouped primary and PDB sensitivity	110	30–403	10–30 (median 20)
Holdout NMR	independent external validation	11	27–75	10–25 (median 20)
AlphaFold proxy	reference-only single-structure analysis	40	matched main-cohort subset	1
Excluded candidates	unresolved mapping or hard QC failure	4	—	—

TABLE I: Expanded cohort summary used in this manuscript. The grouped primary endpoint comprises 110 analysis groups: 109 UniProt-mapped groups and one retained PDB-level fallback group.

Fig. 2: Main results. (a) SCI distribution across the Main110 NMR cohort, matched synthetic incoherent controls, and 40 AlphaFold single-structure proxies. Brackets indicate pairwise Cliff’s

\delta

and corrected

p

-values. (b) SCI versus mean pLDDT for the AlphaFold reference structures, showing a weak non-significant association. (c) ROC curve for the grouped primary Main110 task (AUC

=0.973

); the star marks the Youden-

J

operating point (

\tau=0.811

Test	Statistic	Value	Note
Primary discrimination: grouped Main110
Paired permutation	$p$	$<10^{-4}$	$n=10{,}000$ permutations
AUC-ROC	—	0.9726	SCI as score
Cliff’s $\delta$	$\delta$	$-0.945$	higher SCI in NMR ensembles
Paired difference	mean	0.337	95% CI: $[0.305,0.370]$
Youden’s $J$	$\tau^{*}$	0.811	sens. 95.5%, spec. 89.1%

TABLE II: Statistical summary for the Main110 manuscript.

Function class	$n$ groups	Mean SCI	Mean $\sigma_{R_{g}}$ (Å)	SCI LOFO	$\sigma_{R_{g}}$ LOFO	NMR $<\tau$	Control $>\tau$
chaperone	3	0.824	0.292	1.000	1.000	1	0
other	6	0.830	0.390	0.944	1.000	1	1
toxin	5	0.843	0.106	0.960	1.000	1	0
enzyme	22	0.850	0.484	0.983	0.977	3	0
transport	13	0.854	0.624	0.964	0.988	3	0
viral	8	0.858	0.189	0.906	0.984	1	1

TABLE III: Function-class summary for the grouped Main110 cohort. Mean SCI and mean

\sigma_{R_{g}}

are reported for the NMR side of each analysis group, together with leave-one-function-class-out (LOFO) AUC for SCI and

\sigma_{R_{g}}

and the number of threshold-failed NMR groups and threshold-exceeding synthetic incoherent controls.

PDB	Function class	$L$	$M$	$\Delta$ SCI	Error pattern
2KUX	other	30	20	0.033	control above $\tau$ , $L\leq 50$
2MCR	immune	36	30	0.059	control above $\tau$ , $L\leq 50$
1SKH	idp	30	22	0.060	control above $\tau$ , $L\leq 50$
7OFO	viral	30	20	0.066	control above $\tau$ , $L\leq 50$
5LBJ	signaling	35	20	0.075	control above $\tau$ , $L\leq 50$
6QES	immune	40	10	0.118	NMR below $\tau$ , $M\leq 10$ , $L\leq 50$

TABLE IV: Representative low-margin and threshold-sensitive Main110 groups.

\Delta

SCI denotes the diverse-minus-control SCI margin within each matched group.

Group	$r_{\text{eff}}$	Top-1 frac.	Top-3 frac.	$n$
Main110 NMR	$2.13\pm 0.67$	0.695	0.850	110
Synthetic incoherent controls	$7.93\pm 2.59$	0.340	0.481	110
AlphaFold proxy	$2.20\pm 0.44$	—	—	40

TABLE V: Effective-rank summary by group.

Model	Raw AUC	Grouped CV AUC	LOFO AUC	Holdout AUC
SCI	0.973	0.968	0.971	0.983
$r_{\text{eff}}$	0.991	0.990	0.990	1.000
Top-1 energy fraction	0.992	0.990	0.990	1.000
Top-3 energy fraction	0.992	0.991	0.991	1.000
SCI + $n_{\text{models}}$	0.987	0.984	0.985	1.000
SCI + $n_{\text{residues}}$	0.986	0.983	0.984	0.975

TABLE VI: Spectral rescue analysis. These models are reported to explain why SCI softens on Main110; they do not replace SCI as the canonical headline metric.

Fig. 3: Spectral diagnostics. (a) SCI distribution for Main110 NMR ensembles versus matched synthetic incoherent controls. (b) SCI versus PCA first-component variance ratio. (c) Rank-correlation between SCI and PCA variance ratio. (d) Participation-ratio SCI versus entropy-based SCI, illustrating that the main ranking is driven by the distance-variance representation rather than the specific effective-rank formula.

Metric or model	Raw AUC	Grouped CV AUC	LOFO AUC	Holdout AUC	Direction
SCI	0.9726	0.9681	0.9707	0.9835	higher $\to$ NMR
$\sigma_{R_{g}}$	0.9859	0.9860	0.9803	1.0000	higher $\to$ NMR
PCA $\lambda_{1}/\Sigma$	0.9669	—	—	—	higher $\to$ control
Contact density	0.5051	—	—	—	uninformative
SCI + $\sigma_{R_{g}}$	0.9837	0.9725	0.9742	1.0000	multivariate
QC-full	0.9929	0.9893	0.9896	1.0000	multivariate

TABLE VII: Baseline and multimetric comparison on the expanded cohort.

Fig. 4: Baseline metric comparison. The current manuscript emphasizes grouped CV, leave-one-function-class-out, and independent holdout performance for SCI,

\sigma_{R_{g}}

, SCI+

\sigma_{R_{g}}

, and QC-full.

$\sigma$ (Å)	AUC	$\delta$	95% CI (paired $\Delta$ )	$p$
0.05	0.9728	$-0.9456$	$[0.306,\;0.371]$	$<10^{-4}$
0.10	0.9727	$-0.9455$	$[0.305,\;0.371]$	$<10^{-4}$
0.20	0.9726	$-0.9453$	$[0.305,\;0.370]$	$<10^{-4}$
0.30	0.9726	$-0.9451$	$[0.304,\;0.369]$	$<10^{-4}$
0.50	0.9716	$-0.9431$	$[0.301,\;0.367]$	$<10^{-4}$

TABLE VIII: Noise sweep: SCI discrimination across synthetic-incoherent-control noise levels.

Models retained	Mean abs. deviation	Median abs. deviation	$n$
5	0.548	0.515	790
10	0.154	0.135	790
15	0.050	0.043	790
20	0.005	0.000	790

TABLE IX: Ensemble-size sensitivity of SCI relative to the full-ensemble estimate.

Fig. 5: Robustness analysis. (a) AUC-ROC versus injected noise level. (b) Cliff’s

\delta

versus injected noise. (c) Contact-based SCI versus contact threshold across NMR, matched synthetic incoherent controls, and AlphaFold references.

PDB	$L$	Contribution vs. RMSF	GNM vs. RMSF	Note
1ARK	60	$-0.505$	0.125	negative
1YS5	156	$-0.025$	0.493	negative
1OQY	363	0.056	$-0.632$	weak
2JQF	166	0.081	0.239	weak
6U1O	258	0.110	$-0.329$	weak

TABLE X: Lowest residue-level SCI contribution correlations with experimental RMSF.

Fig. 6: Multi-metric QC diagnostics. (a) SCI versus

\sigma_{R_{g}}

for all samples. (b) Top-eigenvector smoothness score versus SCI. (c) Smoothness versus

\sigma_{R_{g}}

. The weak-to-moderate SCI–smoothness correlation indicates partially independent diagnostic axes.

Failure type	SCI	$\sigma_{R_{g}}$	Smooth.	Combined
i.i.d. Gaussian	14%	96%	0%	96%
Mode collapse	85%	96%	0%	97%
Residue shuffle	16%	100%	0%	100%
High-rank ( $K\!=\!10$ )	41%	97%	0%	97%

TABLE XI: Failure mode detection rates. Individual metric detection rates (percentage of ensembles flagged) and combined QC gate performance across four synthetic failure types. Expanded-cohort NMR false-positive rate under the fixed combined thresholds: 10.1%.

Fig. 7: Failure mode analysis for the expanded QC stress tests. (a) SCI versus $\sigma_{R_{g}}$ for NMR ensembles and four synthetic artifact classes, with dashed screening thresholds. (b) AUC-ROC per metric per failure type. (c) Detection rate heatmap showing individual-metric and combined screening performance.

Magnetic Resonance Spectroscopy

See all 13 papers

May 11, 2026

Physical probes expose and alleviate chemical-environment collapse in molecular representations

Nuclear magnetic resonance (NMR) spectroscopy provides an experimental readout of local chemical environments, but its use in molecular representation learning has been constrained by heterogeneous data and incomplete atom-level assignments. Here we construct complementary high-fidelity experimental and computational 13C NMR resources, which reveal a recurrent form of representational collapse: atoms that are equivalent in molecular topology can remain experimentally distinct in their real chemical environments, whereas explicit 3D descriptions are further limited by static conformations in dynamic regimes. To alleviate this bottleneck, we develop CLAIM (Contrastive Learning for Atom-to-molecule Inference of Molecular NMR), a framework that aligns efficient topological molecular inputs with atom-resolved NMR observables. Through hierarchical chemical priors and cross-level contrastive learning, CLAIM restores lost chemical resolution and markedly improves atom-level molecule-spectrum retrieval. CLAIM remains robust in flexible and tautomeric systems for 13C NMR prediction, improves stereoisomer discrimination without explicit 3D modelling, and transfers to broader molecular property tasks including ADMET prediction and fluorescence estimation. These results establish physically grounded spectral alignment as an effective strategy for alleviating chemical-environment collapse and for guiding experimentally grounded molecular representation learning.

Jiebin Fang, Zidi Yan, Churu Mao +7

Hainan Institute, Zhejiang University, Sanya 572025, China · Institute of Marine Biology and Pharmacology, Ocean College, Zhejiang University, Zhoushan 316021, China · School of Food and Pharmacy, Zhejiang Ocean University, Zhoushan 316021, China +2

Contrastive Learning Cross-Modal Representation Learning Molecular Property Prediction Molecular Representation Learning Magnetic Resonance Spectroscopy

Ask PDF

May 13, 2026

ENSEMBITS: an alphabet of protein conformational ensembles

Protein structure tokenizers (PSTs) are workhorses in protein language modeling, function prediction, and evolutionary analysis. However, existing PSTs only capture local geometry of static structures, and miss the correlated motions and alternative conformational states revealed by protein ensembles. Here we introduce Ensembits, the first tokenizer of protein conformational ensembles. Ensembits address challenges inherent to tokenizing dynamics: deriving informative geometric descriptors across conformations, permutation-invariance encoding of variable-size ensembles, and conquering sparsity in dynamics data. Trained with a Residual VQ-VAE using a frame distillation objective on a large molecular dynamics corpus, Ensembits outperforms all related methods on RMSF prediction, and is the strongest standalone structural tokenizer on an token-conditioned ANOVA test on per-residue motion amplitude. Ensembits further matches or exceeds static tokenizers on EC, GO, binding site/affinity prediction, and zero-shot mutation-effect prediction despite using far less pretraining data. Notably, the distillation objective enables Ensembits to predict dynamics token from one single predicted structure, which alleviates dynamics data sparsity. As the field moves from static structure prediction toward ensemble generation, Ensembits offer the discrete vocabulary needed to bring dynamics into protein language modeling and design.

Kaiwen Shi, Carlos Oliver

Department of Computer Science, Vanderbilt University · Center for AI in Protein Dynamics, Vanderbilt University · Department of Molecular Physiology and Biophysics, Vanderbilt University

Protein Language Models Protein Representation Learning Molecular Representation Learning

Ask PDF

Oct 1, 2025

Speak to a Protein: An Interactive Multimodal Co-Scientist

Building a working mental model of a protein typically requires weeks of reading, cross-referencing crystal and predicted structures, and inspecting ligand complexes, an effort that is slow, unevenly accessible, and often requires specialized computational skills. We introduce Speak to a Protein, a new capability that turns protein analysis into an interactive, multimodal dialogue with an expert co-scientist. The AI system retrieves and synthesizes relevant literature, structures, and ligand data; grounds answers in a live 3D scene; and can highlight, annotate, manipulate and see the visualization. It also generates and runs code when needed, explaining results in both text and graphics. We demonstrate these capabilities on relevant proteins, posing questions about binding pockets, conformational changes, or structure-activity relationships to test ideas in real time. Speak to a Protein reduces the time from question to evidence, lowers the barrier to advanced structural analysis, and enables hypothesis generation by tightly coupling language, code, and 3D structures. Speak to a Protein is freely accessible at https://open.playmolecule.org.

Carles Navarro, Mariona Torrens, Philipp Thölke +2

Acellera Labs, C/ Doctor Trueta 183, 08005 Barcelona, Spain · Computational Science Laboratory, Universitat Pompeu Fabra, PRBB, C/ Doctor Aiguader 88, 08003 Barcelona, Spain · Instituci´o Catalana de Recerca i Estudis Avan¸cats (ICREA), Passeig Llu´ıs Companys 23, 08010 Barcelona, Spain

AI Agents for Scientific Discovery Scientific QA Multimodal Tool Use

Ask PDF

Abstract

Magnetic Resonance Spectroscopy

Explore similar work

Physical probes expose and alleviate chemical-environment collapse in molecular representations

ENSEMBITS: an alphabet of protein conformational ensembles

Speak to a Protein: An Interactive Multimodal Co-Scientist