TALK: One-Shot Batch Design for Protein Variant Effect Prediction
Organizations: Digital Medical Research Center, School of Basic Medical Sciences, Fudan University, China · Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention, China
Abstract
Protein variant effect prediction is central to understanding protein function and guiding protein engineering, yet experimental labels remain costly. In many protein experiments, the variants that can be assayed are dictated by protocol constraints, while the variants that need accurate prediction may lie in a different or broader region of the landscape. This assay-prediction mismatch makes candidate-local criteria such as predicted function, uncertainty, or exploration value insufficient for measurement design. Given a protocol-defined accessible space and a limited budget, we ask which variants should be assayed so that the resulting data best improves landscape prediction. We propose TALK, which estimates each candidate's contribution through posterior coupling between accessible and target variants and reduces redundant information within the batch. Across retrospective ProteinGym and FLIP protocols, TALK generally improves target-space prediction under limited measurement budgets. These results point to a practical two-stage workflow: first, spend a limited assay budget on variants selected to be informative for the target landscape and train a predictive model from the measured labels; then use this model to score and prioritize high-potential variants. Code is available at https://github.com/joehuang618/TALK.
Figures & tables
Appendix figures & tables26 assets
Supplementary material from the paper’s appendix.
Appendix
| Component | Required posterior objects | Role in TALK |
|---|---|---|
| Prediction on | , | Evaluate target predictions and uncertainty |
| Sample variance baseline | Select points with the largest marginal uncertainty in the current query pool | |
| VTL | , , | Compute target-space trace variance reduction |
| Exact ITL | , , , | Compute information gain between target effects and candidate observation |
| Dense conditional batch | , , , | Apply Schur-complement updates to the joint covariance over targets and candidates |
| Low-rank conditional batch | diagonal terms and low-rank factors for and | Approximately maintain covariance updates without explicitly storing large matrices |
| Backend | Main storage | Main cost per score | Batch update | Use case |
|---|---|---|---|---|
| Dense exact | ITL depends on solves with ; VTL is about | dense Schur update; quadratic in joint covariance size | small panels, exact debugging | |
| Exact chunked cross | , or if storing | chunked cross-covariance computation; VTL can be accumulated in streaming form | dense update only if joint covariance is constructed | medium targets, exact cross-covariance |
| Sparse/FITC-style low-rank | cross-coupling about or chunked factor products | low-rank transform, mainly depends on and | large target panels |
| Setting | B | n | mean % | min % | p25 % | median % | p75 % | max % |
|---|---|---|---|---|---|---|---|---|
| 8 | 51 | 0.81 | 0.59 | 0.66 | 0.77 | 0.89 | 1.29 | |
| 16 | 51 | 1.63 | 1.18 | 1.33 | 1.55 | 1.77 | 2.58 | |
| 32 | 51 | 3.25 | 2.36 | 2.65 | 3.09 | 3.54 | 5.15 | |
| 64 | 51 | 6.50 | 4.72 | 5.30 | 6.18 | 7.09 | 10.31 | |
| 128 | 51 | 13.01 | 9.43 | 10.61 | 12.37 | 14.17 | 20.61 | |
| 256 | 51 | 26.01 | 18.87 | 21.22 | 24.73 | 28.34 | 41.22 |
| Setting | n budgets | mean saving % | median saving % | final Random B | best ITL/VTL B for final Random | final saving % |
|---|---|---|---|---|---|---|
| ProteinGym-MM | 6 | 78.5 | 75.5 | 256 | 53.0 | 79.3 |
| ProteinGym-MM mixed-order pool protocol | 6 | 61.1 | 61.2 | 256 | 91.6 | 64.2 |
| ProteinGym-MM Singles-premeasured protocol | 6 | 34.0 | 36.8 | 256 | 192.0 | 25.0 |
| GB1 1-vs-rest | 5 | -31.3 | 21.5 | 28 | 22.0 | 21.5 |
| GB1 2-vs-rest | 6 | 60.0 | 64.3 | 256 | 118.7 | 53.6 |
| GB1 3-vs-rest | 6 | 59.2 | 65.9 | 256 | 110.7 | 56.8 |
| Method | |||||||
|---|---|---|---|---|---|---|---|
| ITL | 0.1888 | 0.3203 | 0.4319 | 0.5190 | 0.5931 | 0.6428 | 0.6518 |
| VTL | 0.1888 | 0.3308 | 0.4310 | 0.5156 | 0.5756 | 0.6172 | 0.6432 |
| SampleVariance | 0.1888 | 0.2579 | 0.3537 | 0.4441 | 0.4931 | 0.5321 | 0.5812 |
| Random | 0.1888 | 0.1887 | 0.2690 | 0.3557 | 0.4328 | 0.5103 | 0.5675 |
| -Random SD (n=5) | 0.0000 | 0.0257 | 0.0218 | 0.0089 | 0.0243 | 0.0141 | 0.0065 |
| top-mean | 0.1888 | 0.1152 | 0.1454 | 0.2115 | 0.2540 | 0.3297 | 0.3920 |
| Method | |||||||
|---|---|---|---|---|---|---|---|
| ITL | 0.1860 | 0.4065 | 0.5649 | 0.6800 | 0.7696 | 0.8302 | 0.8702 |
| VTL | 0.1860 | 0.3922 | 0.5540 | 0.6928 | 0.7855 | 0.8417 | 0.8778 |
| SampleVariance | 0.1860 | 0.3830 | 0.5060 | 0.5985 | 0.6485 | 0.6823 | 0.7163 |
| Random | 0.1860 | 0.2599 | 0.3810 | 0.5157 | 0.6389 | 0.7413 | 0.8098 |
| -Random SD (n=5) | 0.0000 | 0.0195 | 0.0115 | 0.0075 | 0.0053 | 0.0057 | 0.0035 |
| UCB | 0.1860 | 0.1230 | 0.1333 | 0.1961 | 0.2731 | 0.3544 | 0.4604 |
| Method | |||||||
|---|---|---|---|---|---|---|---|
| ITL | 0.6371 | 0.7034 | 0.7387 | 0.7853 | 0.8175 | 0.8670 | 0.8907 |
| VTL | 0.6371 | 0.7095 | 0.7498 | 0.7887 | 0.8255 | 0.8635 | 0.8838 |
| SampleVariance | 0.6371 | 0.6867 | 0.7167 | 0.7509 | 0.7935 | 0.8453 | 0.8831 |
| Random | 0.6371 | 0.6836 | 0.7200 | 0.7575 | 0.7971 | 0.8460 | 0.8801 |
| -Random SD (n=5) | 0.0000 | 0.0039 | 0.0019 | 0.0026 | 0.0017 | 0.0014 | 0.0011 |
| UCB | 0.6371 | 0.6047 | 0.6210 | 0.6301 | 0.6583 | 0.6915 | 0.7288 |
| Method | |||||
|---|---|---|---|---|---|
| ITL | 0.2183 | 0.0939 | 0.2230 | 0.3828 | 0.3839 |
| VTL | 0.1351 | 0.1021 | 0.3702 | 0.3886 | 0.3839 |
| SampleVariance | 0.1042 | 0.0475 | 0.2571 | 0.3740 | 0.3839 |
| Random mean | 0.1608 | 0.1700 | 0.2998 | 0.3656 | 0.3839 |
| -Random SD (n=5) | 0.1406 | 0.1491 | 0.0855 | 0.0601 | 0.0000 |
| top-mean | 0.3235 | 0.3207 | 0.4143 | 0.4501 | 0.3839 |
| Method | ||||||
|---|---|---|---|---|---|---|
| ITL | 0.3743 | 0.2138 | 0.1817 | 0.0188 | 0.4245 | 0.4879 |
| VTL | 0.3890 | 0.2327 | 0.3042 | 0.2632 | 0.4852 | 0.5851 |
| SampleVariance | 0.3753 | 0.2183 | 0.1817 | 0.1678 | 0.4664 | 0.4868 |
| Random | -0.1449 | -0.0820 | -0.0206 | 0.0930 | 0.3426 | 0.4712 |
| -Random SD (n=5) | 0.0765 | 0.2444 | 0.1410 | 0.1435 | 0.0334 | 0.0427 |
| top-mean | 0.2418 | -0.0885 | 0.2643 | 0.3047 | -0.0681 | 0.5113 |
| Method | ||||||
|---|---|---|---|---|---|---|
| ITL | 0.4993 | 0.5561 | 0.4622 | 0.5764 | 0.6193 | 0.6973 |
| VTL | 0.4006 | 0.6187 | 0.6520 | 0.6392 | 0.6905 | 0.7448 |
| SampleVariance | 0.5277 | 0.5359 | 0.4587 | 0.5711 | 0.6232 | 0.6897 |
| Random | 0.2241 | 0.0993 | 0.4111 | 0.5177 | 0.6066 | 0.6801 |
| -Random SD (n=5) | 0.3653 | 0.3971 | 0.1978 | 0.0728 | 0.0403 | 0.0440 |
| top-mean | 0.4385 | 0.5516 | 0.4938 | 0.5038 | 0.4586 | 0.6046 |
| Method | |||||||
|---|---|---|---|---|---|---|---|
| ITL | 0.1676 | 0.2707 | 0.3204 | 0.3719 | 0.4243 | 0.4686 | 0.4563 |
| VTL | 0.1676 | 0.1784 | 0.2642 | 0.3445 | 0.3949 | 0.4408 | 0.4665 |
| SampleVariance | 0.1676 | 0.2099 | 0.2392 | 0.2888 | 0.3199 | 0.3444 | 0.4000 |
| Random | 0.1676 | 0.1614 | 0.1988 | 0.2373 | 0.2751 | 0.3196 | 0.3668 |
| -Random SD (n=5) | 0.0000 | 0.0201 | 0.0047 | 0.0114 | 0.0116 | 0.0172 | 0.0132 |
| UCB | 0.1676 | 0.1470 | 0.1496 | 0.1595 | 0.2216 | 0.2485 | 0.2780 |
| Method | |||||||
|---|---|---|---|---|---|---|---|
| ITL | 0.1697 | 0.2692 | 0.3598 | 0.4564 | 0.5461 | 0.6057 | 0.6590 |
| VTL | 0.1697 | 0.1996 | 0.2817 | 0.4212 | 0.5221 | 0.6084 | 0.6551 |
| SampleVariance | 0.1697 | 0.2484 | 0.3470 | 0.3893 | 0.4329 | 0.4611 | 0.4875 |
| Random | 0.1697 | 0.1809 | 0.2318 | 0.3099 | 0.3949 | 0.4730 | 0.5590 |
| -Random SD (n=5) | 0.0000 | 0.0143 | 0.0063 | 0.0093 | 0.0090 | 0.0051 | 0.0111 |
| UCB | 0.1697 | 0.1287 | 0.1322 | 0.1664 | 0.2250 | 0.2599 | 0.3177 |
| Method | |||||||
|---|---|---|---|---|---|---|---|
| ITL | 0.4537 | 0.4728 | 0.5009 | 0.5357 | 0.5854 | 0.6431 | 0.6949 |
| VTL | 0.4537 | 0.4791 | 0.5028 | 0.5446 | 0.5983 | 0.6428 | 0.6928 |
| SampleVariance | 0.4537 | 0.4810 | 0.5026 | 0.5125 | 0.5564 | 0.6010 | 0.6619 |
| Random | 0.4537 | 0.4730 | 0.4913 | 0.5216 | 0.5636 | 0.6124 | 0.6644 |
| -Random SD (n=5) | 0.0000 | 0.0028 | 0.0071 | 0.0114 | 0.0080 | 0.0061 | 0.0035 |
| top-mean | 0.4537 | 0.4049 | 0.4184 | 0.4503 | 0.4873 | 0.5032 | 0.5327 |
| Method | |||||
|---|---|---|---|---|---|
| ITL | 0.0183 | 0.1193 | 0.1399 | 0.1583 | 0.1583 |
| VTL | 0.0688 | 0.0780 | 0.1261 | 0.1468 | 0.1583 |
| SampleVariance | 0.0069 | 0.1170 | 0.1514 | 0.1583 | 0.1583 |
| Random mean | 0.0656 | 0.0821 | 0.1252 | 0.1592 | 0.1583 |
| -Random SD (n=5) | 0.0368 | 0.0350 | 0.0262 | 0.0077 | 0.0000 |
| top-mean | 0.0482 | 0.0688 | 0.1239 | 0.1606 | 0.1583 |
| Method | ||||||
|---|---|---|---|---|---|---|
| ITL | 0.0216 | 0.0865 | 0.0457 | 0.0481 | 0.1394 | 0.1803 |
| VTL | 0.1298 | 0.1851 | 0.1370 | 0.1346 | 0.1611 | 0.2524 |
| SampleVariance | 0.0216 | 0.0889 | 0.0457 | 0.0601 | 0.1755 | 0.1779 |
| Random | 0.0346 | 0.0365 | 0.0481 | 0.1034 | 0.1567 | 0.2139 |
| -Random SD (n=5) | 0.0132 | 0.0305 | 0.0275 | 0.0476 | 0.0407 | 0.0090 |
| top-mean | 0.0601 | 0.0433 | 0.0841 | 0.1442 | 0.0361 | 0.2332 |
| Method | ||||||
|---|---|---|---|---|---|---|
| ITL | 0.1250 | 0.1493 | 0.2465 | 0.2188 | 0.2778 | 0.3194 |
| VTL | 0.0104 | 0.0590 | 0.1562 | 0.2535 | 0.2812 | 0.3542 |
| SampleVariance | 0.1285 | 0.1667 | 0.2465 | 0.2153 | 0.2708 | 0.3056 |
| Random | 0.0938 | 0.0792 | 0.1486 | 0.2118 | 0.2347 | 0.3326 |
| -Random SD (n=5) | 0.0518 | 0.0649 | 0.0828 | 0.0244 | 0.0615 | 0.0531 |
| top-mean | 0.0903 | 0.1424 | 0.1389 | 0.1528 | 0.1632 | 0.2639 |
| Method | Spearman | RMSE | Top-5% |
|---|---|---|---|
| Target-kernel herding | 0.5934 | 0.9005 | 0.3762 |
| ITL | 0.5931 | 0.8754 | 0.4243 |
| MM-ITL | 0.5877 | 0.8900 | 0.3929 |
| EPIG | 0.5865 | 0.8797 | 0.4045 |
| CTL | 0.5845 | 0.8494 | 0.3779 |
| VTL | 0.5756 | 0.8780 | 0.3949 |
| Budget | Method | vs. Random | vs. SV |
|---|---|---|---|
| 32 | ITL | [0.0208, 0.0351] | [0.0243, 0.0458] |
| 32 | VTL | [0.0238, 0.0385] | [0.0280, 0.0486] |
| 64 | ITL | [0.0150, 0.0262] | [0.0148, 0.0347] |
| 64 | VTL | [0.0233, 0.0338] | [0.0236, 0.0416] |
| 128 | ITL | [0.0175, 0.0247] | [0.0168, 0.0269] |
| 128 | VTL | [0.0127, 0.0220] | [0.0121, 0.0244] |
| Rule | Selection perturbation | Difference | 95% interval |
|---|---|---|---|
| ITL | Noise | [ , ] | |
| ITL | Noise | [ , ] | |
| ITL | Sequence only, no ZS | [ , ] | |
| ITL | Sequence only, ZS | [ , ] | |
| VTL | Noise | [ , ] | |
| VTL | Noise | [ , ] |
| Selector predictor | (51) | (51) | (49) |
|---|---|---|---|
| Kermut Kermut | 0.7835 / 0.7896 | 0.8231 / 0.8272 | 0.8691 / 0.8696 |
| Kermut one-hot | 0.7377 / 0.7385 | 0.7790 / 0.7817 | 0.8350 / 0.8364 |
| One-hot Kermut | 0.7705 / 0.7759 | 0.8128 / 0.8162 | 0.8564 / 0.8584 |
| One-hot one-hot | 0.7462 / 0.7370 | 0.7836 / 0.7795 | 0.8351 / 0.8328 |
| Budget | Method | vs. Random | vs. SV |
|---|---|---|---|
| 32 | ITL | [0.0217, 0.0411] | [ , 0.0187] |
| 32 | VTL | [0.0148, 0.0289] | [ , 0.0096] |
| 64 | ITL | [0.0232, 0.0399] | [0.0019, 0.0182] |
| 64 | VTL | [0.0201, 0.0348] | [ , 0.0162] |
| 128 | ITL | [0.0164, 0.0289] | [0.0004, 0.0065] |
| 128 | VTL | [0.0156, 0.0252] | [ , 0.0052] |
| Method | Spearman | RMSE | top-5% | top-10% |
|---|---|---|---|---|
| ITL | 0.4797 | 1894.52 | 0.2559 | 0.3224 |
| VTL | 0.5017 | 1853.86 | 0.2764 | 0.3457 |
| SampleVariance | 0.4315 | 1987.73 | 0.2215 | 0.2897 |
| Random | 0.4790 | 2009.15 | 0.2403 | 0.3199 |
| top-mean | 0.3120 | 2095.16 | 0.1698 | 0.2421 |
| UCB | 0.2685 | 2065.49 | 0.1708 | 0.2374 |