cs.LGOct 7, 2026

Kernel Autoresearch for Open-Ended Model Discovery

Authors: Richard Cornelius Suwandi, Feng Yin, Kevin Murphy

Organizations: CUHK-Shenzhen · University of British Columbia

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Kernels encode the inductive bias of a wide range of machine learning models, yet automated kernel design faces a fundamental dilemma. A fixed grammar of base kernels and operators guarantees validity but limits the search to structures expressible by those building blocks. Conversely, unrestricted programs remove this limitation but no longer guarantee validity. In our stress tests, 22-58% of LLM-generated kernels that pass numerical checks on random inputs fail when evaluated at different scales or dimensions. We propose Kernel Autoresearch (Kernaut), which treats kernel design as open-ended model discovery. Coding agents write kernels as programs, while construction contracts ensure that every accepted kernel is valid. A quality-diversity archive retains high-performing kernels with distinct behaviors, and novelty screening steers agents toward functionally new candidates. Our experiments demonstrate that the discovered kernels encode reusable inductive biases that generalize to unseen tasks. On held-out black-box optimization families, a discovered kernel outperforms a meta-learned deep kernel trained on the same episodes. Furthermore, kernels discovered from ten enzyme-kinetic rate laws achieve lower error than tuned ARD and deep kernel baselines on five unseen mechanisms. The discovered kernels are also interpretable programs that human researchers can refine: a human-refined version of one further reduces the held-out predictive error by 5.7% and optimization regret by 7.8%.

Figures & tables

Figure 1: The Kernaut framework. The coding agent (blue) writes kernel components under one of four construction contracts. The deterministic backend (orange) applies three acceptance tiers, T0 to T2, and only T2 programs enter the quality-diversity archive, where each cell keeps the best candidate for one behavior. Archived kernels and structured feedback guide later proposals. The grey module computes the posterior and tunes parameters.
Figure 2: Synthetic regression: search archives. (a) Validation continuous ranked probability score (CRPS, Equation ( 11 ), lower is better) of every accepted program in 14 campaigns. E1–E7 use the three-model ensemble, and F1–F7 use GPT-6 Astra as the sole proposer. Stars mark validation-selected programs, and the dashed line marks the best tuned baseline. (b) Lineage of the first ensemble campaign (E1). Edges join each parent to its revisions, and the starred node is the selected program.
Figure 3: Frozen BBO results on 60 held-out test tasks (lower is better). Left: CRPS. Right: regret AUC, the mean normalized simple regret over BO steps. Points are means. Bars are paired 95% bootstrap intervals for the gap to DWF, where each line ends. Rows are ordered by average rank across both metrics. Fixed kernels use the setting selected on training tasks. Grid spectral mixtures (GSM), input warping, and random Fourier features (RFF) average five fitted variants.
Figure 4: Forecasting: held-out greenhouse-gas records. The discovered kernel is the ensemble-selected residual period-bank kernel. Each panel shows, for one record, the window with its median CRPS. Values are in parts per trillion (ppt). SF 6 is forecast forward. The chlorofluorocarbon records CFC-12 and CFC-11 are backcast, so their held-out stretch precedes the dotted line. Bands are 95% intervals. RBF misses the turn in both CFC records. Its smooth posterior mean follows the broader trend of the training window and starts above the last observation.
Figure 5: Enzyme kinetics and glucose dynamics: transfer to unseen mechanisms and patients. (a) CRPS reduction (%) relative to Matérn- 5/25/2 on held-out enzyme mechanisms, with each campaign’s validation CRPS below. Purple means lower error and brown means higher error. Validation selected E1 and F2, shown in bold. (b) Normalized mean squared error (MSE, Equation ( 25 ) of Appendix E.2 , log scale) for adult continuous glucose monitor (CGM) forecasts after a 45-minute prefix, against training episodes per patient. Validation selected the discovered kernels. References are tuned ARD-Matérn, per-patient type-II maximum-likelihood (ML-II) fitting, and the best fixed kernel.
Appendix figures & tables42 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: The archive fills niche by niche over a campaign. MAP-Elites archive cells after runs 3, 6, 9, and 12 for the first ensemble campaign (E1, top) and the first GPT-6 Astra campaign (F1, bottom). Rows are niches and columns are novelty bands. Each filled cell shows its current elite (best search score), colored by validation CRPS (darker is better) and labeled with the run that produced it. A ring marks the validation-selected program, which is DWF in the first run of E1. Cells are indexed by niche and novelty band only, so the feature-growth descriptor of Equation ( 4 ) is omitted.
WeightingWeightsDWF rankElites kept
Original(.55,.30,.10,.05)(.55,.30,.10,.05)19/9
AUC-heavy(.30,.55,.10,.05)(.30,.55,.10,.05)19/9
Equal(.25,.25,.25,.25)(.25,.25,.25,.25)19/9
CRPS-only(1,0,0,0)(1,0,0,0)19/9
Appendix
Table 1: Sensitivity of the first BBO archive to the selection weights. Each row re-scores the nine programs under one weighting of (ΔC,ΔAUC,ΔF,ΔW)(\Delta_{\mathrm{C}},\Delta_{\mathrm{AUC}},\Delta_{\mathrm{F}},\Delta_{\mathrm{W}}) in Equation ( 5 ), using the four stored components. “DWF rank” is the position of DWF among the nine by search score, where 1 is best. “Elites kept” counts the programs that remain elites of their cells.
TLabelWhat is established
0ExecutableSandboxed execution succeeds and the output matches the interface
1EmpiricalRandomized Gram-matrix tests pass on sampled input sets
2Contract certifiedChecked interface conformance and trusted PSD-preserving assembly under the stated component assumptions
Appendix
Table 2: Implemented evidence ladder.
RelationWhat must match
DeterminismThe same input, replayed twice
PermutationThe Gram of a permuted batch equals the permuted Gram
SubsetThe Gram of a random subset equals the corresponding submatrix
ExtensionAppending then truncating points reproduces the original block
DuplicateA duplicated point reproduces its own row
Appendix
Table 3: Tier 2 contract-conformance relations. See Equation ( 9 ).
FieldPurpose
mathematical_formKernel expression and construction
psd_argumentProof sketch for positive semidefiniteness
novelty_claimWhat is new relative to known kernels
closest_known_kernelNearest baseline and how this differs
expected_bo_behaviorPredicted BBO performance characteristics
construction_nicheDeclared construction category
Appendix
Table 4: Formulation fields registered before code submission (§ 3.4 ).
Figure 7: Finite numerical screening admits invalid programs. Outcome of each cohort of kernel programs, split into programs that fail to run, fail the public Tier-1 screen, pass it but are falsified on the hidden suite, and survive the hidden suite. Labels give the number falsified among the programs that passed the public screen. Canaries were written to pass the public screen and fail the hidden suite. Certified controls pass Tier 2. Unrestricted programs that survive the hidden suite are empirically unfalsified, not certified.
FamilyddSplit(s)
Branin2train, val
Ackley2train, val
Cosine8train, val
Hartmann-66train, val
Levy6train, val
Bukin2test
Appendix
Table 5: Task-family instantiation and split (§ 4.1 , § 4.3.1 ). Meta-training and meta-validation use the same five families with disjoint seed offsets ( 00 and 10410^{4} ). Meta-test uses six families never seen during search, at offset 2×1042\times 10^{4} . Each family contributes 10 seeded episodes per split, giving 50 meta-training, 50 meta-validation, and 60 meta-test tasks. Targets are clean function values.
 CRPS ↓\downarrowAUC ↓\downarrow
Candidatevaltestvaltest
Linear0.7830.7910.9130.872
Periodic ( ℓ=1\ell{=}1 , p=2p{=}2 )0.5490.6180.8160.678
Rational quadratic (RQ; ℓ=0.25\ell{=}0.25 , α=1\alpha{=}1 )0.5590.6180.7850.706
Matérn-5/2 ( ℓ=0.5\ell{=}0.5 )0.5510.6120.7970.693
RBF ( ℓ=0.25\ell{=}0.25 )0.5650.6410.8230.756
Appendix
Table 6: Frozen-benchmark candidates and learned-feature references, with validation and test CRPS and regret AUC. Lower is better. Bold marks the best value within each block.
Figure 8: FSBO needs about two orders of magnitude more meta-training data to match DWF. Test CRPS (left) and regret AUC (right) on the 60 BBO test tasks, lower is better. FSBO is meta-trained on the 518 points that search scored (matched budget) or on 64,000 points (rich budget). Solid lines show native FSBO, which is fine-tuned on every episode. Dotted lines show frozen FSBO, which refits only amplitude and noise. DWF (blue) is archived with 518 points, and the dashed line is fixed Matérn-5/2. Lines join the two measured FSBO budgets and do not interpolate.
Figure 9: Seven ensemble and seven frontier campaigns on the 60 BBO test tasks. The frontier campaigns use GPT-6 Astra. Each dot is one campaign’s validation-selected kernel, bars mark the mean per proposer, and the dashed line is fixed Matérn-5/2 ( ℓ=0.5\ell{=}0.5 , test CRPS 0.61170.6117 , regret AUC 0.69310.6931 ). The black diamond is one Qwen-only campaign (test CRPS 0.6160.616 , regret AUC 0.7090.709 ). Lower is better for both metrics. Campaigns that beat Matérn: frontier 7 of 7 on CRPS and 6 of 7 on regret AUC, ensemble 6 of 7 on CRPS and 4 of 7 on regret AUC.
SearchTest CRPSMean AUCFailed calls
Full archive0.571±0.031\mathbf{0.571\pm 0.031}0.68128
Independent proposals0.617±0.0370.617\pm 0.0370.69515
Closure grammar0.618±0.0210.618\pm 0.0210.70622
CKS greedy grammar, depth ≤3\leq 30.6250.6250.684–
Appendix
Table 7: Search-ablation results on the BBO domain. The first three rows are 12 seeded campaigns per arm, reported as mean ±\pm sample SD. CKS is one deterministic greedy search. Failed calls are model or tool calls that did not yield an accepted response, summed over each arm’s 12 campaigns.
Figure 10: A stronger proposer starts ahead and keeps improving. Best validation CRPS among the programs accepted after each discovery run (lower is better). Thin lines are the 14 individual campaigns. Thick lines are the means over the seven ensemble and seven frontier (GPT-6 Astra) campaigns. The dashed line is the best tuned baseline. A run that accepts no program leaves its curve unchanged.
Figure 11: Validation regret of the validation-CRPS leader. At each discovery run, the plotted value is the validation regret AUC of the program with the best validation CRPS among those accepted so far. Thin lines are the 14 campaigns. Thick lines are means over seven ensemble and seven frontier campaigns. The score is validation regret.
Figure 12: DWF outside the search families. CRPS change relative to Matérn-5/2, where positive values favor DWF. Each synthetic point averages ten episodes for one function and domain variant. Color gives the empirical reflection-symmetry index SS , defined in the paragraph below. Circles use the standard domain, and triangles use an off-center subdomain. The refit panel clips eight losses below −50%-50\% , with a minimum of −369%-369\% . Real-regression points average 20 splits for one dataset and training size.
 Validation ( n=50n{=}50 )Test ( n=60n{=}60 )
Competitorwinsmed. Δ\Delta CRPSppwinsmed. Δ\Delta CRPSpp
vs. fixed baselines
Periodic ( ℓ=1\ell{=}1 , p=2p{=}2 )34/50−0.009-0.0090.0130.013 *45/60−0.018-0.0187.0×10−57.0\times 10^{-5} ***
Matérn-5/2 ( ℓ=0.5\ell{=}0.5 )31/50−0.015-0.0150.0690.06939/60−0.013-0.0130.0230.023 *
RQ ( ℓ=0.25\ell{=}0.25 , α=1\alpha{=}1 )39/50−0.018-0.0183.5×10−53.5\times 10^{-5} ***45/60−0.019-0.0192.2×10−52.2\times 10^{-5} ***
vs. discovered kernels
Appendix
Table 8: Paired Wilcoxon tests of DWF’s per-task CRPS against each competitor ( n=50n{=}50 validation, n=60n{=}60 test). “Wins” counts tasks where DWF is lower. Med. Δ\Delta CRPS is the median signed difference, and negative favors DWF. The pp -values are unadjusted. Holm correction over the five competitors within each split leaves every 0.05-level conclusion unchanged. ∗p<0.05{}^{*}p<0.05 , ∗∗p<0.01{}^{**}p<0.01 , ∗∗∗p<0.001{}^{***}p<0.001 .
Figure 13: Frozen DWF geometry, where uu is a coordinate in [0,1][0,1] . Left: the warp stays near the identity, and the fold is one tent peaking at u=0.513u=0.513 (dotted line). Right: covariance with a reference point at u=0.2u=0.2 and the dotted line at its fold mirror u=0.825u=0.825 . Joint DWF matches Matérn- 5/25/2 at the mirror, while the additive split stays near 0.700.70 .
Figure 14: Prior samples under the discovered kernel. Four draws from unit-variance GPs on [0,1][0,1] . Panels (a)–(c) share the same standard normals, so the draws differ only through the kernel: (a) Matérn- 5/25/2 with ℓ=0.5\ell=0.5 , (b) joint DWF at its frozen parameters, (c) the refinement kadd=12kw+12kak_{\mathrm{add}}=\tfrac{1}{2}k_{w}+\tfrac{1}{2}k_{a} . The dotted line marks the fold peak c=1/(2ν)=0.513c=1/(2\nu)=0.513 , and arrows join the mirror pair u=0.2u=0.2 and 1/ν−u=0.8251/\nu-u=0.825 . (d) Finite-difference slopes near cc . They jump for DWF (solid) and stay continuous for Matérn (dashed), apart from small ripples from the 10−810^{-8} jitter. (e) Prior correlation between f(u)f(u) and f(1/ν−u)f(1/\nu-u) . Joint DWF is within 0.0030.003 of Matérn, the additive kernel stays above 0.570.57 , and the fold branch alone fixes it at 11 . (f) One DWF draw on [0,1]2[0,1]^{2} , with contours bending along xj=cx_{j}=c . ‡ Human researcher-defined refinement, not search output.
ConstructionTest CRPS (repeat mean ±\pm SD)Selected val.Selected testSelected AUC
Identity Matérn0.6198±0.00100.6198\pm 0.00100.55460.61890.6824
Warp only0.6245±0.00280.6245\pm 0.00280.55900.62230.6896
Fold only0.6339±0.02830.6339\pm 0.02830.55190.61470.6901
Additive branches ‡0.5668±0.00000.5668\pm 0.00000.50200.56680.6190
Joint branches0.6131±0.00710.6131\pm 0.00710.54040.60110.6712
Rational quadratic0.6123±0.00490.6123\pm 0.00490.54890.60900.6896
Appendix
Table 9: Matched predictive-objective controls (tuning protocol described above). Test CRPS mean ±\pm SD is over five tuning repeats on the same 60 episodes, not over five discovery campaigns. Selected columns report the validation-chosen repeat. Archived DWF is a frozen anchor with a different historical search budget. ‡ Human researcher-defined refinement, not search output.
ConstructionTest CRPSRegret AUC
Joint0.6011460.671159
Joint, smooth0.6011620.671159
Additive ‡0.5668320.618955
Additive, smooth ‡0.5668790.618955
Appendix
Table 10: BBO sharp/smooth ablation. Entries use validation-selected configurations evaluated on the same 60 test episodes. Lower is better. ‡ Human researcher-defined refinement, not search output.
Per discovery runBBOTime-series
Rounds1–108–10
Tool calls0–117–10
Input tokens4k–190k69k–184k
Output tokens0.3k–10k3k–9.6k
Appendix
Table 11: Agent-loop cost per discovery run, min–max over the 12 runs of each reported search.
Figure 15: Share of tool calls by workflow phase, pooled across the 12 runs of each reported search (97 BBO calls, 109 time-series calls). Each phase groups these tools. Formulation: propose_formulation . Draft/verify: stage_candidate and verify_staged_candidate . Parameter search: optimize_parameters and evaluate_parameters . Submission: submit_staged_candidate . Archive query: query_archive and inspect_candidate . Final evaluation: evaluate_candidate .
MethodvalSF 6
Linear0.8260.959
Periodic ( p≈p{\approx} annual)0.2411.799
RQ ( ℓ=0.5\ell{=}0.5 )0.0950.067
Matérn-5/2 ℓ=1\ell{=}10.0910.050
Matérn-3/2 ℓ=1\ell{=}10.0990.075
RBF ℓ=0.5\ell{=}0.50.086\mathbf{0.086}0.0290.029
Appendix
Table 12: Frozen greenhouse-gas candidates, fitted reference families, and the closed-grammar CAKE winner: mean CRPS over validation episodes and over ten held-out SF 6 windows (lower is better).
KernelAll CRPSForward CRPSReversed CRPSForward coverageForward width
Residual period-bank (selected)0.04560.04880.03910.9180.334
Harmonic–trend root0.01940.01740.02350.9250.121
RBF ( ℓ=0.5\ell=0.5 )0.02640.02840.02230.8800.153
Uniform ARD-RBF ( ℓ=1\ell=1 )0.01840.01780.01950.9160.117
Appendix
Table 13: Frozen forecasting diagnostics on 30 SF 6 windows (20 forward, 10 reversed). Coverage and width refer to nominal 95% marginal predictive intervals on forward windows. Targets are clean held-out values, and variance includes fitted observation noise, matching the benchmark CRPS. Widths are in training-standardized units. The results are descriptive, from one record with overlapping windows.
MethodSF 6CFC-12CFC-11MeanFwd.
Fixed (RBF)0.026±0.003\mathbf{0.026}\pm 0.0030.268±0.0610.268\pm 0.0610.320±0.0550.320\pm 0.0550.205±0.0270.205\pm 0.0270.098±0.0070.098\pm 0.007
Best fixed ∗0.026±0.0030.026\pm 0.0030.182±0.0530.182\pm 0.0530.134±0.0340.134\pm 0.0340.114±0.0210.114\pm 0.0210.024±0.0020.024\pm 0.002
GSM1.473±0.0471.473\pm 0.0470.812±0.1730.812\pm 0.1730.633±0.1050.633\pm 0.1050.973±0.0690.973\pm 0.0690.690±0.0170.690\pm 0.017
Input warping0.103±0.0100.103\pm 0.0100.143±0.0390.143\pm 0.0390.129±0.0260.129\pm 0.0260.125±0.0160.125\pm 0.0160.057±0.0040.057\pm 0.004
RFF0.052±0.0080.052\pm 0.0080.252±0.0670.252\pm 0.0670.237±0.0330.237\pm 0.0330.181±0.0250.181\pm 0.0250.105±0.0120.105\pm 0.012
CAKE0.027±0.0030.027\pm 0.0030.220±0.0630.220\pm 0.0630.189±0.0510.189\pm 0.0510.145±0.0270.145\pm 0.0270.022±0.002\mathbf{0.022}\pm 0.002
Appendix
Table 14: Mean CRPS ±\pm SEM over 30 held-out windows per record (lower is better). Each unstarred frozen row is selected on validation data. The overall mean weights the three records equally, and Fwd. uses forward-time windows only. ∗ Best fixed selects a fixed baseline separately for each test record. Harmonic–trend selects a discovery by SF 6 test performance only, then evaluates that same discovery on both CFC records. † AutoGP infers a new structure for every window and averages eight predictions.
Figure 16: The same 90 windows, split by time direction. Each point is the equal-record mean CRPS (mean within SF 6 , CFC-12, and CFC-11, then the mean of those three). Forward and reversed windows are separate summaries inside the 48-month benchmark. The final-120-month diagnostic is a different protocol and is not included. Frontier F1–F3 are the validation-selected kernels from the three GPT-6 Astra forecasting campaigns.
ComparatorSF 6CFC-12CFC-11Pooled
Fixed (RBF, ℓ=0.5\ell{=}0.5 )7/30 ( 7.1×10−57.1\times 10^{-5} )26/30 ( 2.0×10−62.0\times 10^{-6} )25/30 ( 2.4×10−52.4\times 10^{-5} )58/90 ( 7.5×10−67.5\times 10^{-6} )
Input warping26/30 ( 2.6×10−72.6\times 10^{-7} )12/30 ( 0.980.98 )15/30 ( 0.420.42 )53/90 ( 8.7×10−58.7\times 10^{-5} )
Random Fourier features14/30 ( 0.520.52 )21/30 ( 0.00660.0066 )27/30 ( 3.9×10−73.9\times 10^{-7} )62/90 ( 4.0×10−74.0\times 10^{-7} )
Grid spectral mixture30/30 ( 1.9×10−91.9\times 10^{-9} )30/30 ( 1.9×10−91.9\times 10^{-9} )29/30 ( 8.0×10−88.0\times 10^{-8} )89/90 ( 2.7×10−162.7\times 10^{-16} )
CAKE2/30 ( 9.3×10−99.3\times 10^{-9} )16/30 ( 0.0520.052 )16/30 ( 0.240.24 )34/90 ( 0.920.92 )
Best fixed per record ∗7/30 ( 7.1×10−57.1\times 10^{-5} )17/30 ( 0.200.20 )14/30 ( 0.720.72 )–
Appendix
Table 15: Paired comparisons of the validation-selected discovery (residual period-bank + poly) against each validation-selected reference, over 30 windows per record. Entries give windows won by the discovery and the two-sided Wilcoxon pp -value. Pooled combines all 90 windows. Windows overlap within records, so pp -values describe variation within records and not independent replications. Holm correction over the five references within each column leaves every 0.05-level conclusion unchanged.
Figure 17: Posterior forecasts of the validation-selected kernels on the median-CRPS window of each record (rows are methods, columns are records). Black dots are the observed prefix, hollow markers are the held-out 48 months, and light grey dots are the rest of the record. Lines and bands are the predictive mean and 95% interval including observation noise, in parts per trillion (ppt). Both CFC windows are reversed, so the forecast runs backward from the dotted origin. Panel CRPS is on the standardized scale of Table 14 .
Figure 18: Forward forecasts of the final 120 months of each record, a horizon 2.5 times longer than the main benchmark (rows are methods, columns are records). Black dots are the observed prefix, hollow markers are the held-out final ten years, and the dotted line is the forecast origin. Lines and bands are the predictive mean and 95% interval including observation noise, in ppt. Panel CRPS is on the training-standardized scale, and bold marks the lowest value in each column.
Figure 19: Additive posterior decomposition of the selected discovery on CFC-12 (peak-origin split, 120-month horizon). Top row: observed and held-out points with the posterior mean and its two-standard-deviation band. Next rows: the Matérn, constant, and combined sinusoid-bank components, then the training residuals. All rows are deviations from the training mean in ppt, so the component means sum to the full posterior mean. The dashed line is the forecast origin, and the split CRPS is 0.0700.070 .
Figure 20: The discovered ChemBench kernel is assembled from interpretable biochemical features. Top: substrate saturation, Arrhenius temperature dependence, and the pH window, each normalized to compare shape. At their frozen weights, the temperature and pH features vary by less than 10−510^{-5} and 0.020.02 across the input box (Appendix F.2 ). Bottom: the inhibitor–substrate coupling ξI i s(a)\xi_{I}\,i\,s(a) and the bisubstrate coupling 12s(a)s(b)\tfrac{1}{2}s(a)s(b) , the fourth and eighth features of Equation ( 24 ), on a log color scale with white contours at each decade. Both grow with substrate concentration because the saturation midpoint lies above the 100100 mM bound.
Figure 21: Matched ChemBench references and discovery campaigns on the same 75 test episodes. Squares mark each reference’s validation-selected initialization, and bars give mean ±\pm SD over five initializations. Each campaign point is its validation-selected frozen discovery, green if validation CRPS is below 0.700.70 and grey otherwise. Blue rings mark the campaign selected overall. The dotted line is fixed Matérn-5/2. Historical campaigns keep their original budgets.
Reference (test CRPS)Ensemble E1 ( 0.5550.555 )Frontier F2 ( 0.3120.312 )
Optimized ARD ( 0.5670.567 )−0.012-0.012 [−0.033,0.010][-0.033,0.010] , p=0.87p=0.87−0.256-0.256 [−0.326,−0.193][-0.326,-0.193] , 73/75, p=6×10−13p=6\times 10^{-13}
Coordinate-aware DKL ( 0.5810.581 )−0.025-0.025 [−0.054,0.001][-0.054,0.001] , p=0.32p=0.32−0.269-0.269 [−0.346,−0.202][-0.346,-0.202] , 71/75, p=2×10−12p=2\times 10^{-12}
Domain library ( 0.5880.588 )−0.032-0.032 [−0.054,−0.011][-0.054,-0.011] , p=0.13p=0.13−0.276-0.276 [−0.347,−0.215][-0.347,-0.215] , 75/75, p=3×10−13p=3\times 10^{-13}
Monotone warping ( 0.6550.655 )−0.100-0.100 , p=3×10−11p=3\times 10^{-11}−0.344-0.344 , 71/75, p=6×10−13p=6\times 10^{-13}
Isotropic Matérn ( 0.7120.712 )−0.156-0.156 , p=8×10−13p=8\times 10^{-13}−0.400-0.400 , 74/75, p=3×10−13p=3\times 10^{-13}
Shared-coordinate DKL ( 0.7830.783 )−0.228-0.228 , p=1×10−12p=1\times 10^{-12}−0.472-0.472 , 74/75, p=3×10−13p=3\times 10^{-13}
Appendix
Table 16: Paired comparison of the validation-selected ChemBench kernels with the matched references over 75 held-out episodes. Entries give the mean CRPS difference (discovery minus reference, negative favors the discovery), its within-mechanism paired-bootstrap 95% interval where computed, episodes won, and the Holm-adjusted two-sided Wilcoxon pp -value across the six references.
Figure 22: Validation CRPS predicts held-out CRPS across ChemBench campaigns. Validation and test CRPS of the validation-selected kernel of each search campaign, nine ensemble and three GPT-6 Astra, on 75 held-out episodes from five mechanisms never seen during search (lower is better, Spearman ρ=0.95\rho=0.95 over the ensemble campaigns). Rings mark the campaign that validation selects within each proposer, and the dashed line is fixed Matérn-5/2 ( ℓ=1\ell{=}1 , test CRPS 0.7210.721 ).
Figure 23: Local predictive behavior on two held-out biochemical mechanisms. In each panel, shading shows ±1.96\pm 1.96 predictive standard deviations. The lower strip shows the absolute error of each posterior mean, tinted where the discovered kernel is closer.
Figure 24: GlucoseBench adult test error for every method and setting. CGM normalized MSE (arithmetic mean over queries within each patient, then geometric mean over 10 patients), colored on a log scale where darker is lower. In the column codes, b kk means kk training episodes and c mm means an mm -minute observed prefix. Columns show b2 c45 , b3 c45 , b4 c45 , and b6 c45 , followed by prefix lengths of 0, 120, and 180 minutes at six training episodes. Rows are the discovered kernels, the tuned references, and the fixed-kernel and persistence baselines. Rel.-time uses time since the meal and bolus as inputs, and ML-II is per-patient marginal-likelihood fitting. A box marks the best colored cell in each column. Persistence is printed under the heatmap and is left off the color scale.
Discoveryb2 c45b3 c45b4 c45b6 c0b6 c45b6 c120b6 c180
Frontier F110, .002 −67%-67\%9, .004 −48%-48\%9, .027 −31%-31\%10, .002 −26%-26\%10, .002 −23%-23\%10, .002 −18%-18\%6, 1.0 −3%-3\%
Frontier F28, .014 −38%-38\%10, .002 −42%-42\%7, .027 −25%-25\%10, .002 −26%-26\%8, .020 −22%-22\%9, .014 −17%-17\%6, .92 −2%-2\%
Ensemble c29, .004 −64%-64\%10, .002 −50%-50\%8, .037 −27%-27\%8, .020 −20%-20\%7, .049 −15%-15\%8, .014 −12%-12\%5, .62 −3%-3\%
Ensemble c110, .002 −47%-47\%6, .28 −19%-19\%5, .77 −10%-10\%8, .027 −16%-16\%7, .23 −10%-10\%7, .56 −3%-3\%2, .020 +12%+12\%
Ensemble c310, .002 −50%-50\%9, .010 −33%-33\%7, .19 −20%-20\%5, 1.0 +2%+2\%5, .77 +0%+0\%5, .77 +4%+4\%2, .037 +28%+28\%
Appendix
Table 17: Paired per-patient comparison with the strongest reference, relative-time ARD-Matérn. In each column, b kk denotes kk training episodes and c mm denotes an mm -minute observed prefix. Each cell gives patients won out of 10, the raw two-sided exact Wilcoxon pp -value, and the percentage change in geometric-mean nMSE. Negative percentages are lower error. A win count of 5 is an even split. Holm adjustment over these 35 cells leaves every adjusted pp -value above 0.050.05 (minimum 0.0680.068 ).
Figure 25: The selected glucose kernel encodes a delayed meal effect and a fading insulin effect. The two gates of the ensemble campaign-2 feature map after tuning, as functions of time since the meal and since the bolus. Dotted lines mark the 46-minute midpoint of the meal gate and the 23-minute start of the insulin gate. After that start, insulin decays with a time constant of 127 minutes.
Figure 26: With two training episodes the reference undershoots the meal response, with six the gap closes. One adult and one held-out query, forecast after the 45-minute prefix (dotted line) from 2 versus 6 training episodes. Curves are posterior means with 95% intervals for the validation-selected Astra kernel (campaign 1) and relative-time ARD-Matérn. Lower panels show the delivered meal and bolus. The patient is a median case for the ensemble kernel.
Figure 27: Cross-domain reuse of validation-selected kernels. Rows are source kernels and columns are target domains. Each cell gives the target test CRPS change (%) relative to that domain’s fixed reference and the number of evaluation units won. Negative CRPS changes mean lower error and better performance. Purple favors the transferred kernel and brown favors the reference. Stars mark unadjusted paired tests with p<0.05p<0.05 . † The chemistry-source cells use the fixed coordinate conversion described below, so read them as interface-compatibility checks.
KernelReportWhat tuning settled
DWF (BBO)The objective varies smoothly, as under a Matérn-5/2 prior, along a slightly rescaled copy of each coordinate. Values at points mirrored about the middle of a coordinate, uu and 1.025−u1.025-u , are encouraged but not forced to agree. The prior has a kink at the mirror line u=0.513u=0.513 .Tuned geometry: the fold frequency settled at ν=0.975\nu=0.975 , so the fold is a single tent peaking at 0.5130.513 that reflects each coordinate once, and the warp stays within 0.0270.027 of the identity (Appendix C.2 ). The registration had anticipated multi-scale periodicity and boundary sensitivity, which the tuned kernel does not use. The kink is Proposition 4 .
Residual period-bank + poly (forecasting)The record follows a smooth trend, as under a Matérn-5/2 prior with lengthscale 0.50.5 of the normalized span, plus a constant offset, plus a second smooth component built from 36 sinusoids. Every sinusoid has a period between 7 and 47 times the length of the record, so within one record this component acts as a record-wide smooth trend rather than a cycle.Tuned behavior: the sinusoid frequencies settled at 0.0210.021 – 0.1430.143 cycles per normalized span, so the bank models slow curvature across the whole record. The registration described an annual cycle, but it took the annual period (about 0.030.03 of the span) as a frequency. An annual cycle would need 25–47 cycles per span. The “poly” feature is a constant.
ChemBench, ensemble E1 (enzyme kinetics)The rate depends smoothly on a monotone transform of log substrate concentration that is steepest at high concentration, the same transform of the second substrate, their product, and log enzyme loading. Inhibitor and product concentrations matter only in proportion to substrate saturation.Tuned relevance: as in ARD, tuning decides which inputs matter. Enzyme loading and the two substrates matter most, while inhibitor and product matter little (Figure 28 ), and the saturation midpoint lies above the input range (Figure 20 ). The program includes Arrhenius and pH-window features, but tuning set their weights so that the correlation is 1.0000001.000000 at every base point. These probes show little sensitivity to temperature and pH, without establishing exact invariance.
Appendix
Table 18: Plain-language reports of the headline discoveries. Each report is condensed from the agent’s registered formulation and checked against the frozen program with the trusted interpreter. The right column states what tuning settled.
KernelReportWhat tuning settled
ChemBench, frontier F2 (enzyme kinetics)An explicit 248-feature map treats the rate as a linear combination of ten substrate-response shapes modeled on the training mechanisms, each at three substrate scales, their log-substrate sensitivities, and a small linear backbone. The response features are multiplied by enzyme, temperature, and pH factors, so posterior coefficients can select both mechanism shape and environmental modulation.Tuned relevance: enzyme loading and substrate dominate, product, inhibitor, and temperature matter less, and pH is nearly ignored (Figure 28 ). The frozen kernel therefore retains strong enzyme and substrate effects, some product and inhibitor response, and almost no pH dependence.
GlucoseBench, frontier F1An ARD-Matérn-5/2 residual is augmented by 36 intervention features. At three response scales, gamma envelopes encode causal meal and bolus transients, long-period wave packets encode response age, and cumulative channels retain dose exposure after the transients fade. Signed and complementary channels allow meal and bolus responses to oppose or covary.Tuned behavior: signed channels dominate because their mixing parameter is ρ=0.20\rho=0.20 , while cumulative memory remains active with gain 0.560.56 . The base kernel’s time lengthscale is 3.213.21 normalized spans and its four intervention lengthscales are 10610^{6} . On public training inputs, endpoint-sweep correlations are 0.900.90 for time, 0.940.94 for meal size, 0.960.96 for meal start, and 0.980.98 for bolus size and start.
Appendix
Table 19: Plain-language reports of the headline discoveries, continued.
Figure 28: The two ChemBench kernels respond to different inputs. Mean kernel correlation as one normalized input moves from 00 to vv with the others fixed, over 100 base points. A curve near 11 indicates similar normalized kernel features along the sampled sweeps. Left: ensemble campaign E1 depends mainly on enzyme loading and the two substrates, responds little to inhibitor and product, and shows almost no temperature or pH sensitivity in these sweeps. Right: frontier campaign F2, which uses GPT-6 Astra, depends on enzyme loading and substrate, responds mildly to the product, inhibitor, and temperature, and shows almost no pH sensitivity in these sweeps.

Explore similar work

CardsList

  1. Automated Kernel Discovery Towards Understanding High-dimensional Bayesian Optimization

    May 18, 2026Taeyoung Yun, Woocheol Shin, Inhyuck Song +2Bayesian OptimizationGaussian Process

    1Korea Advanced Institute of Science and Technology (KAIST)

  2. KernelGenBench: Can LLMs and Agents Write Efficient Kernels Across Operator Sources and Hardware Platforms?

    Jul 22, 2026Peiyu Zang, Jian Tao, Jialing Zhang +4Graphics Processing Unit KernelsCuda

    Beijing Normal University · BAAI · Beijing Jiaotong University +2

  3. AdaExplore: Failure-Driven Adaptation and Diversity-Preserving Search for Efficient Kernel Generation

    Apr 17, 2026Weihua Du, Jingming Zhuo, Yixin Dong +9Agentic OptimizationCode Optimization

    1Carnegie Mellon University · University of Washington · 3Arm Ltd.