To design a periodic material, engineers choose the shape and dimensions of a repeating cell to meet target properties. With few simulations, the benefit of a learned predictor can be hard to separate from the effect of which designs the search tries and how it uses simulator feedback. We compare two ways to predict properties in a library of 80 cell architectures, with up to 30 simulator calls per target. For each architecture, both searches use a table of predicted properties at different dimensions. One fills the table with predictions from neural networks. The other fits a simple model to the existing simulations for that architecture. Both searches use the difference between the predicted and simulated properties to adjust the next design. We generated 200 targets from simulated designs after fixing the analysis plan. The two searches showed no detectable difference in how often they matched all three properties within 2% or in the mean error of their returned designs. The upper limit of the 95% confidence interval for the gain in overall success rate was 4.5 percentage points. Follow-up controls on these targets changed success from 83.5% to 36.0% with a degraded table and 96.5% with an oracle table. In separate tests, using the simulation result to adjust the next design increased success for both types of table. Results cover one library and a coarse 32^3 voxel simulator. Designers can use this comparison to judge what a learned predictor adds when the search, feedback and budget are the same.
Fig. 1: Design space and target difficulty. a Gyroid sheet and network, three gyroid–Schwarz P network blends, Schwarz P network and octet lattice, drawn as smooth, closed isosurfaces of the underlying geometry evaluated on a 1283 grid at nominal density 0.30. The gyroid sheet drawing omits two small corner pieces (0.3% of its surface) connected to the body only through periodic cell faces; no other drawn cell has such pieces. Simulations use the 323 voxel model; after connected-component filtering its realised density is 0.300 for the level-set examples and 0.354 for the octet. The blend weight on Schwarz P increases from 0.25 to 0.75 between the pure limits. b,c The 358 catalogue designs and 36 test targets in two response planes. Young’s modulus E11 , Voigt bulk modulus Kbulk and conductivity k11 are dimensionless relative to the solid. Insets show TPMS target 2 of Fig. 8 , its 2% box and its ringed nearest catalogue design. This target’s retrieval error is closest to the median across targets. d Nearest-catalogue design error for every target: the largest ∣log10(simulated/target)∣ of the three responses, in log units, with horizontal jitter within each system and the tolerance dashed. No catalogue design meets any target. TPMS denotes triply periodic minimal surface
Set
Targets
Reachable at 323
Role
Development
12
Yes
Select the learned configuration
Confirmatory
200
Yes
Planned table comparison and separate follow-up controls
Test
36
Yes
Benchmark search against retrieval and standard optimisers
Fresh draw
36
Yes
Repeat the benchmark on another draw and check resolution
Interpolated and 643
36 each
Unknown
Test performance when reachability is uncertain
Table 2: Target sets and their role in the evaluation
Fig. 2: Table inversion and residual correction for blend target 1 in Fig. 8 , first seed. a Target, design space, catalogue and budget. b The ensemble supplies one response table per architecture. c Table scores and simulated design errors for the first batch of ten calls, with an architecture key (fcc denotes face-centred cubic). Four designs in this batch are invalid and still count against the budget. The continuation shows panels d,e
Fig. 3: (continued) d The query trace and five proposals in the gyroid–Schwarz P network. Each valid simulated design updates the correction for its architecture. Invalid queries are drawn at the error cap. Call 26 first meets the tolerance, and the last batch brings the total to 27 calls, 11 invalid. e The returned design is drawn as a smooth, closed isosurface of the underlying blended level-set field evaluated on a 1283 grid at its nominal density. Simulations and re-simulation use the 323 voxel model of the same geometry
Comparison
Change assessed
Main evidence
Learned / catalogue tables
Table construction under matched search and feedback
Planned comparison, confirmatory set
Corrected / uncorrected proposal
Residual use to choose the next design under matched decision rules
Test set and post-plan controls on the fresh and confirmatory sets
Degraded / catalogue / oracle tables
Response to larger changes in table quality
Post-plan controls, confirmatory set
Retrieval with and without pattern search
Charged local search within retrieved architectures
Test set
Catalogue-table search / retrieval with pattern search
Table-guided starts, refinement and allocation as a bundle
Test set
Broad / three-architecture search
Architecture allocation conditional on the table type
Test set and secondary confirmatory contrasts
Table 3: Comparison arms, changes assessed and principal target sets
Fig. 4: Declared comparison and exploratory contrasts. a Success on 200 reachable targets (50 TPMS, 50 strut, 100 blend). Bars average seeds, dots show individual seeds and black lines give 95% target-bootstrap intervals. Stochastic configurations use ten seeds, deterministic configurations count once. b,c Learned-table minus catalogue-table differences in success rate and design error. Filled circles denote all targets and open squares blends. Intervals resample targets 20,000 times within system and within set for pooled sets. The shaded confirmatory row uses the declared primary comparison. Smaller-set contrasts are post hoc and unadjusted for multiplicity. All searches use at most 30 simulator calls per target and zero language-model calls. TPMS denotes triply periodic minimal surface
Fig. 5: Search progress on the 54 blend runs. a Share of runs met by each simulator call, with pooled-seed curves thick and single-seed curves thin. b Median best design error, with the last value carried forward after stopping. The vertical marker ends the first batch of ten calls. Bars to the right of call 30 give the interquartile range and median for every configuration, and the horizontal threshold is τ=log10(1.02) , the 2% tolerance. These bars represent call 30. Line styles and markers identify configurations in greyscale. Learned-table search and the uncorrected-proposal control have identical first-batch traces
Fig. 6: Table-quality controls and held-out architectures on 200 confirmatory targets. a Success by system and overall; hatching marks held-out architectures. Bars average seeds, dots show seeds and black lines give 95% target-bootstrap intervals. b Paired success differences for all targets (filled circles) and blends (open squares). c Blend success against median table error at reference designs; the dashed line marks the 2% tolerance. Search rules and the 323 simulator are unchanged, with at most 30 calls and zero language-model calls. Learned arms use seeds 0–2; Fig. 4 uses ten. Only the oracle uses extra simulator data. These post-plan controls are not declared confirmatory endpoints. Intervals use 20,000 target resamples without multiplicity adjustment. TPMS denotes triply periodic minimal surface
Configuration
Runs met
Success rate
Design error
Calls to success
Simulator calls per target
Learned-table search
84/108
0.778 (0.139) [0.676, 0.880]
0.00639 (0.00065) [0.00471, 0.00848]
13.19 (0.22)
17.03
Catalogue-table search
78/108
0.722 (—) [0.611, 0.833]
0.00800 (—) [0.00463, 0.01374]
13.86 (—)
17.86
Bracketing search
72/108
0.667 (—)
0.01138 (—)
12.36 (—)
12.03
Bayesian optimisation
71/108
0.657 (0.083)
0.00958 (0.00100)
14.99 (2.56)
14.65
Nelder–Mead search
69/108
0.639 (—)
0.01627 (—)
14.97 (—)
14.61
Retrieval with pattern search
23/108
0.213 (0.028) [0.102, 0.333]
0.03264 (0.00180) [0.02270, 0.04439]
26.47 (0.39)
29.56
Table 4: Test comparison over 36 targets × three seeds, with at most 30 simulator calls per target. Means are followed by seed ranges in parentheses. A dash denotes a deterministic configuration. Bracketed entries give 95% target-bootstrap intervals. Deterministic counts correspond to 26/36 catalogue-table, 24/36 bracketing, 23/36 Nelder–Mead and 0/36 retrieval targets. Boundary intervals carry no information about new targets. All configurations use zero language-model calls. Calls to success includes failures at 31. Per-system results are in the Supplementary Information
Fig. 7: Test results at a budget of 30 simulator calls. a Success rate by target system and overall. Bars average three seeds, and dots show each seed. b Mean design error on a log scale. The dashed line marks 2%, and floor triangles mark means below 10−4 . c Fraction of runs with a successful query by each call. Thick curves pool seeds, and thin curves show each seed. The retrieval control used zero calls and met zero targets. TPMS denotes triply periodic minimal surface
Fig. 8: Verified successes by test target. Columns follow benchmark order within each system. Each entry shows how many of three seeds met the tolerance after re-simulation of the returned design. The right columns total successes over all 108 runs and over the 54 blend runs. Deterministic variants can show only zero or three successes for a target. TPMS denotes triply periodic minimal surface
Fig. 9: Ablations and sensitivity analyses under the common call budget. a,b Mean design error over all runs and over blend runs. c Blend success rate. d Mean calls to success over all runs. Diamonds show means, connected points show seed results, and the vertical band and line show the learned-table search’s seed range and mean. Deterministic variants have a single point. The threshold in a,b is τ , the 2% tolerance. These displays show seed variation. Target-level intervals are reported in the Supplementary Information
Fig. 10: Target sets and matched stop rules. a Success rates for the seven fixed configurations on the test set, fresh draw, interpolated responses and 643 responses. Each has 36 targets and three seeds, at most 30 simulator calls per target and a 323 search simulator. Bars give means, dots give seeds and numbers give runs met. Reachability is unknown for the last two sets. b,c Test-set and fresh-draw successes at 2%, 3% and 5%. Filled markers use the matched stop rule. Larger open markers show 2%-stop-rule returns re-scored where the counts differ. The vertical axes start at 15 runs. Retrieval with pattern search uses re-scoring because it has no early stop. TPMS denotes triply periodic minimal surface
Fig. 11: Grid-resolution checks. a Fresh-draw designs returned under the 2% stop rule, assessed at 2%, 3% and 5%. Solid lines compare designs and targets on 323 , dashed lines compare the same designs and reference responses on 643 . b Absolute response changes over distinct returned test-set designs. Points show designs with horizontal jitter, bars show means and the dashed line is 2%. Marker shapes and colours identify configurations. These panels measure the 32-to-64 shift, with finer-grid convergence untested
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Runs met
Design error
Success rate
Calls to success
Simulator calls
Variant
all (108)
blend (54)
all
blend
blend
all
per target
Reference
Learned-table search
84
32
0.00639 (0.00065)
0.01046 (0.00098)
0.593 (0.222)
13.19 (0.22)
17.03
Learned component
Catalogue-table search
78
24
0.00800 (—)
0.01391 (—)
0.444 (—)
13.86 (—)
17.86
Categorical-encoding search
79
25
0.01053 (0.00168)
0.01904 (0.00350)
0.463 (0.333)
14.22 (2.75)
17.48
Appendix
Table S1: Component comparisons and sensitivity analyses on the test set. Each variant uses at most 30 simulator calls per target. Metric means are followed by seed ranges in parentheses where shown. A dash denotes a deterministic configuration.
TPMS targets (27 runs)
Strut targets (27 runs)
Variant
Runs met
Design error
Calls to success
Runs met
Design error
Calls to success
Reference
Learned-table search
25
0.00463
7.37
27
<10−4
2.96
Learned component
Catalogue-table search
27
0.00419
4.67
27
<10−4
4.33
Categorical-encoding search
27
0.00405
7.56
27
<10−4
2.19
Appendix
Table S2: Component comparisons and sensitivity analyses on TPMS and strut targets. Each system has 27 test runs. TPMS denotes triply periodic minimal surface
Contrast ( a−b )
Difference
95% interval
Targets better / worse / tied
Sign / Wilcoxon p
Learned-table search − retrieval with pattern search
− 0.02625
[ − 0.03751, − 0.01697]
32 / 0 / 4
< 0.001 / < 0.001
All targets have runs met 84 versus 23, with 61 versus 0 met by one variant only. Seed ranges are 0.00065 and 0.00180. The difference exceeds both seed ranges.
Learned-table search − retrieval with pattern search
− 0.03443
[ − 0.05509, − 0.01820]
18 / 0 / 0
< 0.001 / < 0.001
Blend targets have runs met 32 versus 2, with 30 versus 0 met by one variant only. Seed ranges are 0.00098 and 0.00357. The difference exceeds both seed ranges.
Learned-table search − catalogue-table search
− 0.00161
[ − 0.00605, + 0.00122]
13 / 11 / 12
0.839 / 0.900
All targets have runs met 84 versus 78, with 9 versus 3 met by one variant only. Seed ranges are 0.00065 and 0.00000. The difference exceeds both seed ranges.
Appendix
Table S3: Paired design-error contrasts over all test targets and over blends. Differences subtract the mean of b from that of a . Exceeds both seed ranges indicates that the absolute difference exceeds both seed ranges. The 95% intervals bootstrap paired target means. Sign and Wilcoxon signed-rank tests ( Wilcoxon, 1945 ) use paired target means and omit ties. Boundary intervals carry no information about new targets.
Contrast ( a−b )
Difference
95% interval
Targets better / worse / tied
Sign p
Learned-table search − retrieval with pattern search
+ 0.565
[ + 0.417, + 0.704]
25 / 0 / 11
< 0.001
All targets have runs met 84 versus 23, with 61 versus 0 met by one variant only. Seed ranges are 0.139 and 0.028. The difference exceeds both seed ranges.
Learned-table search − retrieval with pattern search
+ 0.556
[ + 0.370, + 0.741]
14 / 0 / 4
< 0.001
Blend targets have runs met 32 versus 2, with 30 versus 0 met by one variant only. Seed ranges are 0.222 and 0.056. The difference exceeds both seed ranges.
Learned-table search − catalogue-table search
+ 0.056
[ − 0.019, + 0.139]
6 / 2 / 28
0.289
All targets have runs met 84 versus 78, with 9 versus 3 met by one variant only. Seed ranges are 0.139 and 0.000. The difference does not exceed both seed ranges.
Appendix
Table S4: Paired success-rate contrasts over all test targets and over blends. Differences subtract the mean of b from that of a . Exceeds both seed ranges indicates that the absolute difference exceeds both seed ranges. The 95% intervals bootstrap paired target means. The exact sign test uses paired target means and omits ties. Boundary intervals carry no information about new targets.
Variant
Runs met all (36)
Runs met blend (18)
Design error all
Design error blend
Learned-table search
27, 26, 31
10, 9, 13
0.00645, 0.00669, 0.00604
0.01004, 0.01103, 0.01033
Catalogue-table search
26, 26, 26
8, 8, 8
0.00800, 0.00800, 0.00800
0.01391, 0.01391, 0.01391
Categorical-encoding search
24, 25, 30
6, 7, 12
0.00948, 0.01116, 0.01095
0.01681, 0.02031, 0.01998
Uncorrected-proposal control
22, 22, 21
6, 6, 5
0.01342, 0.01427, 0.01536
0.02283, 0.02440, 0.02649
Three-architecture catalogue search
28, 28, 28
10, 10, 10
0.01010, 0.01010, 0.01010
0.01779, 0.01779, 0.01779
First batch of 6 architectures
28, 27, 32
11, 10, 14
0.00698, 0.00672, 0.00505
0.01104, 0.01087, 0.00794
Appendix
Table S5: Test-set results by seed, in order 0, 1 and 2. Each seed contributes 36 runs, including 18 blend runs. The catalogue-only searches and the retrieval control are deterministic
All targets (108 runs)
Blend targets (54 runs)
Variant
2 %
3 %
5 %
2 %
3 %
5 %
Learned-table search
84
94
100
32
42
47
Catalogue-table search
78
99
105
24
45
51
Categorical-encoding search
79
91
98
25
37
44
Uncorrected-proposal control
65
75
87
17
27
36
Three-architecture catalogue search
84
96
99
30
42
45
Appendix
Table S6: Verified successes at relative tolerances of 2%, 3% and 5%. The first panel re-scores designs returned under the 2% stop rule. The second panel gives matched-stop-rule runs, with re-scored counts in parentheses at 3% and 5%. Every row uses at most 30 simulator calls per target and zero language-model calls
Variant
Targets met (TPMS / strut / blend)
Design error all
Design error blend
Calls to success
Simulator calls per target
Reference
Learned-table search
10/12 (3 / 3 / 4)
0.00607
0.00986
11.17
15.17
Learned component
Catalogue-table search
11/12 (3 / 3 / 5)
0.00944
0.01649
8.17
13.67
Categorical-encoding search
11/12 (3 / 3 / 5)
0.01149
0.01996
12.58
17.00
Role of the learned tables
Appendix
Table S7: Development-set outcomes for 12 targets (3 TPMS, 3 strut and 6 blend) with one seed and at most 30 simulator calls per target. TPMS denotes triply periodic minimal surface
Recall of the true architecture
Held-out RMSE ( log10 )
Model
Rank 3
Rank 10
Rank 20
E11
Kbulk
k11
All designs (358)
Learned tables
0.301 (0.031)
0.519 (0.011)
0.703 (0.022)
0.144
0.136
0.093
Categorical-encoding tables
0.334 (0.020)
0.550 (0.031)
0.661 (0.025)
0.143
0.138
0.097
Catalogue-fitted tables
0.388 (—)
0.486 (—)
0.542 (—)
0.316
0.308
0.211
Nearest catalogue design
0.034 (—)
0.137 (—)
0.268 (—)
–
–
–
Appendix
Table S8: Five-fold cross-validation on 358 catalogue designs with zero additional simulator calls. Recall at rank k is the share of held-out designs whose source architecture is in the first k ranks. Entries show means with ranges over three model seeds in parentheses. RMSE is root-mean-square error in log properties. The properties are axial Young’s modulus E11 , Voigt bulk modulus Kbulk and axial conductivity k11 . Catalogue-fitted tables can predict a held-out design only if its architecture retains another design in the training fold
Configuration
Runs met
Success rate
Design error
Calls to success
Simulator calls per target
TPMS targets
Learned-table search
25/27
0.926 (0.111) [0.778, 1.000]
0.00463 (0.00220) [0.00237, 0.00787]
7.37 (0.11)
12.41
Catalogue-table search
27/27
1.000 (—) [1.000, 1.000]
0.00419 (—) [0.00277, 0.00561]
4.67 (—)
11.22
Bracketing search
24/27
0.889 (—)
0.00385 (—)
5.78 (—)
5.67
Bayesian optimisation
23/27
0.852 (0.111)
0.00508 (0.00162)
8.81 (5.67)
8.67
Nelder–Mead search
24/27
0.889 (—)
0.00642 (—)
7.89 (—)
7.78
Appendix
Table S9: Primary comparison arms by target system. Each TPMS and strut block has 9 targets × three seeds, and the blend block has 18 × three. Means, ranges, intervals and failure convention follow Table 4 of the article. TPMS denotes triply periodic minimal surface
Setting
Value
Status
First-batch architectures
10 (3, 6, 15 in analysis)
Varied
Networks in ensemble
3 (1, 9 in analysis)
Varied
Maximum from one system in first batch
6
Fixed
Later-batch allocation
3 refinements and 2 new proposals
Fixed
Initial window half-width
One quarter of each parameter range
Fixed
Window contraction
Half-width halved per additional valid observation
Fixed
Appendix
Table S10: Search and model constants. Varied entries were sensitivity analyses on the test targets. Fixed values were neither tuned nor sensitivity-tested. The call allowance and success tolerance are benchmark requirements
Configuration
Met
Met by system
Design error
Success rate (range)
Calls to success
Simulator calls
Met at 3% / 5%
Fresh reachable draw
Learned-table search
94/108
24 / 27 / 43
0.01230
0.870 (0.056)
10.63
14.76
99 / 103
Catalogue-table search
87/108
24 / 27 / 36
0.01326
0.806 (—)
11.58
16.06
93 / 102
Three-architecture catalogue search
93/108
21 / 27 / 45
0.01389
0.861 (—)
8.67
9.56
93 / 99
Bracketing search
87/108
24 / 27 / 36
0.01121
0.806 (—)
8.83
8.64
93 / 96
Bayesian optimisation
84/108
23 / 27 / 34
0.01337
0.778 (0.083)
11.52
11.30
94 / 102
Appendix
Table S11: Unchanged configurations on additional sets, each 36 targets × three seeds and at most 30 search calls per target. System counts are TPMS / strut / blend (out of 27 / 27 / 54). Searches use 323 responses for every set. The last column re-scores returned designs under the 2% stop rule. Ranges describe seed sensitivity and dashes denote deterministic configurations. Deterministic targets met out of 36 equal the reported runs met divided by three. All configurations use zero language-model calls. TPMS denotes triply periodic minimal surface
System (targets)
Success rate
Design error
Calls to success
Test set. Learned-table search minus Catalogue-table search
All (36)
+0.056 [-0.019, 0.139] p=0.289
-0.00161 [-0.00605, 0.00122] p=0.839
-0.68 [-2.95, 1.44] p=0.701
TPMS (9)
-0.074 [-0.222, 0.000] p=1.000
+0.00044 [-0.00237, 0.00355] p=1.000
+2.70 [-1.56, 7.41] p=0.070
Strut (9)
+0.000 [0.000, 0.000] —
+0.00000 [0.00000, 0.00000] —
-1.37 [-5.37, 1.48] p=1.000
Blend (18)
+0.148 [0.019, 0.296] p=0.125
-0.00344 [-0.01212, 0.00183] p=0.629
-2.02 [-5.50, 1.15] p=0.424
Test set. Learned-table search minus Bracketing search
Appendix
Table S13: Paired contrasts for the test set, additional sets and pooled reachable targets. Entries give mean difference, 95% target-bootstrap interval and two-sided sign-test p value, with seeds averaged within target. Positive success and negative error or calls favour the first configuration. Targets are resampled within system and within set for pooling. These post hoc contrasts are unadjusted for multiplicity. Every search uses at most 30 simulator calls per target and zero language-model calls. Boundary intervals carry no information about performance on new targets.
System
Met / runs
Success [95% interval]
Design error
Calls to success
Simulator calls
Learned-table search
All
1687/2000
0.844 [0.804, 0.881]
0.00669
10.82
15.27
TPMS
494/500
0.988 [0.964, 1.000]
0.00343
5.64
11.33
Strut
490/500
0.980 [0.942, 1.000]
0.00063
3.27
10.85
Blend
703/1000
0.703 [0.626, 0.776]
0.01135
17.18
19.46
Catalogue-table search
Appendix
Table S14: Confirmatory results by system. Stochastic configurations use ten seeds. Deterministic configurations count each target once after identical two-seed runs. Success intervals bootstrap targets. Means include failures at 31 calls to success. All configurations use at most 30 simulator calls per target and zero language-model calls.
System (targets)
Success rate
Design error
Calls to success
Learned-table search minus Catalogue-table search
All (200)
+0.008 [-0.027, 0.045] p=0.233
+0.00002 [-0.00104, 0.00099] p=0.618
+0.21 [-0.65, 1.07] p=0.019
TPMS (50)
+0.008 [-0.006, 0.030] p=1.000
-0.00032 [-0.00110, 0.00045] p=0.878
+0.68 [-0.37, 1.71] p=0.004
Strut (50)
-0.020 [-0.058, 0.000] p=0.500
+0.00022 [-0.00041, 0.00104] p=1.000
+0.13 [-1.12, 1.36] p=0.248
Blend (100)
+0.023 [-0.044, 0.093] p=0.349
+0.00008 [-0.00195, 0.00195] p=0.475
+0.01 [-1.49, 1.51] p=0.826
Learned-table search minus Three-architecture catalogue search
Appendix
Table S15: Confirmatory paired contrasts, formatted as in Table S13 . The primary contrast is learned-table minus catalogue-table search, with success and design error overall and on blends as four declared endpoints. Other contrasts and calls are descriptive. All intervals and tests are unadjusted for multiplicity. Boundary intervals carry no information about new targets. Simulator allowance is 30 per target and language-model calls are zero.
Tables, targets
Runs met
Success difference
95% interval
Targets better / worse ( p )
Error difference
95% interval
Test set (36 targets)
Learned tables, all
84/108 vs 65/108
+ 0.176
[ + 0.093, + 0.278]
11 / 0 ( < 0.001)
− 0.00796
[ − 0.01338, − 0.00355]
Learned tables, blend
32/54 vs 17/54
+ 0.278
[ + 0.130, + 0.444]
9 / 0 (0.004)
− 0.01411
[ − 0.02447, − 0.00557]
Catalogue tables, all
78/108 vs 57/108
+ 0.194
[ + 0.083, + 0.333]
7 / 0 (0.016)
− 0.01885
[ − 0.03589, − 0.00593]
Catalogue tables, blend
24/54 vs 12/54
+ 0.222
[ + 0.056, + 0.444]
4 / 0 (0.125)
− 0.03436
[ − 0.06829, − 0.00868]
Fresh draw (36 targets)
Appendix
Table S16: Corrected minus uncorrected refinement proposal for both table types. The control keeps every decision rule of the search and takes the queried design from the uncorrected table in the same window. Runs met are given as corrected vs uncorrected. Differences in success rate and design error average seeds within target; the 95% intervals resample targets 20,000 times within system. The sign test counts targets better / worse for the corrected proposal, with its two-sided p value in parentheses. Learned tables use three seeds on the 36-target sets and ten seeds on the confirmatory set; catalogue tables are deterministic and count once per target on the confirmatory set. The learned-table rows of the test set repeat the contrast of Tables S3 and S4 . All other uncorrected-proposal runs were added after the confirmatory plan and are not among its declared configurations. All runs use at most 30 simulator calls per target and zero language-model calls
System
Met / runs
Success [95% interval]
Design error
Calls to success
Simulator calls
Degraded table
All
72/200
0.360 [0.295, 0.420]
0.02054
25.98
26.32
TPMS
27/50
0.540 [0.400, 0.680]
0.01254
23.40
24.32
Strut
26/50
0.520 [0.380, 0.660]
0.01668
23.50
24.70
Blend
19/100
0.190 [0.120, 0.270]
0.02647
28.52
28.13
Catalogue tables, full catalogue
Appendix
Table S18: Table-quality controls and held-out architectures on the 200 confirmatory targets (50 TPMS, 50 strut, 100 blend). All searches keep the same schedule, at most 30 simulator calls, a 2% stop rule and the 323 simulator, with zero language-model calls. Learned arms use seeds 0–2; deterministic arms count once. Intervals bootstrap targets within system. The degraded table uses one density fit per system, shared by all its architectures. Only the oracle uses extra simulator data (6,048 construction calls outside search budgets). The held-out test removes 38 designs of 14 blend architectures, leaving 320 designs, and rebuilds both table types. These controls were planned after the confirmatory results were known and are not declared endpoints. All 2,400 existing runs were first reproduced identically. TPMS denotes triply periodic minimal surface
Targets
n
Success difference
95% interval
Error difference
95% interval
Degraded minus catalogue tables
All
200
−0.475
[−0.545,−0.405]
+0.01386
[0.01132,0.01654]
Blend
100
−0.490
[−0.590,−0.380]
+0.01520
[0.01192,0.01870]
Oracle minus catalogue tables
All
200
+0.130
[0.085,0.180]
−0.00351
[−0.00487,−0.00238]
Blend
100
+0.270
[0.190,0.360]
−0.00650
[−0.00915,−0.00434]
Appendix
Table S19: Paired table-control contrasts on the confirmatory targets. Entries are the first arm minus the second, averaged over seeds within target; intervals are 95% percentile target-bootstrap intervals (20,000 resamples within system, or within subgroup), unadjusted for multiplicity. Full-catalogue and held-out learned arms use seeds 0–2, and deterministic arms count once. These post-plan controls are not declared confirmatory endpoints. All searches have at most 30 simulator calls per target and zero language-model calls; the oracle’s extra construction calls are stated in Table S18 . The four held-out endpoints (success rate and design error, all and blend) detect no difference. The subgroup of 22 held-out-reference targets leaves smaller benefits unresolved. One random subset of 14 architectures was tested; the planned 28-architecture level was not run because the earlier stages exceeded the plan’s time limit. The coarse oracle table still misses seven targets. No intermediate degradation was tested.
Many of the most important problems in science and engineering are inverse problems: given a desired outcome, find a design that achieves it. Evaluating whether a candidate meets the spec is often routine; a binding energy can be computed, a reactor yield simulated, a pharmacokinetic profile predicted. But searching a combinatorial design space for inputs that satisfy those targets is fundamentally harder. We introduce SciDesignBench, a benchmark of 520 simulator-grounded tasks across 14 scientific domains and five settings spanning single-shot design, short-horizon feedback, long-horizon refinement, and seed-design optimization. On the 10-domain shared-core subset, the best zero-shot model reaches only 29.0% success despite substantially higher parse rates. Simulator feedback helps, but the leaderboard changes with horizon: Sonnet 4.5 is strongest in one-turn de novo design, whereas Opus 4.6 is strongest after 20 turns of simulator-grounded refinement. Providing a starting seed design reshuffles the leaderboard again, demonstrating that constrained modification requires a fundamentally different capability from unconstrained de novo generation. We then introduce RLSF, a simulator-feedback training recipe. An RLSF-tuned 8B model raises single-turn success rates by 8-17 percentage points across three domains. Together, these results position simulator-grounded inverse design as both a benchmark for scientific reasoning and a practical substrate for amortizing expensive test-time compute into model weights.
Prediction-based approaches are widely used in neural architecture search (NAS), where a predictor estimates the performance of candidate architectures to guide selection. However, existing predictors are typically trained via supervised regression on limited samples, leading to overfitting and poor generalization to unseen architectures. In this work, we propose a fundamentally different formulation that models performance prediction as a conditional function inference problem using a Convolutional Neural Process (ConvNP) with meta-learning capabilities. Instead of fitting a fixed mapping to limited samples, our approach meta-learns to infer performance from partial observations by training with context-target splits across a group of synthesized tasks, explicitly optimizing for generalization under data scarcity and aligning the training procedure with the deployment setting in NAS. We further design simple yet effective meta-features for cell-based architectures and evaluate our method on NAS-Bench-101 and NAS-Bench-201. Extensive experiments show that our approach consistently improves top-K ranking quality and achieves the state-of-the-art architecture selection using limited samples.
Liping Deng, MingQing Xiao
Department of Mathematics University of California, Riverside Riverside, CA 92507 USA · School of Mathematical and Statistical Sciences Southern Illinois University Carbondale Carbondale, IL 62901 USA
Human-imposed design conventions can restrict the alternatives represented in scientific training data and shape the search preferences learned by generative models. We show that scientific performance feedback can redirect the learned search distribution toward productive regions beyond those conventions, without explicitly rewarding departures from them. We test this process in reactor lattice design with simulated feedback and chemical reaction-condition selection with measured yields, using a pretrained sequence policy and a small non-language policy, respectively. In both settings, adaptation improves scientific performance while increasing generation of choices excluded from the corresponding supervised data. In measured reactions, probability on task-specific ligand choices excluded from training rises from 0.29 to 0.70-0.71 as expected yield error decreases. Controls in the lattice setting show that the productive region identified during adaptation provides reusable information to random search, while aligned generation produces useful candidates more frequently than informed random sampling. Removing the target penalty redirects four of five matched runs, linking the direction of adaptation to the scientific objective. Together, these results show how scientific feedback reshapes inherited search preferences and reveals productive alternatives beyond training conventions, with attribution controls distinguishing learned redistribution from candidate availability and reusable search information.