Separating learning from search and feedback in budgeted inverse design of periodic unit cells
Organizations: Santa Clara University, Santa Clara, CA 95053, USA
Abstract
To design a periodic material, engineers choose the shape and dimensions of a repeating cell to meet target properties. With few simulations, the benefit of a learned predictor can be hard to separate from the effect of which designs the search tries and how it uses simulator feedback. We compare two ways to predict properties in a library of 80 cell architectures, with up to 30 simulator calls per target. For each architecture, both searches use a table of predicted properties at different dimensions. One fills the table with predictions from neural networks. The other fits a simple model to the existing simulations for that architecture. Both searches use the difference between the predicted and simulated properties to adjust the next design. We generated 200 targets from simulated designs after fixing the analysis plan. The two searches showed no detectable difference in how often they matched all three properties within 2% or in the mean error of their returned designs. The upper limit of the 95% confidence interval for the gain in overall success rate was 4.5 percentage points. Follow-up controls on these targets changed success from 83.5% to 36.0% with a degraded table and 96.5% with an oracle table. In separate tests, using the simulation result to adjust the next design increased success for both types of table. Results cover one library and a coarse 32^3 voxel simulator. Designers can use this comparison to judge what a learned predictor adds when the search, feedback and budget are the same.
Figures & tables
| System | Discrete choice | Architectures | Density | Blend weight |
|---|---|---|---|---|
| TPMS | 8 families, network or sheet | 16 | – | |
| Strut | 8 periodic strut topologies | 8 | – | |
| Blend | 28 family pairs, network or sheet | 56 | – | – |
| Set | Targets | Reachable at | Role |
|---|---|---|---|
| Development | 12 | Yes | Select the learned configuration |
| Confirmatory | 200 | Yes | Planned table comparison and separate follow-up controls |
| Test | 36 | Yes | Benchmark search against retrieval and standard optimisers |
| Fresh draw | 36 | Yes | Repeat the benchmark on another draw and check resolution |
| Interpolated and | 36 each | Unknown | Test performance when reachability is uncertain |
| Comparison | Change assessed | Main evidence |
|---|---|---|
| Learned / catalogue tables | Table construction under matched search and feedback | Planned comparison, confirmatory set |
| Corrected / uncorrected proposal | Residual use to choose the next design under matched decision rules | Test set and post-plan controls on the fresh and confirmatory sets |
| Degraded / catalogue / oracle tables | Response to larger changes in table quality | Post-plan controls, confirmatory set |
| Retrieval with and without pattern search | Charged local search within retrieved architectures | Test set |
| Catalogue-table search / retrieval with pattern search | Table-guided starts, refinement and allocation as a bundle | Test set |
| Broad / three-architecture search | Architecture allocation conditional on the table type | Test set and secondary confirmatory contrasts |
| Configuration | Runs met | Success rate | Design error | Calls to success | Simulator calls per target |
|---|---|---|---|---|---|
| Learned-table search | 84/108 | 0.778 (0.139) [0.676, 0.880] | 0.00639 (0.00065) [0.00471, 0.00848] | 13.19 (0.22) | 17.03 |
| Catalogue-table search | 78/108 | 0.722 (—) [0.611, 0.833] | 0.00800 (—) [0.00463, 0.01374] | 13.86 (—) | 17.86 |
| Bracketing search | 72/108 | 0.667 (—) | 0.01138 (—) | 12.36 (—) | 12.03 |
| Bayesian optimisation | 71/108 | 0.657 (0.083) | 0.00958 (0.00100) | 14.99 (2.56) | 14.65 |
| Nelder–Mead search | 69/108 | 0.639 (—) | 0.01627 (—) | 14.97 (—) | 14.61 |
| Retrieval with pattern search | 23/108 | 0.213 (0.028) [0.102, 0.333] | 0.03264 (0.00180) [0.02270, 0.04439] | 26.47 (0.39) | 29.56 |
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
| Runs met | Design error | Success rate | Calls to success | Simulator calls | |||
|---|---|---|---|---|---|---|---|
| Variant | all (108) | blend (54) | all | blend | blend | all | per target |
| Reference | |||||||
| Learned-table search | 84 | 32 | 0.00639 (0.00065) | 0.01046 (0.00098) | 0.593 (0.222) | 13.19 (0.22) | 17.03 |
| Learned component | |||||||
| Catalogue-table search | 78 | 24 | 0.00800 (—) | 0.01391 (—) | 0.444 (—) | 13.86 (—) | 17.86 |
| Categorical-encoding search | 79 | 25 | 0.01053 (0.00168) | 0.01904 (0.00350) | 0.463 (0.333) | 14.22 (2.75) | 17.48 |
| TPMS targets (27 runs) | Strut targets (27 runs) | |||||
|---|---|---|---|---|---|---|
| Variant | Runs met | Design error | Calls to success | Runs met | Design error | Calls to success |
| Reference | ||||||
| Learned-table search | 25 | 0.00463 | 7.37 | 27 | 2.96 | |
| Learned component | ||||||
| Catalogue-table search | 27 | 0.00419 | 4.67 | 27 | 4.33 | |
| Categorical-encoding search | 27 | 0.00405 | 7.56 | 27 | 2.19 | |
| Contrast ( ) | Difference | 95% interval | Targets better / worse / tied | Sign / Wilcoxon |
|---|---|---|---|---|
| Learned-table search retrieval with pattern search | 0.02625 | [ 0.03751, 0.01697] | 32 / 0 / 4 | 0.001 / 0.001 |
| All targets have runs met 84 versus 23, with 61 versus 0 met by one variant only. Seed ranges are 0.00065 and 0.00180. The difference exceeds both seed ranges. | ||||
| Learned-table search retrieval with pattern search | 0.03443 | [ 0.05509, 0.01820] | 18 / 0 / 0 | 0.001 / 0.001 |
| Blend targets have runs met 32 versus 2, with 30 versus 0 met by one variant only. Seed ranges are 0.00098 and 0.00357. The difference exceeds both seed ranges. | ||||
| Learned-table search catalogue-table search | 0.00161 | [ 0.00605, 0.00122] | 13 / 11 / 12 | 0.839 / 0.900 |
| All targets have runs met 84 versus 78, with 9 versus 3 met by one variant only. Seed ranges are 0.00065 and 0.00000. The difference exceeds both seed ranges. | ||||
| Contrast ( ) | Difference | 95% interval | Targets better / worse / tied | Sign |
|---|---|---|---|---|
| Learned-table search retrieval with pattern search | 0.565 | [ 0.417, 0.704] | 25 / 0 / 11 | 0.001 |
| All targets have runs met 84 versus 23, with 61 versus 0 met by one variant only. Seed ranges are 0.139 and 0.028. The difference exceeds both seed ranges. | ||||
| Learned-table search retrieval with pattern search | 0.556 | [ 0.370, 0.741] | 14 / 0 / 4 | 0.001 |
| Blend targets have runs met 32 versus 2, with 30 versus 0 met by one variant only. Seed ranges are 0.222 and 0.056. The difference exceeds both seed ranges. | ||||
| Learned-table search catalogue-table search | 0.056 | [ 0.019, 0.139] | 6 / 2 / 28 | 0.289 |
| All targets have runs met 84 versus 78, with 9 versus 3 met by one variant only. Seed ranges are 0.139 and 0.000. The difference does not exceed both seed ranges. | ||||
| Variant | Runs met all (36) | Runs met blend (18) | Design error all | Design error blend |
|---|---|---|---|---|
| Learned-table search | 27, 26, 31 | 10, 9, 13 | 0.00645, 0.00669, 0.00604 | 0.01004, 0.01103, 0.01033 |
| Catalogue-table search | 26, 26, 26 | 8, 8, 8 | 0.00800, 0.00800, 0.00800 | 0.01391, 0.01391, 0.01391 |
| Categorical-encoding search | 24, 25, 30 | 6, 7, 12 | 0.00948, 0.01116, 0.01095 | 0.01681, 0.02031, 0.01998 |
| Uncorrected-proposal control | 22, 22, 21 | 6, 6, 5 | 0.01342, 0.01427, 0.01536 | 0.02283, 0.02440, 0.02649 |
| Three-architecture catalogue search | 28, 28, 28 | 10, 10, 10 | 0.01010, 0.01010, 0.01010 | 0.01779, 0.01779, 0.01779 |
| First batch of 6 architectures | 28, 27, 32 | 11, 10, 14 | 0.00698, 0.00672, 0.00505 | 0.01104, 0.01087, 0.00794 |
| All targets (108 runs) | Blend targets (54 runs) | |||||
|---|---|---|---|---|---|---|
| Variant | 2 % | 3 % | 5 % | 2 % | 3 % | 5 % |
| Learned-table search | 84 | 94 | 100 | 32 | 42 | 47 |
| Catalogue-table search | 78 | 99 | 105 | 24 | 45 | 51 |
| Categorical-encoding search | 79 | 91 | 98 | 25 | 37 | 44 |
| Uncorrected-proposal control | 65 | 75 | 87 | 17 | 27 | 36 |
| Three-architecture catalogue search | 84 | 96 | 99 | 30 | 42 | 45 |
| Variant | Targets met (TPMS / strut / blend) | Design error all | Design error blend | Calls to success | Simulator calls per target |
|---|---|---|---|---|---|
| Reference | |||||
| Learned-table search | 10/12 (3 / 3 / 4) | 0.00607 | 0.00986 | 11.17 | 15.17 |
| Learned component | |||||
| Catalogue-table search | 11/12 (3 / 3 / 5) | 0.00944 | 0.01649 | 8.17 | 13.67 |
| Categorical-encoding search | 11/12 (3 / 3 / 5) | 0.01149 | 0.01996 | 12.58 | 17.00 |
| Role of the learned tables | |||||
| Recall of the true architecture | Held-out RMSE ( ) | |||||
|---|---|---|---|---|---|---|
| Model | Rank 3 | Rank 10 | Rank 20 | |||
| All designs (358) | ||||||
| Learned tables | 0.301 (0.031) | 0.519 (0.011) | 0.703 (0.022) | 0.144 | 0.136 | 0.093 |
| Categorical-encoding tables | 0.334 (0.020) | 0.550 (0.031) | 0.661 (0.025) | 0.143 | 0.138 | 0.097 |
| Catalogue-fitted tables | 0.388 (—) | 0.486 (—) | 0.542 (—) | 0.316 | 0.308 | 0.211 |
| Nearest catalogue design | 0.034 (—) | 0.137 (—) | 0.268 (—) | – | – | – |
| Configuration | Runs met | Success rate | Design error | Calls to success | Simulator calls per target |
|---|---|---|---|---|---|
| TPMS targets | |||||
| Learned-table search | 25/27 | 0.926 (0.111) [0.778, 1.000] | 0.00463 (0.00220) [0.00237, 0.00787] | 7.37 (0.11) | 12.41 |
| Catalogue-table search | 27/27 | 1.000 (—) [1.000, 1.000] | 0.00419 (—) [0.00277, 0.00561] | 4.67 (—) | 11.22 |
| Bracketing search | 24/27 | 0.889 (—) | 0.00385 (—) | 5.78 (—) | 5.67 |
| Bayesian optimisation | 23/27 | 0.852 (0.111) | 0.00508 (0.00162) | 8.81 (5.67) | 8.67 |
| Nelder–Mead search | 24/27 | 0.889 (—) | 0.00642 (—) | 7.89 (—) | 7.78 |
| Setting | Value | Status |
|---|---|---|
| First-batch architectures | 10 (3, 6, 15 in analysis) | Varied |
| Networks in ensemble | 3 (1, 9 in analysis) | Varied |
| Maximum from one system in first batch | 6 | Fixed |
| Later-batch allocation | 3 refinements and 2 new proposals | Fixed |
| Initial window half-width | One quarter of each parameter range | Fixed |
| Window contraction | Half-width halved per additional valid observation | Fixed |
| Configuration | Met | Met by system | Design error | Success rate (range) | Calls to success | Simulator calls | Met at 3% / 5% |
|---|---|---|---|---|---|---|---|
| Fresh reachable draw | |||||||
| Learned-table search | 94/108 | 24 / 27 / 43 | 0.01230 | 0.870 (0.056) | 10.63 | 14.76 | 99 / 103 |
| Catalogue-table search | 87/108 | 24 / 27 / 36 | 0.01326 | 0.806 (—) | 11.58 | 16.06 | 93 / 102 |
| Three-architecture catalogue search | 93/108 | 21 / 27 / 45 | 0.01389 | 0.861 (—) | 8.67 | 9.56 | 93 / 99 |
| Bracketing search | 87/108 | 24 / 27 / 36 | 0.01121 | 0.806 (—) | 8.83 | 8.64 | 93 / 96 |
| Bayesian optimisation | 84/108 | 23 / 27 / 34 | 0.01337 | 0.778 (0.083) | 11.52 | 11.30 | 94 / 102 |
| System (targets) | Success rate | Design error | Calls to success |
|---|---|---|---|
| Test set. Learned-table search minus Catalogue-table search | |||
| All (36) | +0.056 [-0.019, 0.139] | -0.00161 [-0.00605, 0.00122] | -0.68 [-2.95, 1.44] |
| TPMS (9) | -0.074 [-0.222, 0.000] | +0.00044 [-0.00237, 0.00355] | +2.70 [-1.56, 7.41] |
| Strut (9) | +0.000 [0.000, 0.000] — | +0.00000 [0.00000, 0.00000] — | -1.37 [-5.37, 1.48] |
| Blend (18) | +0.148 [0.019, 0.296] | -0.00344 [-0.01212, 0.00183] | -2.02 [-5.50, 1.15] |
| Test set. Learned-table search minus Bracketing search | |||
| System | Met / runs | Success [95% interval] | Design error | Calls to success | Simulator calls |
|---|---|---|---|---|---|
| Learned-table search | |||||
| All | 1687/2000 | 0.844 [0.804, 0.881] | 0.00669 | 10.82 | 15.27 |
| TPMS | 494/500 | 0.988 [0.964, 1.000] | 0.00343 | 5.64 | 11.33 |
| Strut | 490/500 | 0.980 [0.942, 1.000] | 0.00063 | 3.27 | 10.85 |
| Blend | 703/1000 | 0.703 [0.626, 0.776] | 0.01135 | 17.18 | 19.46 |
| Catalogue-table search | |||||
| System (targets) | Success rate | Design error | Calls to success |
|---|---|---|---|
| Learned-table search minus Catalogue-table search | |||
| All (200) | +0.008 [-0.027, 0.045] | +0.00002 [-0.00104, 0.00099] | +0.21 [-0.65, 1.07] |
| TPMS (50) | +0.008 [-0.006, 0.030] | -0.00032 [-0.00110, 0.00045] | +0.68 [-0.37, 1.71] |
| Strut (50) | -0.020 [-0.058, 0.000] | +0.00022 [-0.00041, 0.00104] | +0.13 [-1.12, 1.36] |
| Blend (100) | +0.023 [-0.044, 0.093] | +0.00008 [-0.00195, 0.00195] | +0.01 [-1.49, 1.51] |
| Learned-table search minus Three-architecture catalogue search | |||
| Tables, targets | Runs met | Success difference | 95% interval | Targets better / worse ( ) | Error difference | 95% interval |
|---|---|---|---|---|---|---|
| Test set (36 targets) | ||||||
| Learned tables, all | 84/108 vs 65/108 | 0.176 | [ 0.093, 0.278] | 11 / 0 ( 0.001) | 0.00796 | [ 0.01338, 0.00355] |
| Learned tables, blend | 32/54 vs 17/54 | 0.278 | [ 0.130, 0.444] | 9 / 0 (0.004) | 0.01411 | [ 0.02447, 0.00557] |
| Catalogue tables, all | 78/108 vs 57/108 | 0.194 | [ 0.083, 0.333] | 7 / 0 (0.016) | 0.01885 | [ 0.03589, 0.00593] |
| Catalogue tables, blend | 24/54 vs 12/54 | 0.222 | [ 0.056, 0.444] | 4 / 0 (0.125) | 0.03436 | [ 0.06829, 0.00868] |
| Fresh draw (36 targets) | ||||||
| System | Met / runs | Success [95% interval] | Design error | Calls to success | Simulator calls |
|---|---|---|---|---|---|
| Degraded table | |||||
| All | 72/200 | 0.360 [0.295, 0.420] | 0.02054 | 25.98 | 26.32 |
| TPMS | 27/50 | 0.540 [0.400, 0.680] | 0.01254 | 23.40 | 24.32 |
| Strut | 26/50 | 0.520 [0.380, 0.660] | 0.01668 | 23.50 | 24.70 |
| Blend | 19/100 | 0.190 [0.120, 0.270] | 0.02647 | 28.52 | 28.13 |
| Catalogue tables, full catalogue | |||||
| Targets | Success difference | 95% interval | Error difference | 95% interval | |
|---|---|---|---|---|---|
| Degraded minus catalogue tables | |||||
| All | 200 | ||||
| Blend | 100 | ||||
| Oracle minus catalogue tables | |||||
| All | 200 | ||||
| Blend | 100 | ||||
Inverse Design
Inverse design maps desired properties or behaviors to materials, geometries, molecules, and systems that can realize them.
See all 118 papers