How the Classical Baseline Shapes Machine-Learning Speed Claims in Topology Optimization
Organizations: Santa Clara University Santa Clara, CA 95053, USA
Abstract
Topology optimization places material in a design domain to optimize a structure's performance, such as its stiffness or heat conduction. Machine-learning methods often report large speed-ups over classical, non-learned optimization, but a slow classical code or a lower-quality learned design can exaggerate them. We re-ran released code for eight methods on one workstation and studied two in depth: Optimize Any Topology (OAT), a diffusion model that generates designs, and a graph-based neural optimizer. Replacing the classical code with an efficient implementation of the same algorithm cut these two methods' speed-ups by up to 20-fold. We then timed the classical code until its design was at least as stiff as the learned one; for OAT, we picked this point after the runs, which gives the classical code its best case. Published timings imply a 134x speed-up for OAT on a 256 x 256 grid; our matched single-design comparisons gave 1.5x to 3.5x. At matched stiffness, the graph-based optimizer was slower than the classical code in all six two-dimensional problems. Of the six other methods, two kept their advantage, one shrank, one reversed, and two were not comparable. In an exploratory audit, language models judged that 8 of 28 speed comparisons in randomly sampled, openly accessible articles lacked evidence of equal design quality; our own review counted 11, so the count depends on who judges. We provide a comparison procedure and a minimum report: classical code, hardware, design quality, what each timing includes, and data-generation and training costs.
Figures & tables
| Study | Domain | Design | Main finding or recommendation |
|---|---|---|---|
| Woldseth et al. [13] | Neural topology optimization | Critical review | Asks for comparisons at equal volume and quality, and for training cost in any break-even claim |
| Sigmund [14] | Topology optimization | Recommendations | Recommends fair evaluation and benchmarking practice |
| Shin et al. [11] | Topology optimization | Review | Surveys machine-learning-based topology optimization |
| Regenwetter et al. [12] | Engineering design | Review | Surveys deep generative models in design, including structural optimization |
| Herrmann et al. [15] | Topology optimization | Controlled study | Neural-network discretizations can find better local optima under certain conditions |
| Sanu et al. [16] | Topology optimization | Controlled study | Neural networks add non-convexity, which can delay convergence in convex problems |
| Part | What is tested | Quality criterion | Ladder levels | Results |
|---|---|---|---|---|
| OAT re-test (Sect. 3.3.1 ) | A diffusion model against our efficient classical code on five test problems, fixed in advance, at and ; classical runs of 5 to 150 iterations | Compliance error from OAT’s evaluator; a volume condition, added after the runs, also limits material | , , , | Sects. 4.1 and 4.2 |
| HGTO re-test (Sect. 3.3.2 ) | A graph-based method against the authors’ classical code and our GPU port of it on six linear two-dimensional problems | Time to convergence, and time to reach HGTO’s final compliance from its independent evaluator | , , , | Sects. 4.1 and 4.3 |
| Census (Sect. 3.3.3 ) | The six sampled speed claims whose public code ran, each in the article’s own comparison | Each article’s own quality measure (Table 3 ) | L0 (code runs), L1 (as ), L2 (as – ) | Sect. 4.4 |
| Article audit (Sects. 3.4 – 3.6 ) | 57 sampled articles, 28 with speed claims, coded against rules R1–R4 by two language-model passes with model adjudication, then reviewed by the authors | Rule R1: matched quality or time | – | Sects. 4.6 – 4.8 |
| Method | Reported | Same-machine outcome | Quality criterion |
| In-depth re-tests | |||
| OAT diffusion generator, [ 8 ] | 134× (implied by published timings) | 78.6 against OAT’s released SIMP code; 18.6× with an efficient GPU solve; at matched quality, single samples: 1.72× under the pre-registered rule, 3.47× with the volume condition | Compliance error from OAT’s evaluator; the volume condition also limits material |
| HGTO graph-based reparameterization [ 9 ] | 4.2× and 9.6× (from the article’s reported times) | 4.9× and 10.8× against the authors’ SIMP–OC; the same algorithm with our GPU solve was faster on all six configurations at matched quality (HGTO took 2.0× to 6.5× as long) | Time to convergence; final compliance from HGTO’s independent evaluator |
| Halle et al. predictor [ 10 ] | 259 | Not compared: stopped at the pre-registered reconstruction check (57 of 100 within 1 %, 95 required) | Reconstruction check before timing |
| Census | |||
| Conductivity CNN [ 2 ] | 2,280–24,747 | 320–1,112 at L1; 3.6 at L2 with more accurate solve | Effective conductivity against the published values |
| Stratum | Rule | Fail | Fail or partial |
|---|---|---|---|
| Speed | R1 matched quality/time | 8/28 (29 %) | 22/28 (79 %) |
| Speed | R2 efficient baseline | 0/28 (0 %) | 11/28 (39 %) |
| Speed | R3 full cost | 7/25 (28 %) | 20/25 (80 %) |
| Speed | R4 verified output | 4/28 (14 %) | 20/28 (71 %) |
| Speed | R1 or R2 | 8/28 (29 %) | 24/28 (86 %) |
| Speed | Any rule | 13/28 (46 %) | 27/28 (96 %) |
| Item | Report |
|---|---|
| Classical method | Formulation, update, filter, linear solver, and stopping rule |
| Hardware | CPU and GPU model, threads, and precision for each method |
| Timing | Setup, post-processing, warm-up, repeats, and statistic |
| Quality | Common finite-element objective, constraints, failures, and compared point |
| Offline cost | Data generation, training, hardware, and break-even use count |
| Artifacts | Code, weights, timing harness, and pinned environment |
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| Article | Method | Physics | Dim. | Factor | R1 | R2 | R3 | R4 |
|---|---|---|---|---|---|---|---|---|
| Adam et al. [2] | Convolutional network | Thermal | 2D | 2,280 | Pass | Partial | Pass | Pass |
| Al Ali [56] | Language-model agent | Compliance | 2D, 3D | 3.33 | Fail | Partial | – | Partial |
| Banga et al. [39] | Convolutional network | Compliance | 3D | 1.67 | Partial | Pass | Fail | Fail |
| Black and Najafi [3] | Surrogate | Stress | 2D | 10,000 | Partial | Pass | Partial | Pass |
| Bujny et al. [46] | Other | Compliance | 2D, 3D | 12 | Partial | Pass | Partial | Partial |
| Cang et al. [29] | Other | Compliance | 2D | 10,000 | Fail | Partial | Partial | Pass |
| Article | Method | Physics | Dim. | R1 | R4 |
|---|---|---|---|---|---|
| Abueidda et al. [57] | Convolutional network | Compliance | 2D | Partial | Fail |
| Ait Ouchaoui et al. [58] | GAN | Compliance | 2D | Partial | Partial |
| Behzadi and Ilies [59] | GAN | Compliance | 2D | Partial | Partial |
| Bhagat et al. [60] | Transformer | Multiphysics | 2D, 3D | Partial | Fail |
| Chadha and Kumar [61] | Convolutional network | Compliance | 2D | Partial | Partial |
| Chandrasekhar and Suresh [62] | Neural reparameterization | Compliance | 2D | Partial | Partial |
| Rule | Fail | Partial |
|---|---|---|
| R1: matched quality or time | Inference against a converged classical run with worse learned output and no match; speed claim without quality comparison; or post-processing counted in quality but not in time | Loose matching or unstated stopping |
| R2: efficient baseline | Our re-implementation at least three times faster on the same hardware class and size; an abstract reports a weaker baseline than the results; unjustified GPU–CPU comparison at degrees of freedom; excessive iterations; or classical iteration at least ten times slower than a strong baseline | Suspected inefficiency without a comparable measure |
| R3: full cost | Neither data nor training cost given | Training cost without data-generation cost, or no amortization or per-query scope |
| R4: verified output | Only image-similarity metrics or surrogate predictions, or examples chosen without a stated rule | Finite-element verification without violations or failure rates |
| Characteristic | Speed | Quality | All |
|---|---|---|---|
| Convolutional network | 13 | 4 | 17 |
| Neural reparameterization | 1 | 7 | 8 |
| Surrogate | 5 | 2 | 7 |
| Compliance physics | 18 | 20 | 38 |
| Architected-material homogenization | 6 | 2 | 8 |
| Two-dimensional | 17 | 19 | 36 |
| Item | Raw agreement | Claude | OpenAI | OpenAI vs Claude | |
|---|---|---|---|---|---|
| Inclusion | 110 | 88 % | 0.62 (0.42–0.80) | 0.61 (0.39–0.79) | 0.48–0.56 |
| Stratum | 110 | 70 % | 0.53 (0.40–0.65) | 0.59 (0.47–0.71) | 0.46–0.60 |
| R1 | 82 | 55 % | 0.35 (0.21–0.49) | 0.23 (0.11–0.36) | 0.16–0.25 |
| R2 | 82 | 56 % | 0.28 (0.12–0.43) | 0.26 (0.11–0.41) | 0.23–0.39 |
| R3 | 82 | 83 % | 0.66 (0.49–0.80) | 0.47 (0.30–0.63) | 0.42–0.56 |
| R4 | 82 | 70 % | 0.56 (0.42–0.70) | 0.71 (0.58–0.83) | 0.44–0.56 |
Topology Optimization
Topology optimization finds material layouts that meet performance and manufacturing constraints across engineering systems.
See all 34 papers