What a Reporting Convention Hides: A Matched-Budget Audit of Quantum Natural Gradient with an Exactly Computed Metric
Organizations: Stony Brook University · Westlake University
Abstract
Several published comparisons of variational quantum optimizers time only runs that reach a target loss, or read the verdict at a single target. Either convention could decide whether an optimizer's costlier steps pay off. We measure how much each convention changes verdicts among Adam, simultaneous perturbation stochastic approximation (SPSA) and quantum natural gradient (QNG), on initializations held out from the selection of settings. We compute the exact metric that preconditions QNG, price every step in circuit evaluations and give every method the same budget. In a median pooled over circuit widths, cost families and a sweep of settings with common misses, SPSA needs more than twice Adam's evaluations to reach a loose target. Dropping the censored runs that miss the target hides this gap. On the global-cost family we hold fixed the settings selected for a strict target. QNG then usually reaches the loose target after Adam but the strict target first. QNG's strict-target lead disappears when the metric's simulator price, linear in the number of parameters, is replaced by an assumed hardware count quadratic in that number. We recommend charging every miss the budget and reporting verdicts across targets.
Figures & tables
| Choice | Comparison; held fixed | Reading 1 | Reading 2 | Verdict change absent when |
|---|---|---|---|---|
| Which runs enter (Sec. 4 ) | SPSA vs. Adam, ratio of median times pooled over eight configurations; original grid and initializations, loose target, SPSA’s shared perturbation sequence | Reached runs only 1.11 (0.84–1.40) | Misses charged the budget 2.24 (1.60–2.69) | Settings selected for the loose target (every Adam and SPSA run reaches it on both initialization sets, so the readings coincide) or one configuration taken alone (Reading 1 still shrinks the ratio but leaves SPSA slower in 7 of 8) |
| At which target (Sec. 5 ) | QNG vs. Adam, number of initializations with QNG first and median paired difference; global-cost family, 80 new initializations, strict-target settings, simulator price | Loose target, 12 of 80 (121.5–255) | Strict target, 77 of 80 ( to ) | Settings selected for the loose target, evaluated at the strict target (40–29–11, sign-test p-value 0.23; Adam first at 5 and 6 qubits, 2–14–4 and 1–12–7), or the metric charged evaluations per step (QNG first from 0 of 80 at the loose and 3 of 59 non-tied at the strict target) |
| Depth | 1 | 2 | 3 | 4 | 5 | 6 |
|---|---|---|---|---|---|---|
| true rank | 12 | 24 | 36 | 48 | 60 | 72 |
| 1.87 | 3.64 | 4.05 | 4.89 | 5.44 | 5.22 | |
| Effective rank | 8.9 | 8.0 | 9.2 | 9.5 | 9.6 | 10.5 |
| Alignment | 1.000 | 0.434 | 0.436 | 0.330 | 0.301 | 0.294 |
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
| Run set | Settings | Initializations | Targets | Used for |
|---|---|---|---|---|
| Original grid | Adam and SPSA ; QNG also | 10 original | six, from 0.5 to 0.01 | Section 4 ; best original setting; fixing the strict target |
| Extended grid | six from 0.03 to 10 for Adam and SPSA; 27 settings for QNG, from to 1 | 10 original | 0.20, 0.01 | The 48 selections (Table 4 ) |
| New initializations | every original and every selected setting | 20 new, held out from selection | 0.20, 0.01; six at the strict-target selection | Replication of Section 4 ; main comparison and out-of-sample tests of Section 5 |
| Five SPSA sequences | every SPSA setting in the rows above | 10 original; 20 new | 0.20, 0.01 | Per-initialization median of five sequences (Appendix E ) |
| Budgets and | strict-target selections; global-cost loose-target selections | 10 original and 20 new; 10 original | 0.20, 0.01 and final loss; final loss | Counts and final loss at longer budgets (Appendices J and L ) |
| Adam | QNG | SPSA | |||||||
| Family | |||||||||
| global-cost | 3 | 0.3 | 0.3 | 1 | 1 | 1 | 1 | ||
| 4 | 0.3 | 0.3 | 1 | 1 | 1 | 1 | |||
| 5 | 0.3 | 0.3 | 1 | 1 | 1 | 1 | |||
| 6 | 0.3 | 0.3 | 1 | 1 | 1 | 1 | |||
| Original initializations | New initializations | |||||||
|---|---|---|---|---|---|---|---|---|
| SPSA/Adam, | SPSA, | SPSA/Adam, | SPSA, | |||||
| Perturbation sequence | Conditional | Budget- capped | Reached of 160 | Conditional | Conditional | Budget- capped | Reached of 320 | Conditional |
| Shared sequence | 1.114 | 2.243 | 45 | 536 | 0.989 | 2.289 | 98 | 637 |
| Sequence 1 | 1.114 | 2.229 | 46 | 532 | 1.014 | 2.304 | 89 | 674 |
| Sequence 2 | 0.979 | 2.318 | 43 | 544 | 0.971 | 2.239 | 95 | 694 |
| Sequence 3 | 1.004 | 2.211 | 46 | 653 | 0.936 | 2.221 | 94 | 657 |
| Median | Median | ||||||
|---|---|---|---|---|---|---|---|
| Method | Runs | Reached | Conditional | Budget- capped | Reached | Conditional | Budget- capped |
| Adam | 160 | 160 | 280 | 280 | 113 | 864 | 1000 |
| QNG | 320 | 188 | 584 | 935 | 85 | 776 | 1080 ≥B |
| SPSA (shared sequence) | 160 | 109 | 312 | 628 | 45 | 536 | 1080 ≥B |
| , settings pooled | , best setting | , best setting | ||||||||||||||||||
| Family | Adam | QNG | SPSA | Adam | QNG | SPSA | Adam | QNG | SPSA | |||||||||||
| global-cost | 3 | 972 | 198 | 522 | .5 | 326 | 144 | 275 | 127 | 702 | 385 | 304 | ||||||||
| 4 | 1728 | 320 | 970 | 848 | 288 | 533 | .5 | 338 | 1504 | 776 | 1009 | |||||||||
| 5 | 2700 | 650 | 2633 | .5 ≥B | 2700 | =B | 450 | 1434 | .5 | 844 | 2400 | 1812 | 2700 | =B | ||||||
| 6 | 3888 | 1008 | 3888 | =B | 3888 | =B | 720 | 2821 | 3505 | 3672 | 3255 | 3888 | =B | |||||||
| local-cost | 3 | 648 | 84 | 222 | 98 | 60 | 129 | .5 | 28 | 312 | 370 | 302 | ||||||||
| Original initializations | New initializations | |||||||
|---|---|---|---|---|---|---|---|---|
| SPSA | SPSA/Adam | SPSA | SPSA/Adam | |||||
| Family | Reached of 20 | Conditional | Budget- capped | Reached of 40 | Conditional | Budget- capped | ||
| global-cost | 3 | 972 | 18 | 1.348 | 1.646 | 35 | 0.972 | 1.009 |
| 4 | 1728 | 15 | 1.269 | 2.650 | 27 | 1.057 | 2.659 | |
| 5 | 2700 | 9 | 1.148 | 4.154 =B | 23 | 1.333 | 3.070 | |
| 6 | 3888 | 6 | 2.245 | 3.857 =B | 15 | 2.366 | 4.500 =B | |
| (a) Each family pooled over | Evaluated at | Evaluated at | Final loss | |||||
| Selected at | Family | QNG search | Count | Count | Count | |||
| global-cost | 27 settings | 12–68–0 | 77–3–0 | 79–1–0 | ||||
| 6 at | 1–79–0 | 73–7–0 | 76–4–0 | |||||
| local-cost | 27 settings | 36–43–1 | 79–1–0 | 80–0–0 | ||||
| global-cost | 27 settings | 14–66–0 | 40–29–11 | 46–34–0 | ||||
| local-cost | 27 settings | 24–55–1 | 78–2–0 | 80–0–0 | ||||
| Target | 0.50 | 0.20 | 0.10 | 0.05 | 0.02 | 0.01 |
|---|---|---|---|---|---|---|
| Global-cost family | ||||||
| QNG first–Adam first–tied | 3–77–0 | 12–68–0 | 35–45–0 | 63–17–0 | 75–5–0 | 77–3–0 |
| Sign-test | ||||||
| Adam reached | 80 | 80 | 80 | 80 | 80 | 65 |
| QNG reached | 79 | 79 | 79 | 79 | 79 | 79 |
| Local-cost family | ||||||
| Budget-capped time to | Final loss | |||||||||||
| New initializations | Original initializations | Best original setting | New initializations | |||||||||
| (95% interval) | (95% interval) | (95% interval) | ||||||||||
| Global-cost family ( ) | ||||||||||||
| 3 | 19 | 171 | 161 | (91–197) | 0.13 | 104 | (97–179) | 0.06 | 64 | 158 | (126–207) | 0.14 |
| 4 | 33 | 528 | 223 | (170–255) | 0 | 236 | (179–249) | 0 | 123 | 224 | (182–281) | 0 |
| 5 | 51 | 1275 | 271 | (188–333) | 0 | 199 | (155–249) | 0 | 99 | 285 | (200–350) | 0 |
| (a) Strict-target settings | Count | Pooled ratio | ||||
| Comparison | ||||||
| New initializations (20 per configuration), global-cost family | ||||||
| QNG–Adam, first to | 12–68–0 | |||||
| QNG–Adam, first to | 77–3–0 | |||||
| QNG–Adam, lower final loss | 79–1 | 79–1 | 62–18 | |||
| SPSA–Adam, lower final loss | 46–34 (39–41) | 1–79 | 0–80 | |||