Quantum mechanical framework for quantization-based optimization: from Gradient flow to Schroedinger equation
Organizations: On-Device AIM Research Section AIC Research Lab, ETRI Daejeon, Rep. Korea
Abstract
This work presents a quantum mechanical framework for analyzing quantization-based optimization algorithms. The sampling process of the quantization-based search is modeled as a gradient-flow dissipative system, leading to a Hamilton-Jacobi-Bellman (HJB) representation. Through a suitable transformation of the objective function, this formulation yields the Schroedinger equation, which reveals that quantum tunneling enables escape from local minima and guarantees access to the global optimum. By establishing the connection to the Fokker-Planck equation, the framework provides a thermodynamic interpretation of global convergence. Such an analysis between the thermodynamic and the quantum dynamic methodology unifies combinatorial and continuous optimization, and extends naturally to machine learning tasks such as image classification. Numerical experiments demonstrate that quantization-based optimization consistently outperforms conventional algorithms across both combinatorial problems and nonconvex continuous functions.
Figures & tables
| Cost of Solution Path | Sample Standard Deviation | Improvement Ratio | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Cities | QTZ* | SA | QIA | QTZ* | SA | QIA | QTZ to SA | QTZ to QIA | QTZ to NN | NN |
| 100 | 1691.76 | 1732.16 | 1721.07 | 24.34 | 38.93 | 42.77 | 2.33 | 3.16 | 21.65 | 2159.27 |
| 125 | 1920.42 | 2013.31 | 2054.59 | 37.71 | 57.60 | 37.96 | 4.61 | 6.53 | 16.43 | 2297.87 |
| 150 | 2013.61 | 2218.52 | 2208.50 | 49.44 | 56.93 | 48.41 | 9.24 | 8.82 | 19.38 | 2497.65 |
| 175 | 2176.76 | 2532.26 | 2617.09 | 30.80 | 58.19 | 61.21 | 9.54 | 16.83 | 8.56 | 2380.53 |
| 200 | 2366.72 | 2924.87 | 2988.78 | 28.24 | 90.36 | 62.62 | 18.72 | 20.45 | 14.16 | 2769.73 |
| Iterations | Improvement Ratio | Solution vs Exact Minimum Ration | ||||||
|---|---|---|---|---|---|---|---|---|
| Function | QTZ | SA | QIA | SA/QTZ | QIA/QTZ | QTZ | SA | QIA |
| Xin-She Yang N4 | 3144 | 6420 | 17* | 2.04 | 54.57% | 54.57% | 35.22% | |
| Salomon | 1727 | 1312 | 7092 | 0.76 | 4.11 | 100.00% | 99.99% | 99.99% |
| Drop-Wave | 254 | 907 | 3311 | 3.57 | 13.04 | 100.00% | 99.99% | 99.99% |
| Shaffer N2 | 2073 | 7609 | 9657 | 3.67 | 4.66 | 100.00% | 99.99% | 99.99% |
| Data Set | FashionMNIST | CIFAR10 | CIFAR100 | STL10 |
|---|---|---|---|---|
| Model | CNN 3 Layers | ResNet-50 (56 Layer Blocks) | ||
| QSLGD | 89.29 | 73.8 | 37.77 | 50.68 |
| SGD | 91.47 | 63.31 | 25.90 | 46.92 |
| ASGD | 91.42 | 63.46 | 26.43 | 47.90 |
| QSLD | 91.59 | 85.09 | 49.60 | 58.04 |
| ADAM | 87.12 | 82.08 | 46.32 | 57.32 |
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
| PC Name | OS | GPU | CPU |
|---|---|---|---|
| PC-1 | Linux Ubuntu 24.0 | NVIDIA GeForce GTX 1080Ti | Intel i9 7900 |
| PC-2 | Windows 11 | NVIDIA GeForce RTX 3050 | Intel i9 7900 |
| PC-3 | Windows 11 | NVIDIA GeForce GTXTi | Intel i7 6700 |
| Criterion | Narrow band | Wide band | ||||
|---|---|---|---|---|---|---|
| SA | QIA | QTZ | SA | QIA | QTZ | |
| Average minimum cost | 0.0031 | 0.0031 | 0.0036 | 0.034 | 0.216 | 0.034 |
| Improvement ratio to the initial setting | 99.93 | 99.93 | 99.92 | 97.75 | 85.73 | 97.75 |
| Function | QTZ | SA | QIA |
|---|---|---|---|
| Xin-She Yang N4 | 3144 | 6420 | 17* |
| Salomon | 1727 | 1312 | 7092 |
| Drop-Wave | 254 | 907 | 3311 |
| Shaffer N2 | 2073 | 7609 | 9657 |
| Benchmark Function | Equation | Difficulty |
|---|---|---|
| Ackley | 48.25 | |
| Whitley | 4.92 | |
| Rosenbrock 2D | 44.17 | |
| Rosenbrock 100D | None | |
| EggHolder | 18.92 | |
| Xin-She Yang N.4 | 26.33 |
| Benchmark Function | Algorithm | Conventional Algorithm | QTZ Algorithm | Imp. ratio of Steps | Imp. ratio of Succ. | ||
| Final step | Success | Final step | Success | ||||
| Ackley | Gradient Descent | 8 | 43.0 | 3 | 49.0 | 62.50 | 13.95 |
| Conjugate Gradient | 16 | 35.0 | 4 | 32.0 | 75.00 | 8.57 | |
| Quasi Newton(BFGS) | 23 | 34.0 | 6 | 23.0 | 73.91 | 32.35 | |
| Whitley | Gradient Descent | 13 | 54.0 | 12 | 54.0 | 7.69 | 0.00 |
| Conjugate Gradient | 9 | 53.0 | 7 | 53.0 | 22.22 | 0.00 | |
| Benchmark Function | Input Domain | Optimal Point |
|---|---|---|
| Ackley | ||
| Whitley | ||
| Rosenbrock 2D | ||
| Rosenbrock 100D | ||
| EggHolder | ||
| Xin-She Yang N.4 |
| Optimizer | Learning Equation | Auxiliary Equation |
|---|---|---|
| QSLD | ||
| QSLGD | ||
| SGD | ||
| ASGD | ||
| ADAM | ||
| NADAM |
| Learning Algorithm | Hyperparameters |
|---|---|
| QSLD | |
| QSLGD | Equivalent to QSLD |
| SGD | |
| ASGD | Not Defined |
| ADAM | |
| NADAM | Equivalent to ADAM |
| Data Set | FashionMNIST | CIFAR10 | CIFAR100 | STL10 |
|---|---|---|---|---|
| Model | CNN 3 Layers | ResNet-50 (56 Layer Blocks) | ||
| QSLGD | 89.29 | 73.8 | 37.77 | 50.68 |
| SGD | 91.47 | 63.31 | 25.90 | 46.92 |
| ASGD | 91.42 | 63.46 | 26.43 | 47.90 |
| QSLD | 91.59 | 85.09 | 49.60 | 58.04 |
| ADAM | 87.12 | 82.08 | 46.32 | 57.32 |
| Data Set | FashionMNIST | CIFAR10 | CIFAR100 | STL10 |
|---|---|---|---|---|
| Model | CNN 3 Layers | ResNet-50 (56 Layer Blocks) | ||
| QSLGD | 0.085426 | 0.009253 | 0.030104 | 0.007205 |
| SGD | 0.132747 | 0.001042 | 0.005478 | 2.214468 |
| ASGD | 0.130992 | 0.001166 | 0.004981 | 2.001648 |
| QSLD | 0.059952 | 0.011456 | 0.037855 | 0.005939 |
| ADAM | 0.176379 | 0.012421 | 0.038741 | 0.53936 |