Jointly Optimizing Memory, Computing and Offloading for Energy-Efficient DNN Inference
Organizations: Beijing National Research Center for Information Science and Technology, Department of Electronic Engineering, Tsinghua University, Beijing 100084, China · School of Electronics and Internet of Things, Chongqing Polytechnic University of Electronic Technology, Chongqing 401331, China
Abstract
In edge intelligence systems, computation offloading and dynamic voltage and frequency scaling (DVFS) are widely used to improve energy efficiency. However, as applications like deep neural network (DNN) inference evolve in complexity, their demands for memory resources have progressively increased, establishing memory as a critical bottleneck that impacts both latency and energy consumption. In this paper, we propose a joint optimization scheme integrating memory, computing and offloading for energy-efficient DNN inference. We formulate the energy minimization problem under deadline constraints as a mixed-integer nonlinear programming (MINLP) problem, where the main challenge arises from the coupling between offloading and resource optimization. To address this issue, we decompose the original problem into two subproblems under a fixed offloading policy. The local subproblem is transformed into an equivalent geometric programming (GP) problem to derive the optimal memory frequency and computing frequency, while the edge subproblem is solved via convex optimization. An efficient coordinate descent (CD) algorithm is further developed to jointly optimize all variables. Simulation results demonstrate that our proposed algorithm achieves near-optimal performance with the gap from the optimal policy being at most 9.3%, and reduces the energy consumption of devices by up to 16.1% compared with benchmarks without memory frequency optimization.
Figures & tables
| Symbol | Description | Symbol | Description |
|---|---|---|---|
| Index of the th mobile device | Memory energy coefficient of mobile device | ||
| Set of mobile devices | Computing energy coefficient of mobile device | ||
| Memory frequency of mobile device | Minimum memory frequency of mobile device | ||
| Computing frequency of mobile device | Maximum memory frequency of mobile device | ||
| Local inference time of mobile device | Minimum computing frequency of mobile device | ||
| Local energy consumption of mobile device | Maximum computing frequency of mobile device |
| Parameters | Jetson TX1 | Jetson Orin Nano | Jetson Orin NX | ||||
| ResNet152 | VGG19 | LSTM | ResNet152 | ViT-B/16 | GNN | Swin-B | |
| 0.039 | 0.110 | 0.051 | 0.012 | 0.012 | 0.060 | 0.020 | |
| 1.552 | 1.296 | 1.193 | 1.426 | 1.638 | 1.423 | 1.445 | |
| 0.211 | 0.099 | 0.029 | 0.112 | 0.043 | 0.081 | 0.131 | |
| 0.358 | 0.629 | 0.370 | 0.036 | 0.241 | 0.700 | 0.159 | |
| 0.9917 | 0.9961 | 0.9989 | 0.9993 | 0.9996 | 0.9925 | 0.9791 | |
| DNN Model | Mobile Device | ||||||||||
| ResNet152 | Jetson TX1 | [0.1, 1.6] GHz | [0.1, 1.0] GHz | 0.039 | 1.552 | 0.211 | 0.358 | 0.152 | 1.115 | 2.097 | 12 ms |
| VGG19 | 0.110 | 1.296 | 0.099 | 0.629 | 4 ms | ||||||
| LSTM | 0.051 | 1.193 | 0.029 | 0.370 | 1 ms | ||||||
| ResNet152 | Jetson Orin Nano | [0.2, 2.1] GHz | [0.3, 0.63] GHz | 0.012 | 1.426 | 0.112 | 0.036 | 0.033 | 1.268 | 1.936 | 12 ms |
| ViT-B/16 | 0.012 | 1.638 | 0.043 | 0.241 | 6 ms | ||||||
| GNN | Jetson Orin NX | [0.2, 3.2] GHz | [0.3, 1.2] GHz | 0.060 | 1.423 | 0.081 | 0.700 | 0.043 | 0.698 | 4.250 | 5 ms |
| Policy | ( s, MHz) | ( , MHz) | ( , s) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 12 | 16 | 20 | 24 | 0.06 s | 0.08 s | 0.10 s | 0.12 s | 10 MHz | 15 MHz | 20 MHz | 25 MHz | |
| Computing-only | 0.1398 | 0.1519 | 0.1595 | 0.1635 | 0.1574 | 0.1474 | 0.1398 | 0.1379 | 0.1624 | 0.1511 | 0.1398 | 0.1290 |
| Memory-only | 0.1588 | 0.1754 | 0.1845 | 0.1892 | 0.1708 | 0.1684 | 0.1588 | 0.1544 | 0.1867 | 0.1726 | 0.1588 | 0.1433 |
| Discrete-points | 0.1398 | 0.1520 | 0.1595 | 0.1636 | 0.1575 | 0.1474 | 0.1398 | 0.1380 | 0.1625 | 0.1512 | 0.1398 | 0.1292 |
| Proposed (ours) | 0.1395 | 0.1516 | 0.1591 | 0.1628 | 0.1574 | 0.1446 | 0.1395 | 0.1377 | 0.1619 | 0.1508 | 0.1395 | 0.1284 |
| Policy | ( s, MHz) | ( , MHz) | ( , s) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 12 | 16 | 20 | 24 | 0.09 s | 0.10 s | 0.11 s | 0.12 s | 10 MHz | 15 MHz | 20 MHz | 25 MHz | |
| Computing-only | 0.3569 | 0.4046 | 0.4361 | 0.4524 | 0.3791 | 0.3569 | 0.3316 | 0.3040 | 0.4479 | 0.3981 | 0.3569 | 0.2938 |
| Memory-only | 0.3380 | 0.3824 | 0.4117 | 0.4268 | 0.3584 | 0.3380 | 0.3149 | 0.2896 | 0.4226 | 0.3763 | 0.3380 | 0.2795 |
| Discrete-points | 0.3394 | 0.3840 | 0.4135 | 0.4287 | 0.3791 | 0.3394 | 0.3161 | 0.2907 | 0.4244 | 0.3779 | 0.3394 | 0.2805 |
| Proposed (ours) | 0.3380 | 0.3824 | 0.4117 | 0.4268 | 0.3584 | 0.3380 | 0.3149 | 0.2896 | 0.4226 | 0.3763 | 0.3380 | 0.2794 |