SparseDVFS: Sparse-Aware DVFS for Energy-Efficient Edge Inference

Ziyang Zhang; Zheshun Wu; Jie Liu; Luca Mottola

cs.LGMar 23, 2026

SparseDVFS: Sparse-Aware DVFS for Energy-Efficient Edge Inference

Authors: Ziyang Zhang, Zheshun Wu, Jie Liu, Luca Mottola

Organizations: Politecnico di Milano, Milan, Italy · Harbin Institute of Technology, Shenzhen, China

Abstract

Deploying deep neural networks (DNNs) on power-sensitive edge devices presents a formidable challenge. While Dynamic Voltage and Frequency Scaling (DVFS) is widely employed for energy optimization, traditional model-level scaling is often too coarse to capture intra-inference variations, whereas fine-grained operator-level scaling suffers from prohibitive performance degradation due to significant hardware switching latency. This paper presents SparseDVFS, a fine-grained, sparse-aware DVFS framework designed for energy-efficient edge inference. Our key insight is that operator sparsity is a primary metric for hardware frequency modulation. By distinguishing between compute-bound dense operators and memory-bound sparse operators, the system can apply specialized frequency triplets to maximize energy efficiency. To overcome switching overheads and component interference, SparseDVFS incorporates three key innovations: (1) an offline modeler that established a deterministic mapping between operator sparsity and optimal frequency triplets (CPU/GPU/EMC) via white-box timeline analysis; (2) a runtime graph partitioner that utilizes a greedy merging heuristic to aggregate operators into super-blocks, balancing scaling granularity and DVFS switching latency through a latency amortization constraint; and (3) a unified co-governor that employs a frequency unified scaling engine (FUSE) and a look-ahead instruction queue to eliminate antagonistic effects between independent controllers and hide hardware transition latencies. Extensive evaluations show that SparseDVFS achieves an average 78.17% energy efficiency gain over state-of-the-art solutions while maintaining a superior 14% cost-gain ratio.

Figures & tables

Figure 1. Comparison of DVFS governor granularities for edge DNN inference. (a) Model-level DVFS governor employs a single V/f setting. (b) Operator-level fine-grained DVFS governor. (c) Our sparsity-aware block-level DVFS governor. Comparison of DVFS governor granularities for edge DNN inference. (a) Model-level DVFS governor employs a single V/f setting. (b) Operator-level fine-grained DVFS governor. (c) Our sparsity-aware block-level DVFS governor.

Panel of Figure 2 — Figure 2. Roofline model across diverse DNN architectures. For CNN-based models (i.e., ResNet-18/101), dense operators (e.g., Conv2d, Linear) reside on the horizontal plateau (i.e., Compute-Bound), while activation and normalization layers fall into the sloped region (i.e., Memory-Bound). For Transformer-based models (i.e., ViT-B16/L16), dense operators (e.g., Conv2d) reside on the horizontal plateau (i.e., Compute-Bound), while linear and normalization layers fall into the sloped region (i.e., Memory-Bound). Roofline model across diverse DNN architectures. For CNN-based models (i.e., ResNet-18/101), dense operators (e.g., Conv2d, Linear) reside on the horizontal plateau (i.e., Compute-Bound), while activation and normalization layers fall into the sloped region (i.e., Memory-Bound). For Transformer-based models (i.e., ViT-B16/L16), dense operators (e.g., Conv2d) reside on the horizontal plateau (i.e., Compute-Bound), while linear and normalization layers fall into the sloped region (i.e., Memory-Bound).

Panel of Figure 3 — Figure 3. Dynamic sparsity distribution (CDF) for CNN and Transformer models on ImageNet-2012 validation dataset (about 50k images). A substantial portion of operators in ResNet and ViT exhibit high sparsity (>50%). Dynamic sparsity distribution (CDF) for CNN and Transformer models on ImageNet-2012 validation dataset (about 50k images). A substantial portion of operators in ResNet and ViT exhibit high sparsity (>50%).

Figure 4. Runtime traces of CPU and GPU frequencies under default DVFS governors for ResNet-18. Independent and often inverse fluctuations between frequencies create an antagonistic effect. Runtime traces of CPU and GPU frequencies under default DVFS governors for ResNet-18. Independent and often inverse fluctuations between frequencies create an antagonistic effect.

Figure 5. Comparison between DVFS switching overhead and end-to-end inference latency for ResNet-18. Comparison between DVFS switching overhead and end-to-end inference latency for ResNet-18.

Panel of Figure 6 — Figure 6. The impact of CPU and GPU clock frequencies on system performance for ResNet-18. The heatmaps illustrate the non-linear sensitivity of (a) inference latency and (b) power consumption to frequency scaling. The impact of CPU and GPU clock frequencies on system performance for ResNet-18. The heatmaps illustrate the non-linear sensitivity of (a) inference latency and (b) power consumption to frequency scaling.

Figure 7. Overview of the SparseDVFS framework. The system is architected in three synergistic stages: (1) the offline modeler establishes a deterministic mapping between operator characteristics and optimal frequency triplets through timeline-based performance analysis and thermal-aware power modeling. (2) the runtime graph partitioner performs greedy operator aggregation to construct super-blocks, ensuring that fine-grained DVFS scaling satisfies the latency amortization constraint. (3) the unified co-governor executes coordinated frequency scaling across heterogeneous components via the frequency unified scaling engine (FUSE) strategy, while masking DVFS switching latency through a pipelined look-ahead mechanism. Overview of the SparseDVFS framework. The system is architected in three synergistic stages: (1) the offline modeler establishes a deterministic mapping between operator characteristics and optimal frequency triplets through timeline-based performance analysis and thermal-aware power modeling. (2) the runtime graph partitioner performs greedy operator aggregation to construct super-blocks, ensuring that fine-grained DVFS scaling satisfies the latency amortization constraint. (3) the unified co-governor executes coordinated frequency scaling across heterogeneous components via the frequency unified scaling engine (FUSE) strategy, while masking DVFS switching latency through a pipelined look-ahead mechanism.

Predictor	Model	$\pm 10\%$ Accuracy	Model Size
FLOPs-based ( Kim et al., 2019 )	ResNet-18	18.4%	8B
	ResNet-101	12.8%
	ViT-B16	11.5%
	ViT-L16	8.2%
nn-Meter ( Zhang et al., 2021 )	ResNet-18	91.5%	~856MB
	ResNet-101	88.1%

Table 1.

\pm 10\%

latency prediction accuracy and model size of different predictors.

Panel of Figure 8 — Figure 8. An example of offline modeler for ResNet-18. (a) latency prediction under different CPU/GPU frequency combinations and (b) thermal-aware power prediction. An example of offline modeler for ResNet-18. (a) latency prediction under different CPU/GPU frequency combinations and (b) thermal-aware power prediction.

Figure 9. Impact of runtime graph partitioner on operator aggregation granularity and DVFS switching latency. We set aggregation factor $N=5$ . Impact of runtime graph partitioner on operator aggregation granularity and DVFS switching latency. We set aggregation factor $N=5$. — Figure 9. Impact of runtime graph partitioner on operator aggregation granularity and DVFS switching latency. We set aggregation factor $N=5$ . Impact of runtime graph partitioner on operator aggregation granularity and DVFS switching latency. We set aggregation factor $N=5$ .

Figure 10. Pipeline-based Look-Ahead DVFS. Pipeline-based Look-Ahead DVFS.

Model	Traditional DVFS	Pipeline Look-Ahead
ResNet-18	7.23ms	0.12ms
ResNet-101	10.81ms	1.45ms
ViT-B16	5.44ms	0.72ms
ViT-L16	7.92ms	1.08ms

Table 2. Comparison of DVFS Switching Latency between Traditional DVFS (serial execution) and Pipeline-based Look-Ahead. We set aggregation factor

N=5

Model	Op. Count	FLOPs (G)	Size (M)
ResNet-18	21	1.82	11.69
ResNet-101	105	7.87	44.55
ViT-B16	38	11.29	58.07
ViT-L16	74	39.86	203.36

Table 3. Detailed configuration of four DNN models.

Figure 11. Comparison of end-to-end inference latency across different DNN models. Comparison of end-to-end inference latency across different DNN models.

Figure 12. Energy and power profile comparison across different DNN models. Energy and power profile comparison across different DNN models.

Figure 13. Normalized energy efficiency gain relative to the Default DVFS baseline. Normalized energy efficiency gain relative to the Default DVFS baseline.

Figure 14. Comparison of Cost-Gain Ratio across different DVFS strategies. Comparison of Cost-Gain Ratio across different DVFS strategies.

Panel of Figure 15 — Figure 15. Ablation studies of each component of the unified co-governor on (a) DOTA-v1.0 and (b) VisDrone 2019 datasets, respectively. We use ViT-B16 as test load. Ablation studies of each component of the unified co-governor on (a) DOTA-v1.0 and (b) VisDrone 2019 datasets, respectively. We use ViT-B16 as test load.

Figure 16. Thermal stability and frame rate jitter analysis of NVIDIA Jetson Orin Nano under sustained load (ViT-B16). Thermal stability and frame rate jitter analysis under sustained load (ViT-B16).

Figure 17. Impact of the amortization factor

N

on super-block aggregation granularity. Increasing

N

dictates the trade-off between operator aggregation granularity and DVFS switching latency. Impact of the amortization factor

N

on super-block aggregation granularity. Increasing

N

dictates the trade-off between operator aggregation granularity and DVFS switching latency.

Panel of Figure 18 — Figure 18. Trade-off between inference latency and switching overhead relative to $N$ . Trade-off between inference latency and switching overhead relative to $N$ .

Panel of Figure 19 — Figure 19. Impact of the aggregation factor $N$ on system power and energy consumption. Impact of the aggregation factor $N$ on system power and energy consumption.

Efficient Neural Network Inference

Efficient neural network inference reduces the time, memory, and energy needed to run trained models across hardware and deployment settings.

See all 468 papers

Sustainable Edge Vision via Empirically Calibrated DVFS: Eliminating Thermal Throttling on Passively Cooled Hardware
Sep 4, 2026Aayush Marasini, Zhaoxian ZhouEnergy-Efficient ML Edge Inference
School of Computing Sciences and Computer Engineering University of Southern Mississippi, Hattiesburg, MS, USA
Joint Optimization of Memory and Computing Frequency for Energy-Efficient DNN Inference
Aug 14, 2026Yunchu Han, Zhaojun Nan, Sheng Zhou +1Efficient Neural Network Inference Energy-Efficient ML
Beijing National Research Center for Information Science and Technology Department of Electronic Engineering, Tsinghua University, Beijing 100084, China · School of Electronics and Internet of Things Chongqing Polytechnic University of Electronic Technology, Chongqing 401331, China
Energy-Efficient GPU DVFS for Fine-Tuning of SLMs on Resource-constrained Embedded Devices
Jul 7, 2026Jurn-Gyu Park, Sanzhar Zholdybayev, Aidar Amangeldi +1Fine-Tuning Energy-Efficient ML
School of Engineering and Digital Sciences, Nazarbayev University, 010000 Astana, Kazakhstan

Sep 4, 2026J/K move · Enter open · S save

Sustainable Edge Vision via Empirically Calibrated DVFS: Eliminating Thermal Throttling on Passively Cooled Hardware

Aayush Marasini, Zhaoxian Zhou

School of Computing Sciences and Computer Engineering University of Southern Mississippi, Hattiesburg, MS, USA

Energy-Efficient ML Edge Inference

Open paper AskPDF arXiv

Passive cooling eliminates the energy overhead and mechanical failure modes of fans, making it attractive for edge deployment, yet sustained Deep Neural Network (DNN) inference on passively cooled edge Systems-on-Chip (SoCs) is bottlenecked by thermal throttling. To address this, we propose an empirically calibrated, state-aware Dynamic Voltage and Frequency Scaling (DVFS) scheduler. Unlike heuristic-driven controllers, our methodology utilizes time-domain guards and absolute temperature bounds, with derivative triggers acting as safeguards against sharp thermal spikes. Evaluated on a passively cooled Raspberry Pi 5 running YOLOv8n, our scheduler eliminates all observed thermal throttling events during sustained 30-minute workloads. It outperforms a temperature-only reactive baseline by achieving a 6.8% higher frame rate (Cohen's d = 8.73) while consuming 1.9% less energy per frame. Furthermore, our optimized passive scheduling surpasses an actively cooled reference system in energy efficiency (Joules/frame), though active cooling remains superior for raw throughput. Through isolated ablations, we show that the dwell guard is necessary for run-to-run reproducibility. Finally, exploratory boundary probes indicate that the passive operating envelope closes at ambient temperatures (

\ge 27^\circ

C) where nonlinear leakage defeats DVFS-based control. These results indicate that, within the mapped envelope, correct scheduling can make mechanical cooling unnecessary for sustained edge inference on this platform.

Abstract

Efficient Neural Network Inference

Explore similar work

Sustainable Edge Vision via Empirically Calibrated DVFS: Eliminating Thermal Throttling on Passively Cooled Hardware

Joint Optimization of Memory and Computing Frequency for Energy-Efficient DNN Inference

Energy-Efficient GPU DVFS for Fine-Tuning of SLMs on Resource-constrained Embedded Devices