CPUBone: Efficient Vision Backbone Design for Devices with Low Parallelization Capabilities

Moritz Nottebaum; Matteo Dunnhofer; Christian Micheloni

cs.CVMar 27, 2026

CPUBone: Efficient Vision Backbone Design for Devices with Low Parallelization Capabilities

Authors: Moritz Nottebaum, Matteo Dunnhofer, Christian Micheloni

Organizations: University of Udine, Italy · York University, Canada

Abstract

Recent research on vision backbone architectures has predominantly focused on optimizing efficiency for hardware platforms with high parallel processing capabilities. This category increasingly includes embedded systems such as mobile phones and embedded AI accelerator modules. In contrast, CPUs do not have the possibility to parallelize operations in the same manner, wherefore models benefit from a specific design philosophy that balances amount of operations (MACs) and hardware-efficient execution by having high MACs per second (MACpS). In pursuit of this, we investigate two modifications to standard convolutions, aimed at reducing computational cost: grouping convolutions and reducing kernel sizes. While both adaptations substantially decrease the total number of MACs required for inference, sustaining low latency necessitates preserving hardware-efficiency. Our experiments across diverse CPU devices confirm that these adaptations successfully retain high hardware-efficiency on CPUs. Based on these insights, we introduce CPUBone, a new family of vision backbone models optimized for CPU-based inference. CPUBone achieves state-of-the-art Speed-Accuracy Trade-offs (SATs) across a wide range of CPU devices and effectively transfers its efficiency to downstream tasks such as object detection and semantic segmentation. Models and code are available at https://github.com/altair199797/CPUBone.

Figures & tables

Figure 1: Comparison of ARM CPU latency and ImageNet top-1 accuracy of recent image classification architectures, including our proposed CPUBone models. Markers refer to different model sizes within each architecture family. CPUBone consistently achieves lower latency at comparable accuracy, highlighting its CPU-optimized design.

Figure 2: Design of MBConv variants, including the two proposed Grouped MBConv (GrMBConv) and Grouped Fused MBConv (GrFuMBConv). In both grouped variants, the first convolution is configured with $groups=2$ . All variants expand the channel dimension in the first convolution by the expansion factor (set to four in this figure) and reduce it again by the expansion factor in the last convolution.

Grouping Experiment		Channel Dimension
Device	Variant	32	64	128	256	512	Avg.
	MBConv	7.8	13.3	21.4	26.4	26.2	19.0
Pi5 CPU	Gr MBConv	6.3	10.6	16.7	22.1	23.3	15.8 (-16%)
(MMACs/ms)	Fu MBConv	40.6	45.3	40.7	32.4	24.5	36.7
	GrFu MBConv	31.0	42.1	44.4	34.7	30.4	36.5 (-0%)
	MBConv	51.0	59.8	60.9	53.4	44.5	53.9

Table 1: Execution efficiency of MACs, measured in MMACs/ms, for the four MBConv variants (MBConv, GrMBConv, FuMBConv and GrFuMBConv) for different channel dimensions. Bold entries indicate whether the grouped or ungrouped variant has higher MACpS.

Dwise		Channel Dimension
Resol.	Type	128	256	512	1024	Avg.
7×7	nmk	0.75	0.78	0.78	0.80	0.78
7×7	smk	0.66	0.70	0.70	0.73	0.70
14×14	nmk	1.26	1.28	1.27	1.24	1.26
14×14	smk	1.13	1.15	1.16	1.08	1.13
28×28	nmk	1.97	1.82	1.64	1.64	1.77

Table 2: Execution efficiency experiment on ARM CPU on the effect of kernel size reduction for depthwise convolutions, by measuring MMACs/ms. Resol. refers to operating resolution, Type refers to the kernel size (

nmk=3\times 3

smk=2\times 2

). Bold entries indicate whether nmk or smk variant has higher MACpS.

		Ungrouped			Groups=2
Resol.	Type	Channel Dim.			Channel Dim.
		128	256	Avg.	128	256	Avg.
7×7	nmk	39.36	28.63	33.99	37.88	32.42	35.15
7×7	smk	41.21	33.13	37.17	36.11	36.42	36.27
14×14	nmk	40.90	33.45	37.17	44.56	35.21	39.88
14×14	smk	46.94	36.01	41.48	43.86	40.58	42.22

Table 3: Execution efficiency experiment on ARM CPU on the effect of kernel size reduction. We test the GrFuMBConv (Groups=2) and FuMBConv (Ungrouped) block by measuring MMACs/ms. Resol. refers to operating resolution, Type refers to the kernel size (

nmk=3\times 3

smk=2\times 2

). Bold entries indicate whether nmk or smk variant has higher MACpS.

Model	Params	MACs	Pi5 CPU	Pixel 7 Pro CPU	Intel CPU	Top-1
Model	(M)	(M)	(ms) $\downarrow$	(ms) $\downarrow$	(ms) $\downarrow$	(%)
EffFormerV2-S0 [ 14 ]	3.5	400	36.3	39.4	7.9	73.7
GhostNetV2 x1.0 [ 27 ]	6.2	183	35.2	5.8	9.0	75.3
FastViT-T8 [ 30 ]	3.6	705	65.9	44.0	10.6	75.6
MobileViG-Ti [ 21 ]	5.3	661	48.2	45.2	9.2	75.7
EfficientMod-xxs [ 20 ]	4.7	579	47.5	25.3	9.2	76.0

Table 4: Performance on ImageNet-1K validation set. Evaluation resolution is 224×224, except for FastViT models who operate on 256x256. Besides MobileVig [ 21 ] , no distillation nor pretraining is used for fair comparison. Models are sorted and grouped by top-1 accuracy. The highest top-1 accuracy in each group is bold.

Model version	Params	MACs	Pi5 CPU Lat.	Pixel 4 CPU	Pixel 7 Pro CPU Lat.	Intel CPU Lat.	Top-1
Model version	(M)	(M)	(ms)	(ms)	(ms)	(ms)	(%)
Just original MBConvs	13.3	758	48.6	25.4	20.1	12.63	78.5 (-0.2%)
Using transpose	14.4	840	39.7	26.3	20.3	10.73	78.9 (+0.2%)
Groups=8	11.0	560	28.7	21.2	16.0	8.8	78.0 (-0.7%)
Groups=4	11.5	622	29.6	21.9	17.9	9.0	78.4 (-0.3%)
B0-plain	14.1	944	39.1	29.6	20.8	9.4	78.4 (-0.3%)

Table 6: Ablation study of CPUBone-B1, featuring singular changes. We also include B0-plain, an ablation to CPUBone-B0.

Backbone	Pi5 CPU Lat.	AP
Backbone	(ms)	(%)
MobileNetV3 [ 12 ]	158.0	29.9
MNv4-Conv-M [ 24 ]	299.5	32.6
PVTv2-B0 [ 32 ]	587.7	37.2
CPUBone-B0 (ours)	131.5	37.5
LowFormer-B0 [ 22 ]	226.9	38.6

Table 7: Results on object detection using RetinaNet head [ 16 ] . Backbone latency is measured under resolution

512\times 512

. Models are grouped by AP. Best value in each group is made bold.

Dwise GPU		Channel Dimension
Resol.	Type	128	256	512	1024	Avg.
7×7	nmk	6.6	13.1	26.5	52.4	24.7
7×7	smk	1.2	2.3	4.7	9.3	4.4
14×14	nmk	26.4	52.7	103.8	205.3	97.0
14×14	smk	4.7	9.0	18.7	36.3	17.2
28×28	nmk	103.2	203.5	300.3	308.0	228.7

Table 8: Execution efficiency experiment on GPU on the effect of kernel size reduction for depthwise convolutions, by measuring MMACs/ms. Resol. refers to operating resolution, Type refers to the kernel size (

nmk=3\times 3

smk=2\times 2

). Bold entries indicate whether nmk or smk variant has higher MACpS. Same as Table 2 , but on GPU.

GPU		Ungrouped			Groups=2
Resol.	Type	Channel Dim.			Channel Dim.
		128	256	Avg.	128	256	Avg.
7×7	nmk	661.6	1075.7	868.6	355.1	893.7	624.4
7×7	smk	284.4	882.9	583.6	145.3	627.2	386.2
14×14	nmk	2374.3	3545.5	2959.9	1422.6	3026.8	2224.7
14×14	smk	1081.8	2021.3	1551.5	627.0	1671.9	1149.4

Table 9: Execution efficiency experiment on GPU on the effect of kernel size reduction. We test the GrFuMBConv (Groups=2) and FuMBConv (Ungrouped) block by measuring MMACs/ms. Resol. refers to operating resolution, Type refers to the kernel size (

nmk=3\times 3

smk=2\times 2

). Bold entries indicate whether nmk or smk variant has higher MACpS. Same as Table 3 , but on GPU.

Grouping=2 Experiment ARM CPU		Channel Dimension
Resolution	Variant	32	64	128	256	512	Avg.
	MBConv	4.1	8.5	14.9	22.4	23.5	14.7
7x7	Gr MBConv	3.7	6.5	11.6	17.7	19.5	11.8 (-19%)
(MMACs/ms)	Fu MBConv	25.5	36.3	38.7	27.8	24.3	30.5
	GrFu MBConv	14.7	28.3	37.2	32.3	26.5	27.8 (-8%)
	MBConv	7.8	13.3	21.4	26.4	26.2	19.0

Table 10: Execution efficiency of MACs, measured in MMACs/ms on the ARM CPU of the Raspberry Pi5, for the four MBConv variants (MBConv, GrMBConv, FuMBConv and GrFuMBConv) for different channel dimensions and operating resolutions. Bold entries indicate whether the grouped or ungrouped variant has higher MACpS. All grouped variants have

groups=2

. It is similar to Table 1 , but for resolutions

7\times 7

14\times 14

28\times 28

and

56\times 56

and only on ARM CPU.

Grouping=4 Experiment ARM CPU		Channel Dimension
Resolution	Variant	32	64	128	256	512	Avg.
	MBConv	4.1	8.5	14.9	22.4	23.5	14.7
7x7	Gr MBConv	3.1	5.4	10.2	17.4	19.9	11.2 (-23%)
(MMACs/ms)	Fu MBConv	25.5	36.3	38.7	27.8	24.3	30.5
	GrFu MBConv	10.2	18.5	31.4	34.9	32.0	25.4 (-16%)
	MBConv	7.8	13.3	21.4	26.4	26.2	19.0

Table 11: Execution efficiency of MACs, measured in MMACs/ms on the ARM CPU of the Raspberry Pi5, for the four MBConv variants (MBConv, GrMBConv, FuMBConv and GrFuMBConv) for different channel dimensions and operating resolutions. Bold entries indicate whether the grouped or ungrouped variant has higher MACpS. It is similar to Table 1 , but for resolutions

7\times 7

14\times 14

28\times 28

and

56\times 56

, only on ARM CPU and all grouped variants have

groups=4

instead of

groups=2

of Table 1 .

Grouping=2 Experiment GPU		Channel Dimension
Resolution	Variant	32	64	128	256	512	Avg.
	MBConv	8.6	31.7	114.4	474.0	1746.1	475.0
7x7	Gr MBConv	5.6	20.4	82.4	328.4	1094.9	306.3 (-35%)
(MMACs/ms)	Fu MBConv	45.6	182.4	672.7	1089.9	1309.8	660.1
	GrFu MBConv	23.5	85.6	342.2	878.4	1264.3	518.8 (-21%)
	MBConv	34.2	112.3	428.6	1747.3	3494.0	1163.3

Table 12: Execution efficiency of MACs, measured in MMACs/ms on the Nvidia TITAN RTX GPU, for the four MBConv variants (MBConv, GrMBConv, FuMBConv and GrFuMBConv) for different channel dimensions and operating resolutions. Bold entries indicate whether the grouped or ungrouped variant has higher MACpS. All grouped variants have

groups=2

. It is similar to Table 1 , but for resolutions

7\times 7

14\times 14

28\times 28

and

56\times 56

and only on GPU.

Efficient Neural Network Inference

Efficient neural network inference reduces the time, memory, and energy needed to run trained models across hardware and deployment settings.

See all 468 papers

Beyond MACs: Hardware Efficient Architecture Design for Vision Backbones
Mar 27, 2026Moritz Nottebaum, Matteo Dunnhofer, Christian MicheloniEfficient Neural Network Inference Efficient ViTs
Machine Learning and Perception Lab, University of Udine Via delle Scienze 206, Udine, 33100, UD, Italy. · Centre for Vision Research, York University 4700 Keele St, Toronto, M3J 1P3, ON, Canada.
Accelerating Vision Foundation Models with Drop-in Depthwise Convolution
May 21, 2026Carmelo Scribano, Mohammad Mahdi, Nedyalko Prisadnikov +5Vision Foundation Models Vision Foundation Model Adaptation
University of Modena and Reggio Emilia, Modena, Italy. · INSAIT, Sofia University “St. Kliment Ohridski”, Sofia, Bulgaria
MicroViTv2: Beyond the FLOPS for Edge Energy-Friendly Vision Transformers
May 11, 2026Novendra Setyawan, Chi-Chia Sun, Mao-Hsiu Hsu +2Energy-Efficient ML Edge Inference
Department of Electro-Optics, National Formosa University, Taiwan · Department of Electrical Engineering, University of Muhammadiyah Malang, Indonesia · Department of Electrical Engineering, National Taipei University, Taiwan +1

Mar 27, 2026J/K move · Enter open · S save

Beyond MACs: Hardware Efficient Architecture Design for Vision Backbones

Moritz Nottebaum, Matteo Dunnhofer, Christian Micheloni

Machine Learning and Perception Lab, University of Udine Via delle Scienze 206, Udine, 33100, UD, Italy. · Centre for Vision Research, York University 4700 Keele St, Toronto, M3J 1P3, ON, Canada.

Efficient Neural Network Inference Efficient ViTs Efficient Attention Edge Inference

Open paper AskPDF arXiv

Vision backbone networks play a central role in modern computer vision. Enhancing their efficiency directly benefits a wide range of downstream applications. To measure efficiency, many publications rely on MACs (Multiply Accumulate operations) as a predictor of execution time. In this paper, we experimentally demonstrate the shortcomings of such a metric, especially in the context of edge devices. By contrasting the MAC count and execution time of common architectural design elements, we identify key factors for efficient execution and provide insights to optimize backbone design. Based on these insights, we present LowFormer, a novel vision backbone family. LowFormer features a streamlined macro and micro design that includes Lowtention, a lightweight alternative to Multi-Head Self-Attention. Lowtention not only proves more efficient, but also enables superior results on ImageNet. Additionally, we present an edge GPU version of LowFormer, that can further improve upon its baseline's speed on edge GPU and desktop GPU. We demonstrate LowFormer's wide applicability by evaluating it on smaller image classification datasets, as well as adapting it to several downstream tasks, such as object detection, semantic segmentation, image retrieval, and visual object tracking. LowFormer models consistently achieve remarkable speed-ups across various hardware platforms compared to recent state-of-the-art backbones. Code and models are available at https://github.com/altair199797/LowFormer/blob/main/Beyond_MACs.md.

Abstract

Efficient Neural Network Inference

Explore similar work

Beyond MACs: Hardware Efficient Architecture Design for Vision Backbones

Accelerating Vision Foundation Models with Drop-in Depthwise Convolution

MicroViTv2: Beyond the FLOPS for Edge Energy-Friendly Vision Transformers