Time-series sensing increasingly relies on neural networks, yet deployment on resource-constrained microcontrollers remains challenging. Aggressive model compression can reduce accuracy, while restricting weights to multiplication-free representations introduces further losses. We present BeFit, an end-to-end framework that shifts model capacity from weights to learnable Bernstein polynomial activations. BeFit (i) finds compact Bernstein-activated convolutional neural networks (BeCNNs) through hardware-aware neural architecture search and trains them via knowledge distillation; (ii) projects their weights to multiplication-free codes, freezes the codes, and recovers the resulting accuracy loss through the learnable activations; and (iii) generates model-specific kernels that exploit the projected weights for multiplication-free convolutions. On human activity recognition, BeCNNs are 2.75 percentage points (pp) more accurate on average than ReLU CNNs of identical architecture. On a TI CC3220SF (Cortex-M4), weight projection speeds BeCNNs up by 2.2x on average while losing 0.8 pp, compared with 2.4 pp for ReLU CNNs. Against the MLPerf Tiny DS-CNN reference model running on TensorFlow Lite Micro with CMSIS-NN, the unprojected BeCNN achieves 1.0 pp higher accuracy, 2.2x lower latency, and 6.4x lower flash, while the projected model remains within 1.0 pp of the reference accuracy and runs 5.9x faster. In emulation on a RISC-V core without a hardware multiplier, the instruction-count reduction relative to the DS-CNN increases from 5.2x on the Cortex-M4 to 147x. BeFit also extends to transformers and MLPs, and across all tasks the final models stay within 1.56 pp of their float32 teachers on average.
Figures & tables
Figure 1. BeFit pipeline. A hardware-aware NAS finds a compact BeCNN, and a high-accuracy teacher helps train it via KD. The BeCNN’s convolutional weights are projected to multiplication-free codes, the learnable activations recover the lost accuracy, and a generated kernel exploits these codes for MCU deployment. Weights and curves are illustrative. Five stages from left to right: a pretrained teacher; a hardware-aware NAS that searches for a compact student for higher accuracy and lower instruction count under memory constraints; the compressed student BeCNN with fewer filters and wider strides, trained by distillation and quantized to int8, whose weights are projected to multiplication-free codes while learnable Bernstein activations recover accuracy with self-distillation, shown as one curve before and after recovery; projection-aware generated kernels, where one filter's codes become add and subtract instructions, stored as code, data or a mix, and each activation is one lookup-table load; and deployment on the TI CC3220SF, an 80 MHz Cortex-M4 with 256 KiB RAM and 1 MiB flash. Knowledge distillation links the teacher and the student.
Figure 2. A BeCNN: a CNN whose activations are learnable Bernstein polynomials (Eq. ( 1 )). Each activation normalizes its input to t∈[0,1] and outputs ∑iciBN,i(t) ; only the coefficients are learned, and the curve stays within the convex hull of its control points. Layer count is schematic. Top: a CNN pipeline from an input signal through convolution, Bernstein activation, pooling, convolution, Bernstein activation, global average pooling and a fully connected layer to class logits. Bottom, inside one Bernstein activation: the pre-activation is normalized to t in [0,1], evaluated on five degree-4 Bernstein basis functions, weighted by five learnable coefficients, and summed into a smooth activation curve that lies within the convex hull of its control points.
Figure 3. Projection, recovery and multiplication-free convolution. Convolutional weights are approximated as W≈γP with a per-filter scale γ and codes P . Recovery freezes P and trains the Bernstein coefficients, biases, scales and classifier. At inference, nonzero codes are implemented using additions and shifts, so the convolution needs no multiplier. Matrices and curves are illustrative. Three panels. Left: data-aware projection of floating-point convolutional weights to power-of-two codes in {0, plus or minus 1, 2, 4}, times a per-filter scale. Middle: recovery with the codes frozen, training the Bernstein activation, convolutional biases, per-filter scales and the classifier with the task loss and self-distillation; the activation curve is shown after projection and after recovery, followed by pooling and an int8 classifier. Right: multiplication-free convolution, where each code becomes a shift and add: a zero code is skipped, a code of plus 2 is a left shift by one and a code of minus 1 is a subtraction.
Figure 4. A schematic four-filter chunk of projected weights, stored two ways with identical outputs. As code (panel 3a), each nonzero weight becomes an addition or subtraction: faster, more flash. As data (panel 3b), index lists drive one shared loop: slower, less flash. Panel 4 shows the Cortex-M4 optimizations. A grid of projected weights for four filters and six taps. Each weight value follows one rule: zero emits no instruction, plus or minus one is a SIMD add or subtract, plus or minus a power of two adds into its power group. Filter 0 is shown two ways: as generated code, and as index lists read by one shared loop. On the MCU, one 32-bit register holds an input at two output positions. As code, one load of this input serves four filters; as data, each nonzero weight loads its index, loads its input and adds or subtracts it. Both options sum the same power groups.
Task
Input
Classes
Train
Val
Test
UCI-HAR
6×128
6
5,867
1,485
2,947
HHAR
6×128
6
30,850
5,374
10,755
PAMAP2
18×128
12
10,982
2,107
1,934
Speech Cmds v2
49×10 MFCC
12
85,511
10,102
4,890
Speech Cmds v2 (KWT)
98×40 MFCC
35
84,843
9,981
11,005
UJIIndoorLoc
520 RSSI
3+5
13,955
5,982
1,111
Table 1. Tasks and splits.
Task
Model
Acc. (%)
Lat. (ms)
Flash (KiB)
RAM (KiB)
UCI-HAR
ResNet
96.01 ± 0.32 (96.52)
4,569
610.9
77.6
BeFit dense
94.70 ± 0.09
0.92 ( ↓ 4,971 × )
11.9 ( ↓ 51.2 × )
2.5 ( ↓ 31.6 × )
BeFit projected
93.96 ± 0.79
0.48 ( ↓ 9,580 × )
8.6 ( ↓ 70.7 × )
2.5 ( ↓ 31.4 × )
HHAR
DeepSense
94.72 ± 0.57 (95.08)
934.4
660.1
52.9
BeFit dense
92.91 ± 0.25
2.49 ( ↓ 375 × )
16.6 ( ↓ 39.8 × )
2.7 ( ↓ 19.3 × )
BeFit projected
92.16 ± 0.53
1.12 ( ↓ 834 × )
18.1 ( ↓ 36.4 × )
2.7 ( ↓ 19.7 × )
Table 2. Each task’s int8 teacher on TFLM with CMSIS-NN against BeFit . Parentheses: teacher’s float32 accuracy; ( ↓× ): TFLM teacher / BeFit . UJIIndoorLoc: joint building-and-floor accuracy. ∗ Does not fit the board: AST measured in QEMU on TFLM’s reference kernels, so it has no latency; DeepConvLSTM has no Cortex-M build, so its values are lower bounds.
Instr. (M)
Flash
RAM
Task
Model
Acc. (%)
M4
RV32EC
(KiB)
(KiB)
UCI-HAR
ResNet
96.01 ± 0.32
120.4
5,103.6
621.6
77.4
BeFit projected
93.96 ± 0.79
0.013 ( ↓ 8,962 × )
0.093 ( ↓ 54,798 × )
25.8 ( ↓ 24.1 × )
3.2 ( ↓ 24.4 × )
HHAR
DeepSense
94.72 ± 0.57
24.8
790.7
682.9
53.3
BeFit projected
92.16 ± 0.53
0.032 ( ↓ 767 × )
0.274 ( ↓ 2,890 × )
82.0 ( ↓ 8.3 × )
3.5 ( ↓ 15.4 × )
PAMAP2
DeepConvLSTM
84.13 ± 1.86
—
25,770.9
941.5
291.8
Table 3. Teachers against BeFit projected : QEMU instructions per inference on the Cortex-M4 and RV32EC, with RV32EC flash and RAM; ( ↓× ): TFLM teacher / student. —: no Cortex-M build. On RV32EC, teachers use CMSIS-NN’s portable C code, with TinyEngine’s Arm DSP instructions rewritten in plain C.
Dimension
Values
Dilation groups G
1, 2, 3 (dilations 1,…,2G−1 )
Filters per group f
8, 12, 16, 24, 32, 48, 64, 96, 128
Kernel size k
3, 5, 7, 9
Stride s (all groups)
1, 2, 4, 8, 16, 32
Normalization
none, BatchNorm
Learning rate
3×10−3 , 10−2 , 3×10−2
Table 4. HAR search space. The ReLU control omits the last two rows.
Figure 5. Int8 test accuracy of the BeCNN minus the ReLU model, in pp, for the models selected on validation accuracy at every instruction budget. Dashed lines: the budgets carried forward. Four panels, one per task, show the test accuracy difference between the BeCNN and the ReLU model selected under each instruction budget, on a log axis. Blue areas above zero mark budgets where the BeCNN is ahead and orange areas below zero where ReLU is ahead. Blue covers almost all of the range on UCI-HAR, HHAR and PAMAP2 and most of the range below one million instructions on Speech Commands. Dashed lines mark the budgets used in the paper.
Task
Budget
Model
Shape
Instr.
Test (%)
Δ Acc.
UCI-HAR
6k
BeCNN
1 × 8, k7, s32, N5
5,530
83.33±0.63
+3.82
ReLU
1 × 8, k7, s32
5,497
79.50±0.07
15k
BeCNN
1 × 8, k9, s8, N31
12,334
91.81±1.01
+2.41
ReLU
3 × 8, k3, s16
10,727
89.40±0.65
25k
BeCNN
3 × 12, k3, s8, BN, N31
22,841
94.23±0.29
+2.31
ReLU
3 × 16, k5, s16
23,189
91.92±0.15
Table 5. Best model per instruction budget (HAR). Shape: dilation groups × filters per group, kernel size (k), stride (s), BatchNorm (BN), Bernstein degree (N). Δ Acc.: BeCNN minus ReLU, in pp. Models are selected on validation accuracy; Test: int8 test accuracy, three seeds.
Budget
Model
Shape
Instr.
Test (%)
Δ Acc.
0.5M
BeCNN
w16, k6, s4, h4, N15
423,591
79.14±1.46
+1.58
ReLU
w16, k6, s4, h4
438,713
77.56±1.03
1M
BeCNN
w32, k6, s4, h1, N15
872,678
86.00±0.11
−1.69
ReLU
w32, k6, s4, h4
953,880
87.69±0.52
2M
BeCNN
w48, k6, s3, h1, N15
1,934,222
89.86±0.58
+0.68
ReLU
w48, k10, s4, h1
1,595,564
89.18±0.21
Table 6. Best DS-CNN per instruction budget. Shape: width (w), first-convolution kernel (k × 4) and stride (s × 2), curves per channel at the last activation (h), Bernstein degree (N). Models are selected on validation accuracy; Test: int8 test accuracy, three seeds.
Task
Budget
Model
Test (%)
Δ Acc.
Flash
RAM
Lat.
UCI-HAR
6k
BeCNN
83.92±0.08
+2.73
9.1
2.4
0.221
ReLU
81.19±1.01
8.2
2.4
0.215
15k
BeCNN
92.29±1.12
+2.32
9.3
2.5
0.511
ReLU
89.97±0.61
8.7
2.4
0.418
25k
BeCNN
94.70±0.09
+1.19
11.9
2.5
0.919
ReLU
93.51±0.73
11.0
2.5
0.926
Table 7. HAR NAS models (Table 5 ) after KD. Test: int8, three seeds. Flash and RAM (KiB) and latency (ms) on the CC3220SF. Δ Acc.: BeCNN minus ReLU (pp).
Budget
Model
Test (%)
Δ Acc.
Flash
RAM
Lat.
0.5M
BeCNN
81.07±0.35
+2.09
8.0
2.7
17.4
ReLU
78.98±0.54
7.8
2.7
16.9
1M
BeCNN
86.41±0.20
−0.82
10.0
4.6
35.0
ReLU
87.23±0.59
11.3
5.0
36.5
2M
BeCNN
90.74±0.46
+0.54
13.8
8.6
76.8
ReLU
90.20±1.63
14.2
6.7
60.8
Table 8. DS-CNN NAS models (Table 6 ) after distillation; Test: int8, three seeds. Flash and RAM (KiB) and latency (ms) on the CC3220SF. Δ Acc.: BeCNN minus ReLU (pp).
Flash (KiB)
RAM (KiB)
Latency (ms)
Test (%)
Dataset
Shape
BeCNN / ReLU
Δ
BeCNN / ReLU
Δ
BeCNN / ReLU
Δ
BeCNN
ReLU
Δ
UCI-HAR
1 × 8, k3, s32
8.7 / 7.8
+11.5%
2.4 / 2.4
+1.2%
0.185 / 0.178
+3.6%
80.00 ± 0.97
77.15 ± 0.51
+2.85
UCI-HAR
1 × 8, k7, s16
9.1 / 8.2
+11.0%
2.4 / 2.4
+1.1%
0.294 / 0.286
+2.8%
88.34 ± 0.58
87.12 ± 1.25
+1.22
UCI-HAR
1 × 8, k9, s8
9.3 / 8.4
+10.8%
2.5 / 2.4
+1.1%
0.511 / 0.500
+2.3%
89.91 ± 1.15
88.36 ± 0.86
+1.55
HHAR
1 × 8, k3, s32, BN
8.7 / 7.8
+11.5%
2.4 / 2.4
+1.2%
0.185 / 0.178
+3.6%
73.71 ± 1.27
70.00 ± 0.14
+3.70
HHAR
3 × 8, k5, s32, BN
11.2 / 9.3
+20.4%
2.5 / 2.5
+1.1%
0.389 / 0.372
+4.8%
85.19 ± 1.90
82.56 ± 0.79
+2.63
Table 9. BeCNN and ReLU CNN with identical architectures, trained without distillation. Hardware on the CC3220SF. Δ : BeCNN minus ReLU, relative to ReLU (%) for hardware and in pp for test acc.
Teacher
BeFit (Ours)
ReLU (TWN ( Li et al., 2016 ) )
Acc.
Acc. (%)
Lat. (ms)
Acc. (%)
Lat. (ms)
Dataset
Budget
(%)
Dense
Proj.
Δ
Δteacher
Dense
Proj.
Spd.
Dense
Proj.
Δ
Δteacher
Dense
Proj.
Spd.
UCI-HAR
6k
96.01
83.92
83.20
−0.72
−12.81
0.22
0.15
1.49 ×
81.19
79.06
−2.13
−16.95
0.22
0.14
1.49 ×
15k
92.29
92.10
−0.19
−3.91
0.51
0.25
2.08 ×
89.97
88.61
−1.36
−7.40
0.42
0.24
1.73 ×
25k
94.70
93.96
−0.74
−2.05
0.92
0.48
1.93 ×
93.51
92.22
−1.29
−3.79
0.93
0.46
2.02 ×
HHAR
7k
94.72
81.55
81.15
−0.40
−13.57
0.26
0.15
1.66 ×
77.02
70.08
−6.94
−24.64
0.25
0.15
1.66 ×
Table 10. Distilled students after powers-of-two projection, on the generated kernel, against each task’s int8 teacher. Δ : Proj. − Dense; Δteacher : Proj. − Teacher (pp). Spd.: Dense/Proj. Bold: smaller drop.
Weights
Test (%)
Instr.
Flash
RAM
Lat.
Speedup
Dense int8
94.70
22,819
11.9
2.5
0.919
1.00
Ternary {0,±1}
70% zeros
92.06
9,121
6.9
2.4
0.352
2.61
50% zeros
91.87
10,043
7.3
2.4
0.385
2.39
30% zeros
90.74
10,989
7.7
2.4
0.418
2.20
Free
93.20
9,710
7.2
2.4
0.378
2.43
Table 11. UCI-HAR 25k BeCNN under each weight set. Flash and RAM in KiB, latency in ms. Speedup: against dense model.
UCI-HAR
HHAR
Ternary codes
None
Coef.
Full
None
Coef.
Full
30% zeros
87.69
88.99
89.31
76.84
78.20
80.79
50% zeros
87.34
90.45
90.78
74.04
78.25
79.57
70% zeros
85.21
90.07
90.76
79.82
82.83
82.96
85% zeros
68.57
86.40
88.51
68.37
78.62
81.95
Table 12. Recovery ablation on 16-filter BeCNNs (dense: 92.00% on UCI-HAR, 89.53% on HHAR). None: no recovery; Coef.: Bernstein coefficients only; Full: coefficients, biases, scales and head. Test (%).
Network
Model
Kernel
Test (%)
Instr.
Lat. (ms)
Flash (KiB)
RAM (KiB)
Speedup
HHAR CNN, 3 × 16, k7, s16
ReLU
TFLM+CMSIS-NN
84.50±0.51
79,406
3.05
70.9
6.8
–
ReLU
Ours
86.30±0.94
28,250
1.15
12.1
2.6
2.66 ×
BeFit dense
Ours
91.63±1.27
28,184
1.19
15.5
2.6
2.57 ×
DS-CNN, 0.5M
ReLU
TFLM+CMSIS-NN
78.36±0.77
674,407
21.33
84.0
9.1
–
ReLU
Ours
78.98±0.54
438,713
16.90
7.8
2.7
1.26 ×
BeFit dense
Ours
81.07±0.35
423,591
17.42
8.0
2.7
1.22 ×
Table 13. The same network on TFLM+CMSIS-NN ( David et al., 2021 ; Lai et al., 2018 ) and on our dense kernel (Ours), with BeFit dense of the same shape on ours. TFLM’s KWT uses one custom GELU operator. Speedup: against TFLM+CMSIS-NN.
Figure 6. Board speedup of the projected BeCNNs over their unprojected models. Projected weights only pay off on the generated kernel: on the dense kernel, the latency stays the same ( 1× ).
KWT
HHAR
Weights
Lat. (ms)
Speedup
Flash (KiB)
Lat. (ms)
Speedup
Flash (KiB)
Dense
1,859
1.00 ×
249.5
2.49
1.00 ×
16.6
Code
1,036
1.79 ×
391.3
1.12
2.22 ×
18.1
Data
1,810
1.03 ×
167.9
6.14
0.41 ×
10.2
Mix
1,529
1.22 ×
246.3
2.79
0.89 ×
15.9
Table 14. A projected KWT (6 × 64) and BeCNN (HHAR 60k) with their codes stored three ways: Code, Data and Mix. Dense: the unprojected model on the dense kernel. Speedup: over Dense.
Shape
Model
Test (%)
Instr. (M)
Lat. (ms)
Flash (KiB)
RAM (KiB)
12 × 3072
AST ∗
97.10
95,738
—
83,797.0
2,353.9
6 × 256
GELU
96.06±0.13
80.4
2,908
386.0
60.5
BeFit dense
96.48±0.31
76.5
2,807
471.2
60.5
BeFit projected
95.90±0.21
51.3
1,995
468.0
62.5
6 × 128
GELU
95.33±0.18
62.1
2,224
281.0
60.5
BeFit dense
96.21±0.13
60.2
2,176
323.8
60.5
Table 15. KWTs on Speech Commands (35 words). Shape: blocks × FFN width h . ∗ Does not fit the board: measured in QEMU on TFLM’s reference kernels, so it has no latency.
Shape
Model
Test (%)
Lat. (ms)
Flash (KiB)
RAM (KiB)
520–256–128 –64–128–8
Teacher
92.02±0.64
14.68
211.4
2.9
520–128–8
BeFit dense
92.41±0.40
6.66 ( ↓ 2.21 × )
162.2 ( ↓ 1.30 × )
2.4 ( ↓ 1.19 × )
BeFit projected
92.41±0.52
1.18 ( ↓ 12.5 × )
108.8 ( ↓ 1.94 × )
1.5 ( ↓ 1.98 × )
520–64–8
BeFit dense
92.59±0.36
3.34 ( ↓ 4.40 × )
91.9 ( ↓ 2.30 × )
2.4 ( ↓ 1.19 × )
BeFit projected
92.56±0.04
0.60 ( ↓ 24.4 × )
65.4 ( ↓ 3.23 × )
1.2 ( ↓ 2.41 × )
520–32–8
BeFit dense
91.39±0.40
1.68 ( ↓ 8.74 × )
56.7 ( ↓ 3.73 × )
2.4 ( ↓ 1.19 × )
Table 16. MLPs on UJIIndoorLoc: the teacher on TFLM with CMSIS-NN against BeFit students; ( ↓× ): teacher / student. Test: joint building-and-floor accuracy.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Task
Teacher
Params
Float32
Int8
UCI-HAR
ResNet ( Wang et al., 2017 )
0.51 M
96.52 ± 0.46
96.01 ± 0.32
HHAR
DeepSense ( Yao et al., 2017 )
0.47 M
95.08 ± 0.52
94.72 ± 0.57
PAMAP2
DeepConvLSTM ( Ordóñez and Roggen, 2016 )
0.85 M
86.99 ± 1.10
84.13 ± 1.86
KWS (12)
DS-CNN ( Banbury et al., 2021a ; MLCommons, 2026 )
24.9 K
92.17
91.62
KWS (35)
AST ( Gong et al., 2021 )
85.4 M
98.03
97.10
UJIIndoorLoc
MLP ( Kim et al., 2018 )
0.18 M
92.02 ± 0.57
92.02 ± 0.64
Appendix
Table 17. Teachers. Float32 and int8: test accuracy (%); UJIIndoorLoc: joint building-and-floor accuracy. ± : population SD over three seeds; no ± : one released model.
Teacher
Lat. (ms)
Slower
Flash (KiB)
RAM (KiB)
ResNet
22,205
4.9 ×
576.1
77.1
DeepSense
2,373
2.5 ×
625.3
51.0
MLPerf Tiny DS-CNN
1,591
6.1 ×
92.9
23.5
UJIIndoorLoc MLP
60.8
4.1 ×
200.3
2.38
Appendix
Table 18. Int8 teachers on TFLM’s reference kernels against TFLM with CMSIS-NN, seed 0.
Figure 7. Pareto fronts: candidates are selected on three-seed validation accuracy and plotted at their int8 test accuracy. Markers: front models; step line: best test accuracy up to each instruction count; faint dots: all three-seed candidates. Four plots, one each for UCI-HAR, HHAR, PAMAP2 and Speech Commands, show int8 test accuracy against dense int8 kernel instructions on a log axis for the architecture-search fronts of BeCNNs and ReLU models. On the three HAR datasets the BeCNN staircase lies above the ReLU staircase at most instruction counts, with the largest gaps at the lowest counts. On Speech Commands it lies above the ReLU staircase below about one million instructions; above that the two cross and run close together up to 92 percent.
RV32EMC (M instr.)
Reduction ( × )
Task
Teacher
Student
M4
RV32EMC
RV32EC
UCI-HAR
374.5
0.065
8,962
5,760
54,798
HHAR
68.9
0.237
767
291
2,890
PAMAP2
1,225.1
0.138
—
8,879
154,680
KWS
18.9
2.398
5.2
7.9
147
Appendix
Table 19. Teacher and projected student on RV32EMC: QEMU instructions per inference, seed 0. Reduction: teacher / student, with the Cortex-M4 and RV32EC reductions of Table 3 for comparison.
Dataset
Budget
TWN ( Li et al., 2016 )
BeFit
UCI-HAR
6k
87.23
87.14
15k
93.45
93.54
25k
95.13
93.78
HHAR
7k
80.15
78.02
15k
85.27
84.86
60k
94.21
91.59
Appendix
Table 20. ReLU students of Table 10 at powers of two {0,±1,±2,±4} with a free share of zeros: TWN-style training ( Li et al., 2016 ) (Conventional) against our frozen-code single pass (BeFit). int8 validation accuracy (%), three-seed mean. Bold: better of the pair.