Does synthetic pre-training pay off on a microcontroller? Accuracy, latency and energy across two MCUs
Authors: Achintya Yadav
Organizations: School of Engineering, RMIT University, GPO Box 2476, Melbourne, 3001, VIC, Australia · Birla Institute of Technology and Science, Pilani, Hyderabad Campus, Jawahar Nagar, Kapra Mandal, Medchal District, Hyderabad, 500078, Telangana, India
Rendered images are cheap to label, but a microcontroller classifier must survive 8-bit integer (INT8) quantisation and compute correctly on the chip. Five models, from MobileNetV2 at 224 px to a 61K-parameter convolutional neural network (CNN), were fine-tuned on a six-class scene dataset with and without a synthetic stage, against thresholds fixed in advance, and deployed on an ESP32-S3 and an RP2040. With full real data the stage gave no detectable gain. For the main deployable model the INT8 difference was +0.312 percentage points (pp), its interval below the +1.6 pp gain recomputed from the source study; quantisation did not detectably change it, nor did model size. With 10% of the real data the stage paid off for the CNN, by +3.438 pp on one seed and +2.715 pp over three; its cause is open. On the ESP32-S3 the main model took 152.7 ms and 16.3 mJ above idle per inference, and adding the stage changed neither beyond preset bounds. The model's INT8 outputs matched TensorFlow Lite's reference kernels exactly, but only after a memory overrun in a vendor kernel was patched. On the RP2040 the same model took 1.80 s and 27.3 mJ above idle, about twelve times slower, and the largest 96 px model exceeded the board's flash-capacity limit. Neither the RP2040 nor the host interpreter used for scoring reproduced the reference arithmetic. A synthetic stage is worth adding only when real data are scarce, and on-device INT8 outputs should be checked against reference kernels, not inferred from host scores.
Figures & tables
Row
Model
Input (px)
Parameters
MACs ( 106 )
Start for arms B and D
Seeds per cell (A, C / B, D)
0
MobileNetV2, width 1.0
224
2,265,670
299.50
ImageNet
1 / 1
1
MobileNetV2, width 1.0
96
2,265,670
55.02
ImageNet
1 / 1
2
MobileNetV2, width 0.5
96
713,910
17.61
ImageNet
1 / 1
3
MobileNetV2, width 0.35
96
417,894
10.66
ImageNet
1 / 3
4
CNN (this study)
48
60,902
7.63
Imagenette
1 / 3
Table 1: The model ladder. Rows 3 and 4 carry the confirmatory tests and row 0 the anchor.
Test
Contrast
Data
Decision rule
Multiplicity
Anchor
Row 0, float32, B against D
25% of real data, one seed
Pass if the lower bound is above 0; fail if the upper bound is below +1 ; otherwise inconclusive
Outside Holm
Primary
Row 3, INT8, B against D
Full data, three-seed means
95% interval; a bound if it excludes +1.6
Holm
Secondary
Row 4, INT8, B ′ against D ′
Full data, three-seed means
Corrected t -test
Holm
Difference-in-differences
Row 3, float32 minus INT8
Full data, three-seed means
95% interval; “quantisation removes at most X points of the transfer benefit”
Own family
Trend regression
Slope of Δ on log10 parameters
Full data, seed 1
Corrected test on ten slopes
Outside Holm; optional
Table 2: The registered tests and the registered optional trend regression.
ESP32-S3
RP2040 (Pico H)
Board
ESP32-S3-N16R8 board; 16MB flash, 8MB octal PSRAM
Raspberry Pi Pico H
Clock
240MHz
133MHz
Runtime and kernels
ESP-IDF 6.0.2, esp-tflite-micro 1.4.0, ESP-NN 1.3.1 with a local patch
arduino-pico 5.6.0, MicroTFLite 1.0.4, CMSIS-NN without acceleration
Arena
Internal SRAM (rows 3, 4) or PSRAM (rows 1, 2), checked at boot
Internal SRAM
Supply
3,300mV set, not measured, into the 3V3 pin, after the board’s regulator
3,300mV set, not measured, into the 3V3 pin; on-board regulator shut down
Window
One Invoke()
One ModelRunInference()
Table 3: Measurement conditions for the energy figures of record.
Figure 1: Final test accuracy of the four arms at each row of the model ladder, at full real data. Row 0 is shown in float32, the precision of its registered criterion; rows 1 to 4 are INT8. Arms B and D at rows 3 and 4 are three-seed means; every other bar is seed 1. Bars show the standard deviation across the ten folds, and the vertical axis starts at 40%. At row 4, B and D are B ′ and D ′ . Rows 0 and 1 have the same parameter count. Descriptive: the registered B-against-D tests are in Table 4 .
Test
Contrast
Mean
95% CI
p
Folds >0
Verdict
Anchor
Row 0, float32, 25% data
−1.068
−1.945 to −0.191
0.022
0/10
Fail
Primary
Row 3, INT8
+0.312
−0.849 to +1.472
0.559 a
7/10
No difference; excludes +1.6
Secondary
Row 4, INT8
−0.445
−0.807 to −0.084
0.021 a
1/10
Negative
Difference-in- differences
Row 3, float32 − INT8
−0.267
−1.554 to +1.020
0.650
3/10
No difference
Table 4: Registered confirmatory results. Differences are D − B (row 4: D ′− B ′ ) in percentage points.
Run-to-run SD (pp)
Row
Precision
B
D
Pooled (RMS)
Predicted σd
Measured σd
r
Mean Δ
∣Δ∣ / pooled
3
Float32
1.109
0.825
0.977
0.798
0.324
0.954
+0.044
0.05
3
INT8
1.330
1.069
1.206
0.985
1.116
0.675
+0.312
0.26
4
Float32
1.234
1.207
1.221
0.997
0.297
0.994
−0.334
0.27
4
INT8
1.282
1.436
1.361
1.111
0.348
0.993
−0.445
0.33
Table 5: Run-to-run spread and paired spread at rows 3 and 4, full real data. Differences are D − B (row 4: D ′− B ′ ).
INT8
Float32
Row
Real data
B
D
Mean
95% CI
Folds >0
Mean
95% CI
Folds >0
3
10%
82.402
82.169
−0.233
−1.933 to +1.467
5/10
−0.270
−2.273 to +1.733
4/10
3
25%
85.445
84.975
−0.470
−2.989 to +2.049
2/10
−0.703
−2.490 to +1.085
3/10
3
50%
88.112
87.679
−0.433
−2.503 to +1.637
3/10
−0.135
−1.773 to +1.503
5/10
3
100%
89.685
89.550
−0.135
−2.053 to +1.784
5/10
−0.033
−1.324 to +1.258
4/10
4
10%
62.404
65.842
+3.438
+1.996 to +4.881
10/10
+3.338
+1.697 to +4.979
10/10
Table 6: Data-fraction sweep at rows 3 and 4, seed 1. Accuracies are INT8 means over the ten folds; differences are D − B (row 4: D ′− B ′ ) in percentage points.
Figure 2: INT8 accuracy against the share of real training data, at row 3 (left; MobileNetV2, width 0.35, ImageNet start) and row 4 (right; CNN, Imagenette start, arms B ′ and D ′ ). Top: mean accuracy of each arm over the ten folds, with bars showing the standard deviation across folds. Bottom: mean of the ten paired differences, D − B (row 4: D ′− B ′ ), with bars showing the corrected 95% interval; k /10 is the number of folds with D above B. The horizontal axis is logarithmic. Seed 1 at every fraction. The representative set for INT8 conversion is drawn from each fold’s full training split and held fixed across fractions, so only the training data varies. Descriptive: no test was registered on this sweep.
Figure 3: INT8 differences with reduced and full real data, with the addendum’s controls beside the registered values. Filled circles are registered: the seed-1 sweep values, which are descriptive registered measurements, and the primary and secondary tests at full data. Open squares are exploratory addendum results over three replicates. Row 3 is MobileNetV2 at width 0.35 from ImageNet weights; row 4 is the CNN, arms B ′ and D ′ , from Imagenette weights; the bridge is the row-3 MobileNetV2 from Imagenette weights; “shuffled labels” is row 4 with the labels of the rendered images shuffled. Bars are the corrected 95% intervals over the ten folds. Values are in Tables 6 and 4 and Supplementary Table S2.
Figure 4: Accuracy, latency and energy for the four deployed rows. (a) INT8 test accuracy of arms B and D (B ′ and D ′ at row 4), mean and standard deviation over ten folds, on the host’s default kernel path; rows 3 and 4 are three-seed means. (b) Median latency: ESP32-S3 at 240MHz on the ladder build ( main_T36.cc ), RP2040 at 133MHz on its boundary-test build; row 1 does not fit in the RP2040’s flash. (c) Idle-subtracted energy per inference ( Section 7.5 ); RP2040 energy is a post-registration addition. Shading marks the ESP32-S3 rows whose arena sits in PSRAM. Filled markers are the ESP32-S3, open markers the RP2040. Row 4 on the ESP32-S3 includes ESP-NN 1.3.1’s debug heap check ( Section 7.3 ). Row 0 is not shown.
Row
Images
Tensor
Values on an INT8 limit
ESP32-S3 exact
RP2040 exact (largest diff.)
3
20
Output
111/120
120/120
Table 9
3
20
Pre-softmax a
0/120
120/120
–
1
1
Output
6/6
6/6
Does not fit
2
1
Output
2/6
6/6
4/6 (2)
3
1
Output
5/6
6/6
6/6 (0)
4
1
Output
0/6
6/6
0/6 (13)
Table 8: Device-versus-reference agreement. Each device’s INT8 outputs are compared element by element with TensorFlow Lite’s reference kernels run on the host, with no tolerance.
Row
Exact output vectors
Same predicted class
Largest difference (steps)
4
5/20
18/20
16
3
18/20
19/20
15
2
17/20
20/20
12
Table 9: RP2040 against the reference kernels on 20 photographs (addendum).
Finding
How it showed
Effect on this paper
Status
Source
ESP-NN optimised kernels, ESP32-S3
Out-of-bounds write in the tiled depthwise kernel, version 1.3.1
Board crashed on row 3
Patched locally; every ESP32-S3 figure of record uses the patch
Absent from 1.4.1; fix in upstream commit 85961e4 , whose message does not mention it
Record; addendum ESP2
Max-pool kernel ignores the lower bound of a fused activation when the channel count is 4 modulo 8
Kernel tests on shapes the models do not use
None; not reached by the files deployed at rows 1 to 4
Reported with a fix and tests; merged upstream
Addendum ESP2
Two further defects
Same tests
None; not reached by the files deployed at rows 1 to 4
Reported privately; reproduced by Espressif; fixes due in 1.4.2
Addendum ESP2
Debug heap check inside inference, version 1.3.1
Three heap walks per inference, at row 4 only
Inside row 4’s latency and energy of record: about 0.77ms (1.4%) in SRAM, 1.0ms (1.1%) in PSRAM
Footnoted wherever row 4 appears; rows 1 to 3 unaffected
Addendum ESP2
Host scoring and the RP2040
Table 10: Defects and discrepancies found in the toolchain between the trained model and the boards.
Row
Model size ( B )
Arena used ( B )
Arena reserved ( B )
Arena location
Median latency ( \SIUnitSymbolMicros )
IQR ( \SIUnitSymbolMicros )
1
2,715,632
576,044
1,048,576
PSRAM
858,036.0
14.5
2
977,048
304,972
352,256
PSRAM
287,192.0
2.0
3
627,464
224,476
281,600
Internal SRAM
152,790.5
11.0
4
70,376
85,884
281,600
Internal SRAM
54,134.0
0.0
4, PSRAM control
70,376
85,884
281,600
PSRAM
90,469.0
0.0
Table 11: ESP32-S3 cost ladder, measured on one firmware lineage ( main_T36.cc ).
Row
Model size
Outcome
Flash used
RAM used
Arena granted
Arena used
Median latency ( \SIUnitSymbolMicros )
IQR ( \SIUnitSymbolMicros )
1
2,715,632
Flash overflow
–
–
–
–
–
–
2
977,048
Runs
1,286,948
250,248
235,000
232,412
2,659,899.0
184.5
3
627,464
Runs
937,372
224,144
208,896
205,332
1,795,947.0
133.5
4
70,376
Runs
359,548
146,320
131,072
49,348
1,096,854.5
201.5
Table 12: RP2040 outcome and cost for each row (Raspberry Pi Pico H at 133MHz ). Sizes are in bytes.
Board
Row
Arena
Idle-subtracted ( mJ )
Gross ( mJ )
Idle share of gross (%)
Power above idle ( mW )
ESP32-S3
1
PSRAM
74.025 (0.435)
201.209 (0.436)
63.2
86.5
2
PSRAM
27.587 (0.087)
69.969 (0.086)
60.6
95.7
3
SRAM
16.322 (0.058)
38.641 (0.055)
57.8
106.8
4
SRAM
6.665 (0.039)
14.582 (0.045)
54.3
123.9
4, PSRAM control
PSRAM
9.058 (0.037)
22.374 (0.034)
59.5
100.4
RP2040 a
2
SRAM
41.211 (0.201)
243.156 (0.205)
83.1
15.4
Table 13: Energy per inference on both boards: mean and standard deviation over the 30 windows of one capture.
Figure 5: INT8 accuracy of arm B against idle-subtracted energy per inference, on the ESP32-S3 (filled circles) and the RP2040 (open triangles); labels give the row and arena placement, and the dashed lines join each board’s Pareto frontier. Energy comes from one capture of one deployed file per row; accuracy is the ten-fold mean on the host’s default path (three seeds at rows 3 and 4). RP2040 energy is a post-registration addition, and row 1 does not fit in its flash. Row 4 on the ESP32-S3 includes ESP-NN 1.3.1’s debug heap check ( Table 10 ). The horizontal axis is logarithmic.
Check
Arm A
Arm B
Arm D
Criterion
Outcome
Arena used ( B ), location
224,476, SRAM
224,476, SRAM
224,476, SRAM
Identical
Met, exact
Model file ( B )
627,464
627,464
627,472
Same size
Missed a
Median latency ( \SIUnitSymbolMicros )
152,790.5
152,790.5 c
152,818.5
Within ±100\SIUnitSymbolMicros
Met
IQR ( \SIUnitSymbolMicros )
12.5
11.0 c
13.0
–
Reported
Outputs equal to own reference
6/6
6/6
6/6
Exact
Met
Outputs on an INT8 limit
0/6
5/6
3/6
–
Reported
Table 14: Deployment cost of the row-3 model for arms A, B and D on the ESP32-S3, against the criteria fixed before the device runs.
Vision models deployed on microcontrollers (MCUs) are quantized to integer-only arithmetic and run in inference-only runtimes that do not carry the machinery backpropagation needs: the standard tool for adapting a model to the distribution shift (sensor noise, blur, lighting) it meets in the field. Existing forward-only test-time adaptation (TTA) methods either run only on server- or edge-GPU-class models (not true microcontroller integer execution), or require the batch-normalization (BN) layers that integer deployment fuses away. We present a forward-only TTA method that operates on deployed, BN-folded, integer-only convolutional networks. The key observation is that fusing BN into the preceding convolution, a mandatory step for integer inference, destroys the statistics that normalization-based adaptation relies on. We restore adaptation by re-normalizing each folded convolution's per-channel output to its clean training statistics, using only forward-pass estimates. The method (i) recovers most of gradient-based TENT's accuracy gain (+20.9 vs. +24.9 points) and matches forward-only BN adaptation, while being the only method that runs on a folded integer-only model; (ii) needs to adapt only 3 of 21 layers (selected without seeing the test corruptions) to recover 93% of the benefit; (iii) survives single-sample streaming with a batch-size-scaled momentum; and (iv) generalizes across three datasets (up to 200 classes) and two architectures. We validate bit-exact int8 convolution execution and deploy on an ESP32-S3, where, measured with a Nordic PPK2 power profiler, the forward-only adaptation (a lightweight fp32 recalibration around the int8 convolutions) costs only 8.3 mJ (6.8% of inference energy) and 21.9 ms on the deployed SIMD-optimized model: forward-only adaptation is cheap on a real microcontroller.
On hardware whose fastest matrix datapath is INT8×INT8, such as the Neural Accelerators of Apple's M5 GPUs, 4-bit weights must become INT8 operands. Existing MXFP4 kernels already map FP4 (E2M1) values to integers by doubling them, but apply every 32-element block scale inside the reduction. We show when no per-block work is needed: a row of MXFP4 weights is exactly one INT8 vector times one power-of-two scale if its block exponents span at most three, so a linear layer becomes a single INT8 GEMM with one rescale per output. Enforcing the condition adds at most 1/16 of a row's largest rounding error to the blocks it changes, and real weights already satisfy it: 97.9-99.97% of rows across seven models, including 99.93% of the 95M rows of a 125B-parameter mixture-of-experts (MoE) model, Qwen3.8-Flash-Next. The resulting spread-constrained MXFP4 files stay standard MXFP4. With integer instead of power-of-two block multipliers, an integer-exact format (16-wide blocks, multipliers 1..10, 4.26 bits) runs as one exact INT8 GEMM and is more accurate than NVFP4 (4.5 bits) with its standard scale rule on five small models, within 0.06 perplexity of NVFP4 with scale search. On Qwen3.8-Flash-Next's routed experts with INT8 activations it is as close to the FP8 model as NVFP4 with scale search (KL divergence 0.0557 vs. 0.0553) and closer than MLX's MXFP4 and affine INT4; with every matrix quantized it solves 94.8% of GSM8K vs. 91.0% for MLX's MXFP4. Quantizing only the 74k routed expert matrices (96% of the parameters) to either of our formats with INT8 activations costs at most 0.2% perplexity. A complete W4A8 system on the M5 produces outputs bit-identical to the quantized reference; prefill is 1.34-1.74x faster than MLX on a dense model, 1.10-1.57x on two MoE models up to 35B parameters and 1.22-1.29x on Qwen3.8-Flash-Next, where shared non-GEMM work bounds the gain.
Shinyoung Kang, Junhyung Ahn, Yujun Kim +1
UT Southwestern Medical Center · Department of Data and Decision Sciences, Emory University · Fuqua School of Business, Duke University +1
Deploying neural networks on microcontrollers is constrained by kilobytes of flash and SRAM, where 1x1 pointwise (PW) mixers often dominate memory even after INT8 quantization across vision, audio, and wearable sensing. We present HYPER-TINYPW, a compression-as-generation approach that replaces most stored PW weights with generated weights: a shared micro-MLP synthesizes PW kernels once at load time from tiny per-layer codes, caches them, and executes them with standard integer operators. This preserves commodity MCU runtimes and adds only a one-off synthesis cost; steady-state latency and energy match INT8 separable CNN baselines. Enforcing a shared latent basis across layers removes cross-layer redundancy, while keeping PW1 in INT8 stabilizes early, morphology-sensitive mixing. We contribute (i) TinyML-faithful packed-byte accounting covering generator, heads/factorization, codes, kept PW1, and backbone; (ii) a unified evaluation with validation-tuned t* and bootstrap confidence intervals; and (iii) a deployability analysis covering integer-only inference and boot versus lazy synthesis. On three ECG benchmarks (Apnea-ECG, PTB-XL, MIT-BIH), HYPER-TINYPW shifts the macro-F1 versus flash Pareto frontier: at about 225 kB it matches a roughly 1.4 MB CNN while being 6.31x smaller (84.15% fewer bytes), retaining at least 95% of large-model macro-F1. Under 32-64 kB budgets it sustains balanced detection where compact baselines degrade. The mechanism applies broadly to other 1D biosignals, on-device speech, and embedded sensing tasks where per-layer redundancy dominates, indicating a wider role for compression-as-generation in resource-constrained ML systems. Beyond ECG, HYPER-TINYPW transfers to TinyML audio: on Speech Commands it reaches 96.2% test accuracy (98.2% best validation), supporting broader applicability to embedded sensing workloads where repeated linear mixers dominate memory.