Hardwired Distributed Processing Units for Inference: Neural Decisions at the Sensor that Free the Host

Raj Mali

cs.ARNov 4, 2026

Hardwired Distributed Processing Units for Inference: Neural Decisions at the Sensor that Free the Host

Authors: Raj Mali

Organizations: Independent Researcher

Abstract

We show that a hardwired distributed processing unit (HDPU) can take a neural decision at the sensor and so remove the inference from the host. Built for zebra-crossing detection on a low-cost Artix-7 FPGA, the unit decides on every camera frame in a fixed 2,269,477 clock cycles (34.04 ms depends upon Fpga clk ), inside the 37.49-ms frame period, and on the board it returns the same scores and decisions as its bit-exact software model on all 639 test images. The whole detection path, from camera pixels to decision, takes 1,814 LUTs, with no processor, no operating system and no full-resolution frame store. When the unit answers instead of reporting "not sure", 95.7% of its answers on street photographs are correct, and a hardware contrast stage lifts accuracy on camera-like dashcam frames from 47.7% to 78.2%. The host is spared all of the inference: 8.13 ms of a core in every frame period, where a full-precision ResNet-18 would need more than a frame period even on 16 threads. It also receives 16 bytes per frame instead of 614,400, 38,400 times less. The unit is an 80x60-pixel pipeline feeding a four-layer binarized CNN with 93,408 one-bit weights, verified bit-exactly from a Python model through RTL simulation to the board. Earlier HDPUs did pre-processing (edges, corners, steering); this one shows that the decision itself can move to the sensor.

Figures & tables

Fig. 1: Two ways to get a decision from a camera. (a) The host-centred routine moves every frame to the host, which fetches instructions and weights and runs the network. (b) The inference HDPU reduces the frame as it arrives and takes the decision at the sensor; the host receives 16 bytes and does no work.

Panel of Figure 2 — Fig. 2: The detection path of the inference HDPU (blue) listens to the camera pins in parallel with the picture path (grey) and needs no processor. The network’s weights and thresholds (green) sit in on-chip memory beside the engine. The host, if present, receives a decision of 16 bytes per frame.

Fig. 3: A frame captured from the board’s OV7670 (a), its $80\times 60$ network input, which spans only a narrow range of gray levels (b), and the same input after the hardware auto-contrast stage (c).

Layer	Operation	Output	Clocks	Share
L0	conv $3\times 3$ , 8-bit input	$40\times 30\times 32$	1,382,400	60.9 %
L1	conv $3\times 3$ , XNOR	$20\times 15\times 64$	691,200	30.5 %
L2	conv $3\times 3$ , XNOR	$10\times 7\times 64$	161,280	7.1 %
L3	conv $3\times 3$ , XNOR	$5\times 3\times 64$	34,560	1.5 %
L4	fully connected	score	15	0.0 %
Total, with 22 control clocks			2,269,477

TABLE I: Layers of the deployed network and the clock cycles each takes in the engine (Eq. ( 1 ))

Level	Compared with the model	Result
Simulation	camera pins to the $80\times 60$ image	3/3 frames
Simulation	engine: score, decision, every feature-map word	639/639 images
Simulation	serial link, contrast, engine, decision (real OV7670 frame)	20/20 checks
Board	self-test, golden vectors over the link	8/8 checks
Board	T1, score and decision	524/524 images
Board	T2, score and decision	115/115 images

TABLE II: Verification chain: every level compared with the Python integer model

Result	Value	Source
Decision time, every frame, any image	34.04 ms (2,269,477 clocks)	simulation
Frames skipped on the live camera	0	hardware
Score and decision equal to the model	639 of 639 images	hardware
Accuracy, T1 / T2	79.39 / 79.13 %	hardware
Correct when answering, T1 / T2	90.2 / 95.7 %	hardware
Camera-like frames, T1: without $\rightarrow$ with contrast stage	47.7 % $\rightarrow$ 78.2 %	model

TABLE III: Headline results of the inference HDPU

Check	Result
Self-test	8 of 8 checks pass
Camera frames committed	27.5 frames/s, none rejected
Decisions on the live camera	27/s, no frame skipped
T1, dashcam frames, through the FPGA	524/524 equal to the model, 79.39 %
T2, street photos, through the FPGA	115/115 equal to the model, 79.13 %

TABLE IV: Measured on the EDGE Artix-7 board

Quantity	Value	Source
Camera frame period	37.49 ms	model
Decision time	34.04 ms	simulation
Margin per frame	3.44 ms	derived
Frame end to decision	about 34.2 ms	derived
Engine busy	90.8 % of the frame	derived
Highest decision rate	29.4/s (camera: 26.68/s)	derived

TABLE V: Frame budget of the inference HDPU

Fig. 4: Three consecutive frame periods to scale. The HDPU engine (blue) finishes each decision inside the next frame period, never missing a frame. The two host rows show how long the same task takes on one laptop core (median, PyTorch, Section IV ), not a measured sequence: the binarized network takes 8.13 ms, and the full-precision ResNet-18 takes longer than a frame period.

	T1 (dashcam)	T2 (street)
Images (with a crossing)	524 (292)	115 (67)
True / false detections	197 / 13	56 / 13
Missed / true rejections	95 / 219	11 / 35
Precision / recall (%)	93.8 / 67.5	81.2 / 83.6
F1 / accuracy (%)	78.5 / 79.4	82.4 / 79.1
Answered outside the band (%)	50.8	40.0

TABLE VI: Decisions of the unit on the held-out test sets

Fig. 5: Margin (score minus the output threshold of 477) of every test image, from the bit-exact model; the board returned the same scores for all 639 images. Crossings above and non-crossings below the axis; grey: the not-sure band.

Fig. 6: Share of correct answers against the share of images answered, most confident first, through the camera path (model). The dashed line is the full-precision ResNet-18 on all of T2.

Share answered (%)	30	40	50	60	80	100
T1 (dashcam), correct (%)	92.4	93.8	91.6	90.4	83.5	77.5
T2 (street), correct (%)	97.1	95.7	93.1	88.4	81.5	73.0

TABLE VII: Accuracy of the most confident answers through the camera path (model)

Fig. 7: Accuracy under nine camera effects, mean of T1 and T2 (model), with the network input as the camera gives it and with the hardware contrast stage.

Block	LUT	FF	BRAM36	DSP
Inference engine (BCNN core)	744	321	10	0
Pre-processing (gray, $8\times 8$ means)	211	257	0	0
Auto-contrast, loader, decision	615	593	0	1
Frame banks, serial link	244	132	7	0
Camera, SDRAM, VGA picture path	1,069	769	1.5	0
Overlay, LCD, buzzer, clocks	490	658	0.5	0

TABLE VIII: Resources on the XC7A35T-1 (Vivado 2024.2, routed). ∗ From the hierarchical report; LUT sharing makes the sum of the rows slightly larger than the total.

Fig. 8: (a) LUTs by block of the routed design. (b) Vivado’s vectorless power estimate by block; the inference engine (blue) is a small part of the total.

Host task per frame	1 thread	16 threads	Share of frame, 1 thread
Binarized network	8.13 ms	10.47 ms	21.7 %
ResNet-18, full precision	56.39 ms	42.49 ms	150 %
Inference HDPU	no host time		0 %

TABLE IX: Host time for the same inference (PyTorch, batch 1, median of 200 runs; uncontrolled), against the inference HDPU

Message to the host	Bytes	Smaller than a frame by
Raw RGB565 frame	614,400	1 $\times$
Snapshot: the $80\times 60$ image the network saw, with its decision	4,826	127 $\times$
Status reply	42	14,629 $\times$
Decision (state, score, margin, average, counters)	16	38,400 $\times$

TABLE X: Messages from the unit to the host, per frame (derived from the serial protocol)

Fig. 9: Bytes per frame that the host would receive, on a log scale.

System	Platform	What it needs around it	Rate	Reported result
ZebraRecognizer [ 8 ]	iPhone 5S	phone, operating system	about 25 fps	precision 1.0, recall 0.93
LYTNet [ 9 ]	iPhone 7	phone, operating system	20 fps	94 % †
CDNet [ 10 ]	Jetson Nano	GPU module, operating system	33.1 fps	F1 above 94 %
FINN CNV [ 14 ]	ZC706 (Zynq-7045)	Zynq SoC board	21,906 images/s	80.1 % (CIFAR-10)
This work	XC7A35T and a camera	nothing else: no processor, no OS	26.68 fps (every frame)	79.1 % (T2); 95.7 % when answering

TABLE XI: Positioning against published systems, as reported by their authors. Accuracies refer to each system’s own data and task. † Reported for the classification of pedestrian traffic lights.

FPGA-Based Neural Network Acceleration

Field-programmable gate array (FPGA)-based neural network acceleration maps models to reconfigurable hardware for fast, energy-efficient inference.

See all 95 papers

Evaluating Four FPGA-accelerated Space Use Cases based on Neural Network Algorithms for On-board Inference
Mar 14, 2026Pedro Antunes, Muhammad Ihsan Al Hafiz, Jonah Ekelund +4FPGA-Based Neural Network Acceleration Efficient Neural Network Inference
Department of Computer Science, Kungliga Tekniska högskolan (KTH), Stockholm, Sweden · Plasma-astrophysics, KU Leuven, Leuven, Belgium
DPU or GPU for Accelerating Neural Networks Inference -- Why not both? Split CNN Inference
Apr 30, 2026Ali Emre Oztas, Mahir Demir, James Garside +1AI Accelerator Inference GPU Acceleration
The University of Manchester
FALCON: FPGA-Accelerated Latency-Certifiable Occupancy Networks for Camera-Based Autonomous Driving
Oct 15, 2026Haohua Que, Mingkai Liu, Handong YaoAutonomous Driving Perception FPGA-Based Neural Network Acceleration
College of Engineering, University of Georgia, Athens, Georgia, USA

Mar 14, 2026J/K move · Enter open · S save

Evaluating Four FPGA-accelerated Space Use Cases based on Neural Network Algorithms for On-board Inference

Pedro Antunes, Muhammad Ihsan Al Hafiz, Jonah Ekelund, Ekaterina Dineva+3

Department of Computer Science, Kungliga Tekniska högskolan (KTH), Stockholm, Sweden · Plasma-astrophysics, KU Leuven, Leuven, Belgium

FPGA-Based Neural Network Acceleration Efficient Neural Network Inference On-Device Inference

Open paper AskPDF arXiv

Space missions increasingly deploy high-fidelity sensors that produce data volumes exceeding onboard buffering and downlink capacity. This work evaluates FPGA acceleration of neural networks (NNs) across four space use cases on the AMD ZCU104 board. We use Vitis AI (AMD DPU) and Vitis HLS to implement inference, quantify throughput and energy, and expose toolchain and architectural constraints relevant to deployment. Vitis AI achieves up to 34.16

\times

higher inference rate than the embedded ARM CPU baseline, while custom HLS designs reach up to 5.4

\times

speedup and add support for operators (e.g., sigmoids, 3D layers) absent in the DPU. For these implementations, measured MPSoC inference power spans 1.5-6.75 W, reducing energy per inference versus CPU execution in all use cases. These results show that NN FPGA acceleration can enable onboard filtering, compression, and event detection, easing downlink pressure in future missions.

Abstract

FPGA-Based Neural Network Acceleration

Explore similar work

Evaluating Four FPGA-accelerated Space Use Cases based on Neural Network Algorithms for On-board Inference

DPU or GPU for Accelerating Neural Networks Inference -- Why not both? Split CNN Inference

FALCON: FPGA-Accelerated Latency-Certifiable Occupancy Networks for Camera-Based Autonomous Driving