Large neural networks are increasingly deployed with 1-4 bit weights and 16- or 32-bit activations, yet their dot products still use one multiplier per element. We show that a length-n dot product with k-bit weight codes can be computed with n+2^k additions and at most 2^k decodes and 2^k multiplications. Each weight code is used directly as an address offset that selects one of 2^k accumulators; each activation is added into the selected accumulator; and the accumulated sums are decoded and multiplied only once, at the end, in a fused decode-and-multiply step. The decoded weight vector is never formed. The refactoring is exact for integer arithmetic, works with integer or floating-point operands and with arbitrary, including nonlinear, decoders, and extends to group-wise scales and to matrix and tensor products. The multiplier count falls by n/2^k, while total arithmetic energy improves by a smaller, bounded factor that we quantify with a cost model. We describe a scatter-accumulate processing element (SA-PE) that replaces the MAC unit in a weight-stationary systolic array, with a load/stream/reduce/drain schedule in which a small, time-shareable multiplier runs only 2^k times per tile. Because decoded weights are not stored, local weight memory shrinks by up to w_d/k for w_d-bit decoded weights. An RTL implementation synthesized for FPGA and compared with a matched MAC array shows that an unshared SA-PE is larger than a MAC PE, and that the saving appears when the reduction multiplier is shared across PEs. Software models confirm exact integer equivalence and floating-point error no worse than the conventional pipeline. We do not claim the underlying identity is new, and we discuss related work on weight sharing and lookup-based kernels.
Figures & tables
Figure 1: Dataflow of the conventional and add-before-multiply dot products. In (b) the n -element decoded vector d and the n products are never formed; only 2k bin sums are held.
Conventional
One-hot masked
Direct address offset
Decode / mask generation
n decodes
n masks
0 (offset only)
Coefficient decodes
n
≤2k
≤2k
Multiplications
n
≤2k
≤2k
Accumulator updates
n
n + ≤2k
n + ≤2k
Address-offset ops ( k -bit)
0
0
n
Decoded-weight storage
n values
≤2k coefficients
≤2k coefficients
Table 1: Operation counts for one length- n dot product with k -bit weight codes. Counts are upper bounds if empty bins are skipped.
Figure 2: Arithmetic energy gain G(n,k) from Eq. 5 with Eo=0 . The curves saturate at 1+ρ once n≫2k(1+ρ) .
cycle i
0
1
2
3
4
5
6
7
8
9
bi
.12
.34
.45
.56
.78
.90
.01
.23
.45
.67
code ai
2
1
3
1
0
2
3
2
1
0
update
q2
q1
q3
q1
q0
q2
q3
q2
q1
q0
Table 2: Stream-phase scatter trace and reduce-phase products for the worked example ( n=10 , k=2 ).
Figure 3: Scatter-accumulate processing element. During streaming, the stored code selects a bin and the adder updates it; during the reduce phase the reduction unit multiplies each bin by its decoded coefficient and sums.
Figure 4: Weight-stationary array of SA-PEs. Activation slices are broadcast down columns; the PEs of a row hold codes for the same output element and their partial sums are combined by a row adder chain. (Schematic; array size is arbitrary.)
Figure 5: Phase schedule (not to scale). The load of the next tile overlaps the reduce and drain of the current one; the reduce phase lasts 2k≪L cycles.
Conventional MAC PE
SA-PE
Multiplications per dot product
n
≤2k , independent of n
Additions per dot product
n accumulator updates
n scatter updates + ≤2k reduce updates
Per-cycle streaming operation
1 multiply + 1 accumulate
1 address-selected add
Extra addressing logic
none
k -bit address decode / base+offset
Storage per PE
1 accumulator
2k bins (width = data + ⌈log2n⌉ )
Weight storage per PE
decoded weight, or k -bit + decode
k -bit codes only; decode LUT shared
Table 3: Per-PE resources for an n -length period: conventional MAC systolic PE versus SA-PE.
k
mults ( 2k )
n=256
n=1024
n=4096
1
2
128 ×
512 ×
2048 ×
2
4
64 ×
256 ×
1024 ×
3
8
32 ×
128 ×
512 ×
4
16
16 ×
64 ×
256 ×
Table 4: Multiplication reduction factor n/2k per PE per tile.
k
MA (B)
MB (B)
MN (B)
R=MA/MN
MN/MB
1
2048
132
140
14.6
1.06
2
2048
264
280
7.3
1.06
4
2048
544
608
3.4
1.12
Table 5: Local weight memory per PE ( L=1024 , wd=16 , wb=32 , P=1 ).
Figure 6: Memory reduction R(L) from Eq. 10 for wd=16 , wb=32 , P=1 . The curves approach wd/k=16 , 8 , 4 ; the thin line marks R=1 .
PE
LUT
FF
DSP48
MAC, multiplier in DSP
55
39
1
MAC, multiplier in fabric
819
39
0
SA-PE (RTL), multiplier in DSP
161
134
1
SA-PE (RTL), multiplier in fabric
1193
134
0
SA-PE shared † , DSP engine
122
95
0.06
SA-PE shared † , fabric engine
179
95
0
Table 6: Per-PE FPGA resources ( k=2 , L=64 , wact=16 , Yosys 0.33). † : projection with the reduction multiplier shared by σ=16 PEs.
k
n
Conv. FP32 acc.
Conv. FP16
New, FP16 bins
New, FP32 bins
1
256
1.9×10−8
1.6×10−4
1.1×10−4
2.6×10−9
1
4096
1.7×10−8
1.3×10−4
1.1×10−4
1.1×10−9
2
256
2.2×10−8
2.1×10−4
8.5×10−5
3.7×10−9
2
4096
1.8×10−8
1.5×10−4
7.3×10−5
1.2×10−9
4
256
1.7×10−8
1.6×10−4
7.9×10−5
3.4×10−9
4
4096
1.9×10−8
1.8×10−4
5.8×10−5
1.2×10−9
Table 7: Mean relative error (normalized by ∑i∣dibi∣ ) against an FP64 reference.