Noise-shaped Sampling and Annealing for Scalable On-the-edge Training of Binary and Ternary AI Models

Zihao Chen; Zhili Xiao; Rui-Jie Zhu; Yipeng Wang; Jason Eshraghian; Shantanu Chakrabartty

Figure 1:

\Sigma\Delta

-Sampler for training discrete neural networks. (a) A feedforward binary neural network with weights constrained to

\mathbf{W}\in\{-1,+1\}^{D}

. (b) Equivalence between discrete parameter neural networks and a hierarchical high-order Ising machine under stationary data distribution

(\mathbf{x},\mathbf{y})

. (c) Conceptual loss/energy profiles for increasing Ising order, where low-order expansions give a smooth landscape and higher-order couplings add frustration, many local minima, and high-frequency components in the gradient space. (d) A general sampler architecture for training discrete weight neural networks. (e) The effect of noise-shaping in spectral domain, where the white-noise is pushed out of the low-frequency adaptation band towards high-frequency. (f) Histogram showing noise-shaping of the Ising decomposition, resulting in the smoother optimization contour that benefits generalization. (g) Rugged non-convex optimization landscape due to parameter discretizations, where traditional simulated annealing optimizers are slow and do not perform well. (h) Optimization trajectory of the

\Sigma\Delta

-Sampler on the smoothened landscape is more stable and computationally scalable. (i) Parameter update rate maps for a

\Sigma\Delta

-Sampler at different stages of temperature annealing, early(

T_{0}

), mid(

T_{n}

), and late(

T_{N}

), while training a single layer, showing adaptive regularization on the model parameters tracks the smooth optimization contour.

Figure 2: Spectral and dynamic analysis of

\Sigma\Delta

-Sampler on a planted Quadratic Unconstrained Binary Optimization (QUBO) problem. (a) Block diagram of the

\Sigma\Delta

-Sampler update rule. (b) Optimization plot showing convergence towards the planted global optimum

\mathbf{W}^{*}

over

1{,}000

steps for four different optimizers: SF (single-flip SA), PF and PF-20 (parallel-flip SA at

100\%

and

20\%

flip rates respectively), and

\Sigma\Delta

-Sampler. (c) Principal component visualization of the optimization trajectories, with (d) showing the convergence on a shorter time-scale. (e), (f) Temporally windowed normalized parameter update rate of PF SA and

\Sigma\Delta

-Sampler showing smoothing of the parameter update trajectories due to noise shaping. (g) Power spectra of the quantization errors (difference in the decision signal vs. update signals) in different optimizers, showing the effect of noise-shaping. (h) Linear extrapolation of average

|g^{n}|

and

|\omega^{n}|

against overall parameter update rate during the optimization process.

Figure 3:

\Sigma\Delta

-Sampler training dynamics on CIFAR-10 and the spectral signature of

\Sigma\Delta

noise-shaping. (a) Train (dashed) and test (solid) accuracy of two ternary networks trained end-to-end with the

\Sigma\Delta

-Sampler pipeline on CIFAR-10: VGG-7 and VGG-8. (b) Block diagram of the used convolution model. (c) Per-epoch probability density function (pdf) of per-parameter inter-flip intervals where color denotes training stage from early (orange) to late (blue) epochs; inset shows the corresponding cumulative distribution function (cdf), where the dashed line marks the median. (d) Normalized empirical power spectral density of the per-parameter

\Sigma\Delta

integrator state

\omega^{n}_{j}

estimated over a randomly sampled cohort of weights, compared against a

1/f^{2}

reference spectrum (marked in red dashed line) overlaid, with (e) showing the low-frequency zoom of the same PSD. (f) Validation accuracy for

\Sigma\Delta

-Sampler (solid trajectory) and full-precision counterparts (dashed trajectory) in continual-learning configuration, where the different color codes represent different offline-class replay fractions. (g) Per-phase probability density of per-parameter inter-flip intervals across the offline phase (orange to blue) and the online phase (red). (h) Per-parameter reactivation ratio plotted against the phase-1 update rate.

Method	Weight precision	Accuracy (%)	Time to solution (h)
FPWN (fp baseline)	fp32	92.88	0.38
TWN FP classifier	ternary $\{-1,0,+1\}$	91.56*	10.32
TWN discrete	ternary $\{-1,0,+1\}$	91.47	10.85
$\Sigma\Delta$ -Sampler FP classifier	ternary $\{-1,0,+1\}$	92.16	0.69
$\Sigma\Delta$ -Sampler discrete	ternary $\{-1,0,+1\}$	92.20	0.73
$\Sigma\Delta$ -Sampler, $\sim$ 5M parameters	ternary $\{-1,0,+1\}$	91.88	0.67

Table 1: Final test accuracy and time to presented solution on CIFAR-10 for full-precision baseline model, Ternary Weight Networks (TWN) model Liu et al. (2023) and the proposed approach

\Sigma\Delta

-Sampler on the VGG CNN architecture (13M). *The accuracy reported in the original TWN work is

92.56\%

Model	Replay	P1 Off (%)	P2 Off (start $\rightarrow$ end, %)	P2 On (%)
FP	$20\%$	$92.83$	$72.40\rightarrow 83.85$	$98.35$
FP	$50\%$	$92.99$	$84.94\rightarrow 88.23$	$97.80$
$\Sigma\Delta$ -Sampler	$20\%$	$92.15$	$64.84\rightarrow 77.51$	$97.60$
$\Sigma\Delta$ -Sampler	$50\%$	$92.21$	$78.36\rightarrow 84.36$	$97.00$

Table 2: CIFAR-10 validation accuracy for

\Sigma\Delta

-Sampler and the full-precision baseline on a 12.98M VGG-8 model with various training configuration and stages.

Method	Top-5 (%)	Top-1 (%)
FP	$86.76$	$65.40$
TWN	$84.20$	$61.80$
$\Sigma\Delta$ -Sampler	$83.81$	$61.65$

Table 3: ImageNet-1000 validation accuracy on a ResNet-18 backbone, comparing

\Sigma\Delta

-Sampler against a ternary-weight-network (TWN) baseline with a full-precision classification layer and the full-precision (FP) reference.

Figure 4: Scaling up

\Sigma\Delta

-Sampler training on larger dataset (ImageNet1000) and LLMs (a) Training and test accuracy of ResNet18 on Imagenet1000 with

\Sigma\Delta

-Sampler as optimizer. (b) Training loss of various sized MatMul-Free LLMs on FineWeb-Edu with 532.5K tokens per training step. (c) Continued pre-train on the same dataset from the same quantized checkpoint with STE and

\Sigma\Delta

-Sampler optimizers respectively. (d) Per-step DRAM transactions due to model weight updates for different models, where transactions become sparser for

\Sigma\Delta

-Sampler as training progresses.

Model	Discrete params	Mean loss	Tail-100 loss
MMFree-370M	$308.3$ M	$6.8312$	$6.4544$
MMFree-1.3B	$1.233$ B	$6.3630$	$5.9684$
MMFree-2.7B	$2.538$ B	$6.4871$	$5.9846$

Table 4: Numerical results of MatMul-Free language-model pre-training on FineWeb-Edu for all sizes. Tail-100 is the mean training loss over the final

100

steps.

(a) Component	FP / STE	$\Sigma\Delta$ -Sampler discrete	$\Sigma\Delta$ -Sampler continuous
Weight or ternary code	$4$ / $1(4)$	$2$	$4$
Gradient, or decision signal $\omega^{n}$	$4$	$4$	$4$
AdamW first moment $m_{1}$	$4$	$0$	$4$
AdamW second moment $v$	$4$	$0$	$4$
Stochastic draw buffer	—	transient	—
Total (B/param)	$\mathbf{16/17}$	$\mathbf{6}$	$\mathbf{16}$

Table 5: Persistent training-state accounting. (a) Per-parameter cost of each stored component. (b) Analytical aggregated memory allocation across models. Activation memory is excluded throughout, since it is identical across optimizers under matched architecture and batch size.

Model	$H$	Layers	Heads	Intermediate	Discrete	Fraction
MMFree-370M	1024	24	16	2816	308.3 M	82.4%
MMFree-1.3B	2048	24	32	5632	1.23 B	90.3%
MMFree-2.7B	2560	32	40	6912	2.54 B	93.9%

Table 6: MatMul-Free LLM configurations

Symbol	Identifier	Default	Role
$T_{0}$	temp_init	2.0	Initial temperature
$\tau$	temp_decay	5.0	Cooling time-constant
$\beta_{\mathrm{ema}}$	ema_beta_log	0.95	EMA decay for log-gradient statistic
$\mu_{\min}$	thld_min	0.0	Threshold floor
$\gamma$	thld_gain	1.0	Threshold gain on $\bar{G}^{n}$
$\tau_{\mathrm{thld}}$	thld_tau	2000	AGC ramp time-constant (steps)

Table S1: Default hyperparameters of the

\Sigma\Delta

-Sampler per-layer dynamics. Identifiers in monospace are the field names of the Model_HParams dataclass.

Figure S1: Computational graph of a single MatMul-free language model layer.

Model	$H$	Layers	Heads	Intermediate	Discrete params	$\Sigma\Delta$ layers
MMFree-370M	$1024$	$24$	$16$	$2816$	$308{,}281{,}344$	$144$
MMFree-1.3B	$2048$	$24$	$32$	$5632$	$1{,}233{,}125{,}376$	$144$
MMFree-2.7B	$2560$	$32$	$40$	$6912$	$2{,}537{,}553{,}920$	$192$
12L768 (S5.3)	$768$	$12$	$12$	$2048$	$109{,}510{,}656$	$72$

Table S2: Language-model configurations. Vocabulary is

32{,}000

for all sizes, embedding and language-model head are untied, and the SwiGLU intermediate width is

256\lceil(8H/3)/256\rceil

for hidden width

H

. Discrete params counts the ternary parameters under

\Sigma\Delta

-Sampler and layers counts the number of

\Sigma\Delta

-Sampler layers.

Setting	Value	Meaning
mode	direct ternary	$W=W^{\mathrm{on}}-W^{\mathrm{off}}$ , components in $\{0,1\}$
$T_{0}$	$2.0$	initial temperature
$\tau$	$0.5$	$T^{n}=T_{0}/(T_{0}+n/\tau)$
$\Sigma\Delta$ feedback	enabled	second-order temperature scaling, $(T^{n})^{2}$
single accumulator	enabled	difference mode $(\omega^{\mathrm{on}}-\omega^{\mathrm{off}})/2$
pure-integration warmup	$20$ steps	gradient integrated before feedback engages

Table S3:

\Sigma\Delta

-Sampler configuration for the language-model experiments, identical at all three pre-training scales.

Model	Cap	Mean loss	Tail-100
MMFree-370M	uncapped	$7.2055$	$6.7011$
MMFree-370M	$10^{-1}$	$6.7278$	$\mathbf{6.1271}$
MMFree-370M	$10^{-2}$	$6.6890$	$6.1819$
MMFree-370M	$10^{-3}$	$6.8312$	$6.4544$
MMFree-1.3B	uncapped	$7.0237$	$6.5612$
MMFree-1.3B	$10^{-1}$	$6.7439$	$6.1921$

Table S4: Update-rate cap sweep for from-scratch pre-training. Tail-100 is the mean training loss over the final

100

steps. Best tail-100 per size in bold.

Update rule	Eval loss at step 0	Final eval loss	$\Delta$ vs STE
BitNet STE	$3.1721$	$3.1644$	—
$\Sigma\Delta$ -Sampler, cap $10^{-5}$	$3.1951$	$3.0800$	$-0.084$
$\Sigma\Delta$ -Sampler, cap $10^{-4}$	$3.1950$	$\mathbf{3.0241}$	$\mathbf{-0.140}$
$\Sigma\Delta$ -Sampler, cap $10^{-3}$	$3.1951$	$3.0913$	$-0.073$
$\Sigma\Delta$ -Sampler, cap $10^{-1}$	$3.1950$	$7.1105$	$+3.946$

Table S5: Continued pre-training from a common quantized checkpoint,

100

steps, held-out evaluation loss. All arms share the same data, schedule, and continuous-parameter optimizer, and differ only in the ternary weight update rule. The initial loss difference is attributed to initialization of the scaling factors for both arms.

Cap	Mean transactions/step	Reduction	Final loss
uncapped	$186{,}493$	$103\times$	$6.6555$
$10^{-1}$	$86{,}727$	$222\times$	$6.0654$
$10^{-2}$	$27{,}487$	$701\times$	$6.1407$
$10^{-3}$	$4{,}209$	$4{,}578\times$	$6.4093$

Table S6: Weight-update traffic and final loss against the update-rate cap for MMFree-370M. The full-precision reference is

19{,}267{,}584

transactions per step.

Varied parameter	Best val	Final val	Last-10 mean
thld_tau $=5000$	$\mathbf{92.20}$	$91.54$	$91.76$
sd_accum_epochs $=5$	$92.03$	$91.74$	$91.52$
temp_decay $=20$	$92.03$	$91.86$	$91.76$
thld_target_pruned_init $=0.3$	$92.01$	$91.68$	$91.64$
wd $=0$	$91.99$	$91.78$	$91.70$
ema_beta_log $=0.9$	$91.94$	$91.31$	$91.35$

Table S7:

200

-epoch reruns of the ten configurations promoted from the screen, ordered by best validation accuracy. Each row differs from the baseline only in the parameter shown. Last-10 is the mean over the final ten epochs.

Group	Setting	Best val	Final val
Accumulation window	sd_accum_epochs $=5$ $\ast$	$91.24$	$90.57$
	sd_accum_epochs $=10$	$91.18$	$90.91$
	sd_accum_epochs $=20$ $\ast$	$91.36$	$90.99$
Threshold gain	$0.5$ , target $0.5$ $\ast$	$91.32$	$91.14$
$\times$ AGC target	$0.5$ , target $0.7$	$91.07$	$91.05$
	$0.5$ , target $0.9$ $\ast$	$\mathbf{91.44}$	$91.03$

Table S8:

100

-epoch screen over all twenty-four configurations, grouped by the parameter varied. The row marked

\dagger

is the baseline configuration. Rows marked

\ast

were promoted to the

200

-epoch reruns of Table S7 .

Figure S2: GPU power benchmarking of three optimizers on VGG-7 / CIFAR-10. (a) Instantaneous board power against wall-clock time. Full precision and

\Sigma\Delta

-Sampler draw comparable power at comparable device utilization, whereas TWN sustains only

3.2\%

of the above-idle power and

2.6\%

of the step throughput of the full-precision baseline. (b) Energy per epoch. TWN is an order of magnitude above the other two, while

\Sigma\Delta

-Sampler incurs a

1.8\times

overhead relative to full precision that we attribute to unfused kernels rather than to the update rule.

Optimizer	Mean power	Above idle	Util.	Steps / $600$ s	s/epoch	J/epoch	J/step
Full precision	$166.6$ W	$116.8$ W	$42.3\%$	$30{,}133$	$9.96$	$1{,}659$	$3.32$
TWN	$53.5$ W	$3.7$ W	$34.3\%$	$797$	$376.5$	$20{,}145$	$40.29$
$\Sigma\Delta$ -Sampler	$137.9$ W	$88.1$ W	$52.7\%$	$13{,}776$	$21.78$	$3{,}004$	$6.01$

Table S9: Measured power, throughput, and derived energy for the three optimizers. Above-idle power subtracts the

49.8

W idle draw of the device. Energy figures use total board power.

Figure S3: Sensitivity of optimizers to residual connectivity on CIFAR-10. (a) Final training accuracy of STE and

\Sigma\Delta

-Sampler on CIFAR-style ResNets of increasing depth, with and without residual connections. (b) Peak validation accuracy of ResNet-56 against the number of blocks retaining a skip, counted from the input. (c) Test accuracy of the same architecture under higher-order residual connections.

Noise-shaped Sampling and Annealing for Scalable On-the-edge Training of Binary and Ternary AI Models

Abstract

Figures & tables

Binary Neural Networks

CAT-Q: Cost-efficient and Accurate Ternary Quantization for LLMs

Low-Rank Ternary Adaptation for Fine-Tuning Transformers

RoleQ: Role-Aligned Scales for Converting Language Models to Ternary

Abstract

Binary Neural Networks

Explore similar work

CAT-Q: Cost-efficient and Accurate Ternary Quantization for LLMs

Low-Rank Ternary Adaptation for Fine-Tuning Transformers

RoleQ: Role-Aligned Scales for Converting Language Models to Ternary