Analog in-memory computing (AIMC) offers an alternative for model training by executing matrix operations directly where weights are stored. However, scaling AIMC to train modern deep models remains an open challenge due to severe hardware non-idealities, including physical weights with finite dynamic range and write granularity, analog-digital converters with finite resolution, and noisy and asymmetric updates. Guided by the insight that gradient accumulation is sensitive to precision and rounding errors, we adopt a mixed-precision training paradigm: executing forward and backward matrix multiplications in the analog domain while computing weight gradients in the digital domain. To enable scalable training, we present a holistic system-algorithm co-design that co-optimizes weight mapping to ensure well-conditioned physical and logical weight profiles, couples a preconditioned optimizer with threshold-triggered open-loop pulsing to stabilize training trajectories, and aligns converter dynamic ranges to suppress quantization errors. Evaluated via hardware-calibrated architectural simulations calibrated with electrochemical RAM measurements, our framework scales Transformer training up to 123M parameters with validation loss scaling as L∝N−0.231, where N is the parameter count, comparable to L∝N−0.238 for digital training.
Figures & tables
Figure 1: Scaling landscape of analog training, where each point is one work. Labels identify representative work.
Figure 2: Comparison of training partitions. Left: conventional digital mixed-precision training stores weights and executes the optimizer digitally, using reduced precision for selected arithmetic ( Micikevicius et al., 2018 ) . Middle: prior analog mixed-precision training ( Nandakumar et al., 2020 ) assigns MVMs to a DAC–crossbar–ADC path and uses a digital buffer to accumulate an SGD update before pulsed programming. Right: On top of the conventional analog mixed-precision design, our proposed architecture couples logical-to-conductance mapping with DAC/ADC range alignment (bound management, BM) at the MVM boundary, allowing modern logical-space optimizers such as Adam to stabilize the training process.
Figure 3: (a) Final validation loss of digital Shakespeare Transformer training with FP16 or FP8 weight, activation, or W-grad representations. (b) Cancellation factor κ=∑n∣gn∣/∣∑ngn∣ of six projection types. (c) Relative gradient error versus cancellation strength when the accumulator is rounded after every addition under FP32, FP16, and FP8. (d) Single-step linear-layer update error at total batch size 128: green bars use mixed-precision (MP) digital-residual transfers at the indicated micro-batch sizes (#mbs), while the red bar uses direct rank updates. Bars and curves show four-seed means; whiskers and shaded bands span the observed seed min–max range.
Figure 4: Initialization maps and analog Transformer training on the Shakespeare dataset. (a, b) Signed conductance-deviation and logical-weight distributions for the fixed-logical-range control ( Nandakumar et al., 2020 ) , the native-AbsMax control ( Rasch et al., 2020 ) , and our proposed fixed-scale mapping. (c) Final validation loss across model depth for the same mapping-based controls.
Figure 5: Reconstruction distortion under Gaussian and Laplacian distributions. (a) and (b) show the effect of quantization precision. The gray bands mark precisions at which the max-aligned rail is within 10% of the optimal distortion. (c) and (d) show the effect of width.
Figure 6: Scaling and training trajectories for the five completed OpenWebText schedules following the Chinchilla rule, comparing matched digital and analog training on Li-ECRAM tiles. (a–c) Observed endpoints and preliminary power-law extrapolations fit L=AX−α against parameter count N , processed tokens, and the full-model algorithmic compute proxy, respectively. (d–e) Raw trajectories versus processed tokens and the shared compute proxy show the sample-efficiency and compute-normalized comparisons, respectively.
Scale
Params.
MpSGD
MpAdam
+CA
+init (ours)
Digital Adam
Tiny
6.8M
10.7718
10.6035
6.8110
6.6075
6.3451
Small
16.0M
10.8545
10.5566
5.9143
5.6245
5.3033
Medium
29.9M
10.8293
10.7397
5.2073
4.8696
4.5499
Table 1: Ablation study from MpSGD to our recipe.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Work
Evidence
Task
Model
#analog wt.
Dataset/task
#tokens
Li et al. (2021)
H
F
Logic-gate net
9
Truth tables
8
Ando et al. (2025)
H
T
Binary classifier
16
MNIST
5 k
Chen et al. (2023a)
H
F
Bilayer MLP
60
Mushrooms
81 k
Ning et al. (2025)
H
F
Sparse NN
900
EMNIST L
1.0 k
Wang et al. (2019)
H
F
CNN/RNN
1.0 k
MNIST
240 k
Yao et al. (2020)
H
F
CNN
2.8 k
MNIST
240 k
Appendix
Table 2: Representative on-chip AIMC training studies. Evidence is H when the machine-learning task is executed with physical hardware and S for simulation only. Task is T for transfer learning from a pretrained model and F for training from scratch. Device characterization used to calibrate a simulation does not by itself qualify as H evidence. Studies reporting both forms of task-level evidence occupy separate H and S rows. The table includes only experiments in which all learned matrix-multiplication and convolution weights of the evaluated network are analog-mapped; experiments that retain any such network layers digitally and analog-map only selected layers are excluded. #analog wt. counts distinct logical weights mapped to analog arrays for the stated workload, whether trainable or fixed; digitally held non-matrix parameters, unused capacity, differential-device multiplicity, auxiliary/reference arrays, and replicated mappings are excluded. “Dataset/task” gives the largest qualifying machine-learning workload with a determinable analog-mapped weight count. A superscript ∗ after a work indicates that its paper also reports a larger-scale experiment that leaves one or more learned linear layers digital; it is excluded to preserve a fair full-network analog-mapping comparison. EN denotes an equilibrium net.
Figure 7: Reduction of bound-management MVM retries through output-rail alignment.
Model
# blocks
Width
# heads
# parameters
Updates
Final validation step
Tiny
2
128
4
6.83M
278
273
Small
4
256
4
16.01M
652
640
Medium
6
384
6
29.92M
1,218
1,200
Large
8
512
8
50.91M
2,072
2,060
XLarge
12
768
12
123.55M
5,028
5,020
Appendix
Table 3: Decoder-only GPT model specifications and training schedules for the scaling-law simulations.
Task accuracy ↑
Avg ↑
Model
HSw
PIQA
ARCe
ARCc
OBQA
Wino
CSQA
COPA
SciQ
WSC
Digital Tiny
29.7
57.0
29.7
23.4
21.9
47.7
18.8
53.0
35.9
36.5
35.4
Digital Small
28.9
61.7
33.6
22.7
17.2
50.0
21.1
49.0
43.0
59.6
38.7
Digital Medium
27.3
58.6
33.6
18.8
21.9
49.2
27.3
52.0
48.4
60.6
39.8
Digital Large
26.6
60.2
33.6
21.1
18.0
50.8
25.0
54.0
39.8
63.5
39.3
Digital Xlarge
28.9
64.1
35.9
21.1
21.1
50.8
21.9
59.0
46.1
41.3
39.0
Appendix
Table 4: Zero-shot commonsense and multiple-choice reasoning screening accuracy (%, length-normalized conditional likelihood). Digital AdamW checkpoints appear first, followed by separately trained analog checkpoints. Each task uses 128 fixed-seed labelled examples where available, and otherwise its complete labelled evaluation split (COPA: 100 and WSC: 104 examples). HSw: HellaSwag; ARCe/ARCc: ARC-Easy/ARC-Challenge; OBQA: OpenBookQA; Wino: WinoGrande; CSQA: CommonsenseQA. Avg is the unweighted average across the ten displayed tasks.
Task accuracy ↑
Avg ↑
Model
AG
Bool
CB
CoLA
MNLI
MRPC
QNLI
QQP
RTE
SST-2
WiC
WNLI
Digital Tiny
18.8
39.8
41.1
46.1
35.9
39.1
47.7
51.6
52.3
58.6
49.2
56.3
44.7
Digital Small
21.1
42.2
41.1
43.0
38.3
35.9
46.1
39.8
52.3
57.0
48.4
62.0
43.9
Digital Medium
21.9
40.6
41.1
41.4
38.3
32.8
56.2
68.0
51.6
64.8
46.1
59.2
46.8
Digital Large
25.0
40.6
41.1
40.6
40.6
32.8
47.7
65.6
51.6
60.9
46.1
56.3
45.7
Digital Xlarge
22.7
56.2
41.1
39.1
38.3
33.6
58.6
60.2
52.3
57.0
46.1
63.4
47.4
Appendix
Table 5: Complementary zero-shot language-understanding screening accuracy (%, length-normalized conditional likelihood). Digital AdamW checkpoints appear first, followed by separately trained analog checkpoints. Each task uses 128 fixed-seed labelled examples where available, and otherwise its complete labelled evaluation split (CB: 56 and WNLI: 71 examples). Avg is the unweighted average across the 12 displayed tasks.
Preset device
dwmin
up_down
wmax
wmin
γup
γdown
# states
ReRamES Gong et al. (2022)
0.00135
0.259359
1
-1
5
5
1481.5
ReRamSB Gong et al. (2022)
0.002
0
1.25
-0.75
–
–
1000
Capacitor Li et al. (2018b)
0.005
0
1
-1
0.05
0.05
400
Li-ECRAM Tang et al. (2018)
0.002
0
1.1724
-0.8276
0.1153
0.5085
1000
MO-ECRAM Kim et al. (2019b)
0.00028214
0
1.1714
-0.8286
0.4152
0.7342
7088.7
PCM Nandakumar et al. (2019)
0.01
0
2
0
2.5
2.5
200
Appendix
Table 6: AIHWKit SingleRPUConfig presets fitted to measured physical devices or arrays: primary update and conductance parameters. # states denotes the nominal ratio (wmax−wmin)/dwmin . The citation in each row identifies the source measurement.
Preset device
dwmin,d2d
up_downd2d
wmax,d2d
wmin,d2d
γup,d2d
γdown,d2d
dwmin,std
write_noisestd
ReRamES Gong et al. (2022)
0.2
0.05
0.3
0.3
–
–
5
75
ReRamSB Gong et al. (2022)
0.3
0.01
0.24
0.4
–
–
3.75
56
Capacitor Li et al. (2018b)
0.1
0.06
0.07
0.07
0.01
0.01
0.3
0
Li-ECRAM Tang et al. (2018)
0.1
0.01
0.05
0.05
0.05
0.05
0.3
0
MO-ECRAM Kim et al. (2019b)
0.1
0.01
0.05
0.05
0.05
0.05
2
0
PCM Nandakumar et al. (2019)
0.2
0.05
0.1
0
–
–
0.6
0
Appendix
Table 7: AIHWKit SingleRPUConfig presets fitted to measured physical devices or arrays: device and cycle variation parameters. The citation in each row identifies the source measurement.
Figure 8: Response curves for all measured-device SingleRPUConfig presets in Tables 6 – 7 . Each panel shows five simulated traces under 1,000 potentiation pulses (pale blue) followed by 1,000 depression pulses (pale orange). The dashed line marks the reversal.
Device
L2
L4
L6
L8
Digital
2.2276
2.0239
1.9006
1.8422
SoftBounds
2.3302
2.0935
1.9733
1.8949
ReRamES
2.4486
2.3741
2.2386
2.1687
ReRamSB
2.4782
2.4455
2.3214
2.2486
Capacitor
2.3305
2.0934
1.9517
1.8699
Li-ECRAM
2.3139
2.0665
1.9424
1.8637
Appendix
Table 8: Final Shakespeare validation loss after 5,000 iterations for method S and the matched Digital baseline, across all general RPU device models. Lower is better; each cell is one seed.
Device
L2
L4
L6
L8
Digital
2.2276
2.0239
1.9006
1.8422
SoftBounds
2.4357
2.3168
2.2038
2.1422
ReRamES
2.5540
2.4946
2.4970
2.4409
ReRamSB
2.5309
2.5390
2.4911
2.4616
Capacitor
2.5324
2.5215
2.4885
2.5088
Li-ECRAM
2.4887
2.4689
2.3347
2.2591
Appendix
Table 9: Final Shakespeare validation loss after 5,000 iterations for method SFB and the matched Digital baseline, across all general RPU device models. Lower is better; each cell is one seed.
Figure 9: Final Shakespeare validation perplexity after 5,000 updates for S under device variation, cycle variation, input noise and output noise, with one curve per transformer depth and all other settings held fixed. Device variation jointly changes the device variation of the conductance limits and minimum update step; cycle variation changes the cycle update-step standard deviation. Input- and output-noise values are matched between the forward and backward directions and are labelled as standard deviations relative to the simulator DAC and ADC bounds, respectively. The input-noise panel ends at 8% (the first evaluated level above 7.5%) so that the two noise panels share a common ordinate. Dashed vertical lines identify the corresponding AIHWKit default: 30% for device and cycle variation, zero input noise, and 0.5% of the ADC bound for output noise. The gray background marks the common prefix in which all four depths remain within 10% of their respective zero-perturbation final perplexities. Each condition uses one fixed data and device-construction seed, so the curves show observed sensitivity in this simulator configuration rather than uncertainty estimates.
Figure 10: Final Shakespeare validation loss after 5,000 iterations for method S under converter-resolution sweeps. Rows encode forward bits and columns backward bits; lower loss is better. The ADC panel sweeps the forward and backward output resolution while holding DAC at its default resolution, and the DAC panel analogously sweeps input resolution while holding ADC at its default resolution.
Bits
L2
L4
L6
L8
3
2.9552
2.9166
2.9166
2.8549
4
2.5548
2.5036
2.4457
2.3802
5
2.4658
2.4212
2.2929
2.2218
6
2.4379
2.3800
2.2233
2.1778
7
2.4362
2.3370
2.2036
2.1418
8
2.4212
2.3420
2.1939
2.1389
Appendix
Table 10: Final Shakespeare validation loss after 5,000 iterations for SFB with matched DAC and ADC resolution.
Profile
L=2
L=4
L=6
L=8
Pure-digital initialization controls
Digital
2.2276
2.0238
1.9006
1.8422
Digital-I
2.2355
2.0073
1.9018
1.8258
Original stack and factorization of S (analog)
MpSGD
3.4444
3.4605
3.4571
3.4403
Native-map Adam ( ωmap=1 )
2.4576
2.4279
2.3902
2.3472
Appendix
Table 11: Integrated profile ablation: validation loss after 5,000 updates (lower is better). All entries use seed 1337 and evaluations averaged over 200 held-out mini-batches. All analog profiles retain a digital positional embedding; the definitions and interventions are given in the surrounding text.
Optimizer
Profile
L=2
L=4
L=6
L=8
AdamW
Digital
2.2355
2.0073
1.9018
1.8258
S
2.3422
2.0925
1.9696
1.8999
SFB
2.4419
2.3393
2.2098
2.1432
Muon
Digital
2.0141
1.8948
1.7922
1.7252
S
2.3053
2.0798
1.9650
1.8956
SFB
2.3988
2.2972
2.1825
2.1384
Appendix
Table 12: Optimizer screen under matched digital, S, and SFB profiles: final Shakespeare validation loss after 5,000 updates (lower is better). Rows are grouped by optimizer. Digital uses the same width-stable initialization as S/SFB but no AIMC conversion. All cells use seed 1337 and evaluation over 200 held-out mini-batches.
Analog in-memory computing (AIMC) performs computation directly within resistive crossbar arrays, offering an energy-efficient platform to scale large vision and language models. However, non-ideal analog device properties make the training on AIMC devices challenging. In particular, its update asymmetry can induce a systematic drift of weight updates towards a device-specific symmetric point (SP), which typically does not align with the optimum of the training objective. To mitigate this bias, most existing works assume the SP is known and pre-calibrate it to zero before training by setting the reference point as the SP. Nevertheless, calibrating AIMC devices requires costly pulse updates, and residual calibration error can directly degrade training performance. In this work, we present the first theoretical characterization of the pulse complexity of SP calibration and the resulting estimation error. We further propose a dynamic SP estimation method that tracks the SP during model training, and establishes its convergence guarantees. In addition, we develop an enhanced variant based on chopping and filtering techniques from digital signal processing. Numerical experiments demonstrate both the efficiency and effectiveness of the proposed method.
Quan Xiao, Jindan Li, Zhaoxian Wu +2
Department of Electrical and Computer Engineering, Cornell University, New York, NY · IBM T. J. Watson Research Center, Yorktown Heights, NY · Rensselaer Polytechnic Institute, Troy, NY
Analog in-memory computing is a promising platform for on-device execution of large language models because it performs matrix--vector multiplications (MVMs) in memory and in parallel, reducing data movement. However, limited digital-to-analog converter precision, input noise, and finite conductance states can degrade model accuracy, while full-model retraining to address these effects can be costly. We develop an optimizer-agnostic, parameter-efficient adaptation method based on Low-Rank Adaptation (LoRA), keeping the pretrained weights stored on analog arrays fixed while training the LoRA weights to adapt to downstream tasks and hardware non-idealities. Reliable adaptation requires handling errors in both forward and backward MVMs and physical weight updates. We use input reshaping to reduce input-induced MVM errors and update accumulation to retain small updates before programming them to finite-state analog devices. Across Llama-3.2-1B-Instruct and Llama-3-8B with both Muon and AdamW, input reshaping improves analog LoRA fine-tuning under noisy MVM computation. Update accumulation separately preserves sub-threshold updates and substantially improves adaptation under finite-resolution programming, including configurations with as few as 20 conductance states. Additional experiments show consistent held-out negative log-likelihood improvements across noisy analog settings.
Analog in-memory computing (IMC) offers a promising path toward energy-efficient large language model (LLM) inference by executing matrix multiplications (MatMul) directly within memory arrays in the analog domain. Its efficiency, however, comes with an additional source of error: limited-precision analog-to-digital converters (ADCs) quantize accumulated analog partial sums, introducing output-side error distinct from conventional activation and weight quantization at the MatMul inputs. Clipping can mitigate both operand and ADC quantization errors, but the optimal clipping factors must jointly balance activation rounding and clipping, weight rounding and clipping, and ADC quantization. Existing clipping methods, designed for digital quantization, do not explicitly optimize these coupled sources of IMC error and often rely on costly search-based calibration. We introduce IMC-CLINIC (Coupled Loss-Informed Newton Iterations for Clipping), a clipping calibration framework based on an analytical surrogate for IMC MatMul output error. The surrogate jointly models operand quantization, accumulated clipping-induced bias, and ADC quantization, enabling efficient evaluation of its gradient and approximate curvature from a small calibration set. IMC-CLINIC jointly optimizes activation and weight clipping factors using a safeguarded Newton-type method. Across multiple models and datasets, it improves average zero-shot accuracy by 6.5-11.5 percentage points over the grid search baseline while reducing calibration time by factors of 10.0-12.1. Its analytical surrogate closely tracks empirical IMC output error, and its optimizer is certified within 1% of the global optimum under the loss objective across all projections on two representative models.
Yung-Chin Chen, Chia-Yu Chen, Naveen Verma
Princeton University, NJ, USA · EnCharge AI, CA, USA