Analog compute-in-memory (CIM) accelerators compute matrix-vector products directly inside memory arrays, eliminating weight movement and promising order-of-magnitude gains in inference efficiency. However, converter and device errors make pretrained models unusable on these arrays, and hardware-aware retraining is costly. We enable retraining-free, accurate inference of pretrained models on simple analog arrays with low-cost, shared calibration. Our key observation is that analog error depends on how each layer is represented on the array, and that this representation can be changed offline without changing the computation. We center the inputs and apply Hadamard rotations designed for analog arrays, which keep shared reference errors spread and confine input mixing to each array. Across CNNs, vision transformers and LLMs up to 14B parameters on PCM and ReRAM, the method brings vision models and larger LLMs within a few points of floating-point accuracy and matches or approaches published hardware-aware training without training any weights. On ReRAM with 6-bit ADCs, it restores ViT-B/16 from 37.4% to 82.9% and Phi-4 from 27.0% to 72.2%, both within 2.2 points of floating-point accuracy.
Figures & tables
Observation
Array effect
Our fix
Sec.
Static input components
Static work picks up converter and device error on every input
Center inputs and restore digitally
3.1
Concentrated partial outputs
Peaks force a clip–quantization tradeoff under one ADC range
Regular output rotation to lower peaks
3.2
Concentrated weights
Outliers set the mapping scale and amplify conductance perturbations
Input and output rotations to reduce weight peaks
3.3
Constraint: shared/common-mode terms
Plain Hadamard can concentrate common-mode (e.g., reference) error
Table 1: How the structure of the computation and the array’s constraints determine our isometric reparameterization.
Figure 1: An analog CIM array of width w and height h with its DAC and ADC (left), and a weight matrix split into tiles whose partial sums are added digitally before the bias (right).
Figure 2: Static input component across all analog layers of ViT-B/16, ResNet-50 and Qwen3-1.7B, with 4-bit ADCs and without rotations or device noise. Left: share of ∥x∥22 in the per-channel mean. Right: local ADC error without and with centering. Bars are medians over layers.
Figure 3: Output rotation reduces average local ADC error across the FFN down-projections of ViT-B/16 and Qwen3-1.7B, with MSE-calibrated ranges (top) and ranges set from the largest output (bottom). Projections are untiled, with 8-bit DACs, one 4-bit ADC range per projection, no centering and no device noise. NMSE is averaged and Peak/RMS is the median over layers.
Figure 4: Regular Hadamard blocks are needed to avoid the large ADC error increase of plain blocks with random signs (DHP), which perform worse than no output rotation. Projections are untiled, with 8-bit DACs, one MSE-calibrated 4-bit ADC range per projection and no device noise.
Figure 5: Weight peak-to-RMS for no rotation, output only, input only, and both, across all 72 projections of ViT-B/16 and all 196 of Qwen3-1.7B. Output and input rotations each lower weight concentration on their own, but both are needed for the largest reduction. Faint lines are single projections, and medians are printed.
Figure 6: Local output error from device noise before and after both rotations, across the down projections of ViT-B/16 and Qwen3-1.7B, with PCM and ReRAM models at one day. Projections are untiled, with one weight mapping scale each and no converters or centering.
Figure 7: (a) Our method. Offline: calibrate the channel mean μ , fold c=Wμ+b into the bias, build Rin and Rout , transform the weights to W=RoutWRin⊤ , then program W and calibrate the converters. Online: (1) center, (2) rotate the inputs within each tile, (3) compute in each array and digitize its partial output, (4) sum the digitized partial outputs, (5) undo the output rotation and apply the drift gain g (for PCM), (6) add c . (b) Placement of the rotations in a CNN bottleneck and a transformer block (ViT or LLM).
Table 9
Figure 8: ImageNet top-1 accuracy of ViTs and CNNs on PCM and ReRAM with N-bit ADCs, 1 day after programming.
Model
Device
Unmodified
HWA
Ours
ResNet-50
PCM
61.3±3.1
74.7
73.7±0.1
ResNet-18
PCM
56.1±0.8
68.0
67.5±0.1
ResNet-50
ReRAM
58.1±1.9
72.4
72.8±0.1
ResNet-18
ReRAM
44.9±3.3
65.1
65.6±0.2
Table 4: ResNet accuracy (%) alongside published HWA results Rasch et al. (2023) , with PCM or the ReRAM model of Wan et al. (2022) , 8-bit ADCs and one-day read age, with per-column weight mapping scales as in Rasch et al. (2023) . Unmodified and Ours report mean ± SD over three seeds.
Figure 9: LLM mean accuracy over MMLU, HellaSwag, ARC-Easy and ARC-Challenge on PCM and ReRAM with 4-, 6- and 8-bit ADCs, one day after programming. Lines mark floating-point accuracy.
Model
Unmodified
AFM
Ours
Phi-3-mini
72.9±0.9
79.7
78.4±0.5
Llama-3.2-1B
36.4±1.6
45.5
42.8±1.5
Table 5: LLM mean accuracy (%) compared with the published AFM results Büchel et al. (2025) , in our reproduction of their setting, with their checkpoints, scoring and noise level (Appendix J ). Unmodified and Ours report mean ± SD over ten seeds.
Figure 10: Accuracy as output rotations, input rotations and centering are added in that order, on PCM with 4-bit ADCs at one day and ReRAM with 8-bit ADCs at five days. Dashed lines mark floating-point accuracy.
Figure 11: Accuracy with per-array or per-column weight mapping scales and ADC ranges, one day after programming. NORA uses its native dynamic DAC.
Figure 12: Accuracy against read age with 6-bit ADCs. The arrays are programmed and calibrated once at 1 s and not refreshed.
Figure 13: Accuracy against the digital share of all operations on ReRAM with 8-bit ADCs at five days. Each curve is the frontier of the evaluated configurations, and dotted lines mark its peak accuracy. The unmodified models lie below the plotted range.
Figure 14: Accuracy with 4- and 6-bit ADCs and DACs on PCM at one day and ReRAM at five days: top-1 accuracy for ViT-B/16 and ResNet-50, and WikiText-2 Merity et al. (2017) perplexity for Qwen3-1.7B (lower is better). ViT-B/16 floating-point accuracy is 84.6%, ResNet-50 76.1%.
Figure 15: SRAM CIM accuracy with one weight mapping scale and one ADC range per tensor (open dots, as in PWS) or per array (filled dots, as in FWS). Dots at zero indicate accuracies below 2%.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Input mean μ
Clean pass, per input channel
DAC range
Clean pass, per layer input
ADC range
Clean pass, per array, scaled offline for nominal drift
Drift gain g(t)
PCM only, probe readout at read time Joshi et al. (2020)
Reference column
ReRAM only, one per array, subtracted before the ADC
Weights
Programmed once at t=0 , never refreshed
Appendix
Table 6: What is calibrated, measured or refreshed.
Figure 16: Qwen3-1.7B first-token diagnostic. (a) Maximum absolute layer-2 projection inputs for the first and remaining tokens over four prompts. (b) WikiText NLL, the log perplexity, on PCM with 6-bit ADCs, 8-bit DACs and a one-day read age, showing the impact of executing the first token on the arrays.
Method
Precision
k
MMLU
HellaSwag
ARC-C
ARC-E
Mean
Phi-3-mini
Published results
FP
W16
–
69.4
83.5
84.6
91.1
82.1
Original
W16h
–
63.3
73.4
80.4
89.0
76.5
AFM
SI8-W16h-O8
–
65.1
80.6
83.2
89.8
79.7
LLM-QAT Liu et al. (2024)
SI8-W4h
–
60.1
74.8
77.9
87.2
75.0
Appendix
Table 7: AFM comparison: per-task accuracy (%) and the unweighted four-task mean. k multiplies the programming-noise standard deviation: 1.00 is nominal and 1.10 is adjusted to match the published noise-only baseline. Our SI8/O8 rows use one seed at k=1.00 and ten at k=1.10 , and noise-only rows use four Phi seeds and three/ten Llama seeds at k=1.00/1.10 . Uncertainties are sample SDs. SI/DI denote static/dynamic inputs, and W16h/W4h denote the publication’s hardware-noisy weight formats.
Figure 17: Accuracy with one component removed from our method, on PCM with 4-bit ADCs at one day and ReRAM with 8-bit ADCs at five days. Vision models use 4,898 evaluation images and one seed.
Reference
Regular blocks
Plain blocks
Unmodified
Reference column
82.9
82.7
24.8
Fixed reference
82.7
0.0
0.1
Appendix
Table 8: ViT-B/16 top-1 accuracy (%) on CMO ReRAM with 8-bit ADCs five days after programming, with our method using regular or plain output blocks, on 4,898 evaluation images with one seed.
Model
Unmodified
Ours
ViT
5.7 / 21
0.66 / 2.8
ResNet
2.0 / 13
0.21 / 2.2
Qwen
7.9 / 22
0.92 / 2.4
Appendix
Table 9: Gain spread (%) of per-array compensation, PCM with 4-bit ADCs at one day: the relative difference between the gains it assigns to the arrays of one layer, as median / maximum over layers.
Figure 18: ViT-B/16 accuracy against the digital share of all analog and digital operations, on ReRAM with 8-bit ADCs at five days, with programming, read and drift noise scaled by k . The dashed line marks floating-point accuracy.
ADC
IR off
1×
1.5×
2×
4
80.2/4.0
80.0/3.9
80.1/3.7
79.8/3.7
6
82.2/44.1
82.3/45.0
82.5/44.6
82.3/45.2
8
82.5/44.4
82.5/44.3
82.5/44.6
82.7/44.5
Appendix
Table 10: ViT-B/16 top-1 accuracy (%) on PCM at one day, as ours/unmodified, without IR drop and with IR drop at 1, 1.5 and 2 times the default wire resistance.
Model
Device
U
Q
O
FP
Llama-3.2-1B
PCM
24.6
26.1
41.0
50.1
ReRAM
24.2
26.5
42.0
Qwen3-1.7B
PCM
23.9
31.3
47.2
59.0
ReRAM
23.7
29.6
48.2
Phi-3-mini
PCM
25.7
27.3
60.2
68.4
ReRAM
24.9
26.5
61.2
Appendix
Table 11: LLM mean accuracy (%) of the unmodified model (U), QuaRot (Q) and our method (O) with 4-bit ADCs, one day after programming. Phi models are scored on 512 questions per task for all three methods.
Device / configuration
1 s
1 h
1 day
5 days
PCM, pair + GDC
4.62 (3.32)
4.44 (2.55)
4.27 (2.06)
4.17 (1.86)
ReRAM 0.2% fixed ref.
5.67 (5.60)
4.41 (4.39)
4.06 (4.04)
3.91 (3.89)
ReRAM 2% fixed ref.
4.99 (4.40)
4.22 (3.95)
3.93 (3.74)
3.80 (3.63)
ReRAM 0.2% fixed ref. + GDC
5.69 (4.18)
4.48 (2.28)
4.17 (1.84)
4.04 (1.66)
ReRAM 2% fixed ref. + GDC
5.00 (4.07)
4.30 (2.38)
4.04 (1.92)
3.93 (1.73)
ReRAM 0.2% physical ref.
5.17 (5.14)
4.36 (4.32)
4.14 (4.00)
4.03 (3.86)
Appendix
Table 12: Effective precision beff of continuous ViT-B/16 conductances versus read age. Parentheses give effective precision computed from the worst conditional RMS error over the conductance grid. GDC denotes global drift compensation.
Analog in-memory computing is a promising platform for on-device execution of large language models because it performs matrix--vector multiplications (MVMs) in memory and in parallel, reducing data movement. However, limited digital-to-analog converter precision, input noise, and finite conductance states can degrade model accuracy, while full-model retraining to address these effects can be costly. We develop an optimizer-agnostic, parameter-efficient adaptation method based on Low-Rank Adaptation (LoRA), keeping the pretrained weights stored on analog arrays fixed while training the LoRA weights to adapt to downstream tasks and hardware non-idealities. Reliable adaptation requires handling errors in both forward and backward MVMs and physical weight updates. We use input reshaping to reduce input-induced MVM errors and update accumulation to retain small updates before programming them to finite-state analog devices. Across Llama-3.2-1B-Instruct and Llama-3-8B with both Muon and AdamW, input reshaping improves analog LoRA fine-tuning under noisy MVM computation. Update accumulation separately preserves sub-threshold updates and substantially improves adaptation under finite-resolution programming, including configurations with as few as 20 conductance states. Additional experiments show consistent held-out negative log-likelihood improvements across noisy analog settings.
Analog in-memory computing (AIMC) offers an alternative for model training by executing matrix operations directly where weights are stored. However, scaling AIMC to train modern deep models remains an open challenge due to severe hardware non-idealities, including physical weights with finite dynamic range and write granularity, analog-digital converters with finite resolution, and noisy and asymmetric updates. Guided by the insight that gradient accumulation is sensitive to precision and rounding errors, we adopt a mixed-precision training paradigm: executing forward and backward matrix multiplications in the analog domain while computing weight gradients in the digital domain. To enable scalable training, we present a holistic system-algorithm co-design that co-optimizes weight mapping to ensure well-conditioned physical and logical weight profiles, couples a preconditioned optimizer with threshold-triggered open-loop pulsing to stabilize training trajectories, and aligns converter dynamic ranges to suppress quantization errors. Evaluated via hardware-calibrated architectural simulations calibrated with electrochemical RAM measurements, our framework scales Transformer training up to 123M parameters with validation loss scaling as L∝N−0.231, where N is the parameter count, comparable to L∝N−0.238 for digital training.
We show with experiments and system-level simulations that it is possible to successfully mitigate the impact of retention loss on inference accuracy degradation by using both circuit-level compensation techniques and batch normalization recalibration at the algorithmic level. Experiments are performed on a single-poly floating-gate (FG) analog non-volatile memory array for analog in-memory computing fabricated in a standard 65 nm CMOS. We use a model of retention-loss statistics calibrated with experiments to evaluate the system-level impact on neural network models such as VGG-10/CIFAR-10 and WideResNet-28-10/CIFAR-100. We show that, after 60 days since programming, combined mitigation techniques enable to recover the baseline inference accuracy within 2-4%
Mirko Brazzini, Giulio Filippeschi, Alessandro Catania +2
Dipartimento di Ingegneria dell’Informazione, University of Pisa, Pisa, Italy · Quantavis SRL, Pisa, Italy