Can We Trust LLMs on Memristors? Diving into Reasoning Ability under Non-Ideality

Taiqiang Wu; Yuxin Cheng; Chenchen Ding; Runming Yang; Xincheng Feng; Wenyong Zhou; Zhengwu Liu; Ngai Wong

cs.CLMar 14, 2026

Can We Trust LLMs on Memristors? Diving into Reasoning Ability under Non-Ideality

Authors: Taiqiang Wu, Yuxin Cheng, Chenchen Ding, Runming Yang, Xincheng Feng, Wenyong Zhou, Zhengwu Liu, Ngai Wong

Organizations: The University of Hong Kong

Abstract

Memristor-based analog compute-in-memory (CIM) architectures provide a promising substrate for the efficient deployment of Large Language Models (LLMs), owing to superior energy efficiency and computational density. However, these architectures suffer from precision issues caused by intrinsic non-idealities of memristors. In this paper, we first conduct a comprehensive investigation into the impact of such typical non-idealities on LLM reasoning. Empirical results indicate that reasoning capability decreases significantly but varies for distinct benchmarks. Subsequently, we systematically appraise three training-free strategies, including thinking mode, in-context learning, and module redundancy. We thus summarize valuable guidelines, i.e., shallow layer redundancy is particularly effective for improving robustness, thinking mode performs better under low noise levels but degrades at higher noise, and in-context learning reduces output length with a slight performance trade-off. Our findings offer new insights into LLM reasoning under non-ideality and practical strategies to improve robustness.

Figures & tables

Figure 1: Overview of the non-ideality when deploying LLM on memristor-based analog CIM architectures. In this paper, we study the impact on LLM reasoning and discuss available training-free solutions.

Figure 2: Performance of Qwen3 0.6B under Non-ideality at various noise levels

\sigma

. The SAF ratio

p

is 0.01. A darker color denotes larger noise. We report the results on IFEval, GPQA-D, and MATH-500. The vanilla performance is indicated by the dotted line.

Figure 3: Error analysis of Qwen3 0.6B on Math-500. Larger noise leads to more formatting issues.

Noise	IFEval	GPQA-D	MATH-500
Noise	Mean@8 $\uparrow$	Mean@8 $\uparrow$	Mean@8 $\uparrow$
Vanilla	57.1	27.0	51.5
SAF ( $p=0.0005$ )	57.0 $\pm$ 0.4	26.6 $\pm$ 1.1	51.5 $\pm$ 0.3
SAF ( $p=0.001$ )	57.0 $\pm$ 0.4	26.7 $\pm$ 1.2	51.6 $\pm$ 0.6
SAF ( $p=0.0025$ )	56.4 $\pm$ 0.9	26.7 $\pm$ 0.7	50.9 $\pm$ 1.0
SAF ( $p=0.005$ )	56.8 $\pm$ 0.4	25.4 $\pm$ 1.5	50.5 $\pm$ 0.9

Table 1: Ablation of SAF ratio

p

Figure 4: Comparison between Non-thinking and Thinking mode that introduces an explicit thinking process.

Figure 5: Comparison of Thinking and Non-thinking modes on MATH-500. The SAF ratio

p

is 0.01. We also report the token ratio of Thinking parts.

Figure 6: One example with a step-by-step solution under the in-context learning setting. We omit the details with ellipses.

Figure 7: Performance on MATH-500 with and without in-context learning (2-shots). The SAF ratio

p

is 0.01. ICL effectively decreases the length of output tokens but also introduces longer input.

Module Redundancy
Method	Area	Energy	IFEval		GPQA-D		MATH-500
Method	(mm 2 ) $\downarrow$	(J) $\downarrow$	Mean@8 $\uparrow$	Pass@8 $\uparrow$	Mean@8 $\uparrow$	Vote@8 $\uparrow$	Mean@8 $\uparrow$	Pass@8 $\uparrow$
Vanilla (GPU)	806	1.43	57.1 $\pm$ 0.6	74.1 $\pm$ 0.3	27.0 $\pm$ 1.9	30.3 $\pm$ 2.1	51.5 $\pm$ 1.6	78.4 $\pm$ 0.6
Vanilla (Memristor $\sigma$ =0.02)	75	2.63	39.8 $\pm$ 2.2	63.0 $\pm$ 1.2	14.7 $\pm$ 7.7	13.9 $\pm$ 7.3	11.4 $\pm$ 2.6	30.9 $\pm$ 4.9
Attention ( $\times$ 2)	103	1.81	46.0 $\pm$ 3.8	70.5 $\pm$ 3.2	20.4 $\pm$ 5.4	19.8 $\pm$ 6.1	18.6 $\pm$ 2.9	41.3 $\pm$ 3.8
Attention ( $\times$ 4)	160	0.45	48.3 $\pm$ 0.9	70.6 $\pm$ 3.2	20.7 $\pm$ 6.0	21.4 $\pm$ 6.1	25.1 $\pm$ 2.8	51.9 $\pm$ 3.6

Table 2: Performance (5 runs) of Qwen3 0.6B without and with module redundancy strategies. The Energy is calculated on the MATH-500 task. For the noise simulation,

\sigma

is 0.02 and

p

is 0.01. Green denotes the best results, while blue for the second.

Method	IFEval	GPQA-D	MATH-500	Energy	Area
Method	Mean@8 $\uparrow$	Mean@8 $\uparrow$	Mean@8 $\uparrow$	(J) $\downarrow$	(mm 2 ) $\downarrow$
Llama 3.2 1B
Vanilla (GPU)	50.0	23.2	15.8	12.0	806
Vanilla ( $\sigma=0.02$ )	39.9	17.4	7.6	1.5	121.3
Shallow (4 $\times$ )	47.4	18.7	8.5	0.7	207.1
Qwen3 1.7B

Table 3: Performance on Llama 3.2 1B and Qwen3 1.7B.

Language Model Robustness

Language model robustness studies how reliably language models preserve correct, safe behavior under input changes, distribution shifts, and adversarial pressure.

See all 493 papers

MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use
Aug 20, 2026Mengru Wang, Haozhe Luo, Zhenqian Xu +6Memory-Augmented Language Models LLM Memory
Zhejiang University · National University of Singapore · Northeastern University +2
Unlocking the Working Memory of Large Language Models for Latent Reasoning
May 28, 2026Lukas Aichberger, Sepp HochreiterImplicit Reasoning in Language Models Efficient Language Model Reasoning
ELLIS Unit Linz and LIT AI Lab, Institute for Machine Learning, Johannes Kepler University Linz, Austria · NXAI GmbH, Linz, Austria
ROMER: Expert Replacement and Router Calibration for Robust MoE LLMs on Analog Compute-in-Memory Systems
May 12, 2026Wenyong Zhou, Yuannuo Feng, Yizhe Chen +6Compute-in-Memory Mixture-of-Experts Language Models
The Department of Electrical and Computer Engineering, The University of Hong Kong, Hong Kong · The School of Integrated Circuit Science and Engineering, Beihang University, Beijing, China

Aug 20, 2026J/K move · Enter open · S save

MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use

Mengru Wang, Haozhe Luo, Zhenqian Xu, Zhixiang Cui+5

Zhejiang University · National University of Singapore · Northeastern University+2

Memory-Augmented Language Models LLM Memory LLM Evaluation Memorization in Language Models

Open paper AskPDF arXiv

Memory has become a key component of large language models, enabling them to retain information and learn from long-term interactions. However, existing memory benchmarks mainly evaluate whether information is correctly extracted, stored, and retrieved, while largely overlooking how retrieved memories reshape model reasoning and affect performance on the current task. We identify memory-induced cognitive traps: even faithfully recorded and semantically relevant memories can distort model reasoning or beliefs and degrade current task performance. To systematically evaluate these failure modes, we introduce MemTrapBench, which covers two forms of cognitive traps: Reasoning Fixation and Belief Distortion. Experiments across two model families and five representative memory frameworks show that MemTrapBench is challenging: all evaluated memory strategies underperform the no-memory setting, with even the strongest methods suffering drops of more than 10%. To mitigate these cognitive traps, we propose AdaptiveMem, a simple yet effective inference-time method that instructs LLMs to avoid memory traps. AdaptiveMem mitigates cognitive traps on MemTrapBench while preserving or improving performance on standard memory benchmarks across diverse memory frameworks.

Abstract

Language Model Robustness

Explore similar work

MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use

Unlocking the Working Memory of Large Language Models for Latent Reasoning

ROMER: Expert Replacement and Router Calibration for Robust MoE LLMs on Analog Compute-in-Memory Systems