Organizations: University of Sydney Sydney, Australia · Nanjing University of Science and Technology Nanjing, China · Edith Cowan University Perth, Australia · National University of Singapore Singapore
Large language models are typically evaluated in terms of the correctness of their output. However, output correctness alone does not confirm whether the latent knowledge supporting an answer has stable internal representations. We therefore propose latent state stability (LSS) and knowledge topology stability (KTS) to investigate the stability of individual knowledge states and their relational structure, respectively. Specifically, we first separately average the model's internal representations of the same knowledge under each elicitation condition. Using the mean representation under the baseline condition as a reference, we remove shared shifts to obtain corrected representations. We then formulate LSS as the distance between each corrected representation and its reference representation, normalized against the typical distance between knowledge items of the same type. We formulate KTS as the cross-condition consistency of the ordered pairwise representation distances among knowledge items of the same type. Next, we construct EpiStableBench, which comprises 4,411 items spanning factual knowledge and mathematical reasoning, with 76,766 controlled queries. Across 16 LLMs from four families, representation stability varies substantially even among items answered correctly under the base condition. Within models, higher stability generally corresponds to higher accuracy under held-out elicitation conditions; across models, stability does not consistently increase with model size or overall accuracy. Meanwhile, we present LatentDPO to learn a knowledge-editing policy that accounts for the stability of latent knowledge. Experimental results show that integrating LatentDPO with multiple existing editing methods consistently improves editing success and generalization across different expressions.
Figures & tables
Figure 1: Motivation For The Correctness–Stability Hypothesis. (a) Human intuition suggests that well-established knowledge should consistently yield correct answers. (b) In LLMs, an initially correct answer may either persist or fail in subsequent elicitation conditions. (c) These observations motivate the hypothesis that model outputs may vary with representational stability.
Figure 2: Overview of EpiStableBench. (a) Canonical questions and four controlled-query types varying paraphrase, format, context, and language for factual knowledge ( orange ) and mathematical reasoning ( blue ). (b) Distribution of 4,411 knowledge items across domains and categories.
Table 1: Model-Family and Scaling Comparison. ACC, LSS, and KTS are reported across open-weight LLM families and parameter scales for factual knowledge, mathematical reasoning, and their average. The best and second-best results are shown in bold and underlined . More results, please see the appendix.
Figure 3: Behavioral Errors Accompanied By Latent Shifts. Items answered correctly under baseline elicitation condition produce competing answers under changed elicitation conditions, accompanied by representational shifts toward facts associated with those answers. More results please see the appendix.
Figure 4: Stability and Subsequent Correctness. Among targets answered correctly under the base condition, high-LSS targets and high-KTS groups show higher ACC under held-out elicitation conditions than their low-stability counterparts.
Figure 5: Semantic Sensitivity of LSS and KTS. Hard negatives induce larger metric changes than target-preserving perturbations across all three models.
Figure 6: Separating Shared Shifts from Item-Specific Drift. (Left) Decomposition of normalized squared drift into the portion removed by alignment and the remaining residual. (Right) Gains in answer-change prediction across alignment variants, measured in AUROC points.
Setting
LSS
KTS
Model
Item
Model
Relation
No Norm. / Centering
0.23
0.65
0.99
0.93
Global Norm. / Structure
0.84
0.64
0.77
0.15
Table 2: Relation-Aware Processing. Spearman correlations between ablated and full-method scores at the indicated aggregation levels. Labels before and after the slash denote LSS and KTS variants, respectively.
Setting
LSS
KTS
Model
Item
Model
Relation
Depth 40%–60%
0.14
0.42
-0.10
0.59
Depth 60%–80%
0.68
0.77
0.53
0.85
Depth 80%–100%
0.96
1.00
0.99
1.00
Final Layer Only
0.60
0.84
0.98
0.95
Raw States, Final 25%
0.47
0.82
0.89
0.80
Table 3: Representation and Layer Selection. Spearman correlations with the full method, which aggregates J-Lens representations over the final 25% of layers. Depth is normalized from input to output.
Setting
LSS
KTS
Model
Item
Model
Relation
Leave-one-probe-out
Drop Paraphrase
0.98
0.97
0.99
0.97
Drop Format
0.96
0.95
0.97
0.96
Drop Context
0.95
0.95
0.92
0.91
Drop Multilingual
0.95
0.95
0.93
0.90
Table 4: Probe and Reference Sensitivity. Spearman correlations with the full method after omitting an elicitation family or changing the reference condition.
Figure 7: Geometric Patterns of Internal Knowledge Stability. Representative high- and low-LSS items and high- and low-KTS relations are compared across the base and controlled elicitation conditions.
Figure 8: Layer-Specific Activation Patching. Target-answer recovery from same-item canonical states versus different-item controls within the same relation.
Figure 9: Family-Specific Sensitivity to Elicitation. Condition-wise changes in ACC, LSS, and KTS relative to the base condition across four model families.
Method
GPT2-XL
LLaMA 2-7B
Accuracy ↑
Generality ↑
Specificity ↑
Accuracy ↑
Generality ↑
Specificity ↑
ROME
99.40
73.67
80.61
99.90
67.37
93.87
ROME + Ours
99.65 (+0.25)
80.48 (+6.81)
80.27 (-0.34)
99.90 (+0.00)
75.75 (+8.38)
93.33 (-0.54)
MEMIT
72.37
38.89
87.57
76.08
54.23
74.40
MEMIT + Ours
87.14 (+14.77)
54.98 (+16.09)
86.94 (-0.63)
90.79 (+14.71)
69.17 (+14.94)
73.52 (-0.88)
SAKE
94.59
83.96
84.20
96.70
80.73
84.67
Table 5: Application to Knowledge Editing. Accuracy, Generality, and Specificity (%) under identical edit sets and evaluation protocols. Parentheses report percentage-point changes relative to the baseline.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Symbol
Definition
m,ℓ,i,q
Model, layer, knowledge item, and query, respectively.
I,IF,IM
All knowledge items and the factual-knowledge and mathematical-reasoning subsets, respectively.
ki=(xi,ri,yi)
Knowledge item i , where xi is the queried content, ri is its benchmark-defined comparison group, and yi is the target answer.
τi∈{F,M}
Track containing item i , where F and M denote the factual-knowledge and mathematical-reasoning tracks, respectively.
a,T,Ti
Reference elicitation condition a , elicitation-condition families T , and the conditions Ti⊆T available for item i .
Qi,t
Set of target-preserving queries for item i under elicitation condition t .
Appendix
Table 6: Core notation for Latent State Stability (LSS) and Knowledge Topology Stability (KTS).
Predictor
Qwen2.5-3B
Qwen2.5-7B
Strong Output Baseline
0.5993
0.7150
+ Within-relation CKA
0.6033
0.7166
+ All Standard Features
0.6030
0.7137
+ LSS
0.6102
0.7169
+ KTS
0.6149
0.7262
+ LSS/KTS
0.6154
0.7333
Appendix
Table 7: Comparison With Standard Representation Measures. All representation features augment the same strong output baseline. Values report failure-prediction AUROC.
Ablation
LSS Rank Correlation
Calibration Corpus
0.964 – 0.990
J-Lens Transport Rank
0.903 – 0.991
Raw/PCA Representations
0.373 – 0.780
Question-level Pooling
0.161 – 0.561
Appendix
Table 8: Sensitivity to J-Lens and Representation Choices. LSS rank correlations under changes to the calibration corpus, transport rank, representation space, and token extraction position.
Figure 10: KTS Reliability Across Relation Sizes. Reliability increases with the number of shared items, with both relation-level and condition-pair estimates exceeding the 90% target at 200 items.
Metric
Setting Change
Rank ρ
Max. Shift
KTS
Alternative Tie Conventions
1.000
0
Shared-item Threshold: 100→10
0.985
2
Item-weighted Relations
0.974
2
LSS
Additive Constant: 0 – 10−3
1.000
0
Item-weighted Relations
0.988
2
Appendix
Table 9: Sensitivity to Numerical and Aggregation Choices. Spearman correlations and maximum rank shifts are measured against the default settings across 16 models in the audited factual cohort. The LSS constant row summarizes all seven tested settings; other rows concern the full method.
Figure 11: More Qualitative Analysis of LSS.
Figure 12: More Qualitative Analysis of KTS.
Table 10: Extended Model Rankings. ACC, LSS, and KTS are reported for 43 additional model configurations across 11 model families. The best and second-best results in each column are shown in bold and underlined, respectively.
Backbone
Editor
αb,e
τb,e
GPT2-XL
ROME
0.75
5.25
MEMIT
0.50
5.50
SAKE
1.00
6.75
LLaMA 2-7B
ROME
0.75
5.25
MEMIT
0.25
5.55
SAKE
1.00
6.75
Appendix
Table 11: Frozen Gated-Correction Configurations. Correction strength αb,e and prompt-gate threshold τb,e used to construct action 1 .
Humans use introspection to evaluate their understanding through private internal states inaccessible to external observers. We investigate whether large language models possess similar privileged knowledge about answer correctness, information unavailable through external observation. We train correctness classifiers on question representations from both a model's own hidden states and external models, testing whether self-representations provide a performance advantage. On standard evaluation, we find no advantage: self-probes perform comparably to peer-model probes. We hypothesize this is due to high inter-model agreement of answer correctness. To isolate genuine privileged knowledge, we evaluate on disagreement subsets, where models produce conflicting predictions. Here, we discover domain-specific privileged knowledge: self-representations consistently outperform peer representations in factual knowledge tasks, but show no advantage in math reasoning. We further localize this domain asymmetry across model layers, finding that the factual advantage emerges progressively from early-to-mid layers onward, consistent with model-specific memory retrieval, while math reasoning shows no consistent advantage at any depth.
Tomer Ashuach, Shai Gretz, Yoav Katz +2
Technion – Israel Institute of Technology · IBM Research · Kempner Institute, Harvard University
Large Language Models (LLMs) internalize vast world knowledge as parametric memory, yet inevitably inherit the staleness and errors of their source corpora. Consequently, ensuring the reliability and malleability of these internal representations is imperative for trustworthy real-world deployment. Knowledge editing offers a pivotal paradigm for surgically modifying memory without retraining. However, while recent editors demonstrate high success rates on standard benchmarks, it remains questionable whether current evaluation frameworks that rely on assessing output under specific prompting conditions can reliably authenticate genuine memory modification. In this work, we introduce a simple diagnostic framework that subjects models to discriminative self-assessment under in-context learning (ICL) settings that better reflect real-world application environments, specifically designed to scrutinize the subtle behavioral nuances induced by memory modifications. This probing reveals a pervasive phenomenon of Surface Compliance, where editors achieve high benchmark scores by merely mimicking target outputs without structurally overwriting internal beliefs. Moreover, we find that recursive modifications accumulate representational residues, triggering cognitive instability and permanently diminishing the reversibility of the model's memory state. These insights underscore the risks of current editing paradigms and highlight the pivotal role of robust memory modification in building trustworthy, long-term sustainable LLM systems. Code is available at https://github.com/XiaojieGu/SA-MCQ.
Xiaojie Gu, Ziying Huang, Weicong Hong +3
Independent Researcher · Cornell Tech · The Ohio State University +1
Large language models (LLMs) often produce incorrect or outdated content after being employed. Efficient and accurate knowledge updates without costly retraining are a major challenge. This problem is particularly challenging in lifelong settings, where complex, unstructured knowledge must coexist without interference. We introduce RILKE (Representation Intervention for Lifelong KnowledgE Control), a robust and scalable method that treats knowledge control as interventions within the model's representation space. Leveraging representation-space expressiveness, we identify two key properties enabling RILKE to achieve fine-grained control over complex, unstructured knowledge while maintaining general utility with frozen base weights. During training, RILKE learns paraphrase-robust and edit-localized modules that limit each update to a low-dimensional subspace to minimize cross-edit interference. At inference, a query-adaptive router selects the appropriate module to guide the model's generation. Across LLaMA and Qwen models, RILKE scales effectively to large-scale benchmarks, demonstrating high edit success and strong paraphrase generalization while preserving general utility with modest memory overhead. These results show RILKE is an effective and scalable solution for lifelong knowledge control in LLMs.
Xuyuan Liu, Shengyu Chen, Xinshuai Dong +6
Dartmouth College · NEC Laboratories America · Carnegie Mellon University