cs.CLNov 7, 2026

The Truth Beneath the Tokens: Prompt-Space Recovery of Knowledge Edits in Language Models

Authors: Manit Baser, Rachel Yeo Hui Min, Aditya Nawal, Dinil Mon Divakaran, Mohan Gurusamy

Organizations: National University of Singapore, Singapore · A*STAR Institute of Advanced Intelligence and Computing (A*STAR IAIC), Singapore

Abstract

Model editing provides an efficient way to update factual knowledge in transformer models. Recent approaches reverse edits by modifying, masking, or decomposing model weights, but make recovery dependent on weight-space interventions. We instead show that pre-edit knowledge can be recovered through input-space interventions alone, without modifying the edited model's weights. We introduce Memento, a prompt-space mechanism for auditing residual knowledge that learns a reusable recovery signal to steer an edited model toward its pre-edit response. Memento supports both continuous recovery prefixes and discrete textual prompts for standard text interfaces. Across three LLMs and five editing techniques, Memento recovers 80.07% of held-out edited facts, achieving a 9.58% relative improvement over baselines, suggesting that editing may suppress the facts without fully eliminating their recoverability. We further investigate whether prompt-space recovery extends beyond model editing. On pristine, unedited LLMs, Memento improves average factual recall by 53.9% relative to standard inference. These results demonstrate the potential of prompt-space recovery for auditing residual knowledge in edited models and investigating the gap between factual knowledge and its accessibility during inference. We will release our source code.

Figures & tables

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. One Mask to Rule Them All: On Hidden Facts after Editing and How to Find Them

    Apr 18, 2026Ali Holmov, Paul Youssef, Nandi Schoots +1Knowledge EditingTransformer Interpretability

  2. SCAN: Sparse Circuit Anchor Interpretable Neuron for Lifelong Knowledge Editing

    Mar 16, 2026Yuhuan Liu, Haitian Zhong, Xinyuan Xia +3Sequential Knowledge EditingKnowledge Editing

  3. EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory

    Oct 7, 2026Hongru Cai, Ran Wei, Wenjie Wang +4Knowledge EditingFactual Knowledge in Language Models