Knowledge Conflicts in Language Models

Ask about this topic

Overview

Knowledge conflicts arise when a model’s learned knowledge disagrees with information in a prompt, retrieved documents, tools, or stored conversation memory. Researchers test how models handle counterfactual and outdated facts, contradictory sources, and differences in framing or evidence order, using question-answering benchmarks, retrieval tasks, and multi-turn evaluations. Approaches include conflict-aware retrieval and decoding, confidence-based abstention, fine-tuning, and interventions that probe or steer internal representations. A central challenge is deciding when to follow context and when to resist it: stronger evidence-following can improve faithfulness but also make models more vulnerable to misleading, stale, or malicious information.

Latest papers

CardsList

142 papers16 papers in the last four weeks, down 33% on the four weeks before. 0.1% of all new papers.

Apr 8, 2026

Language Bias under Conflicting Information in Multilingual LLMs

Large Language Models (LLMs) have been shown to contain biases in the process of integrating conflicting information when answering questions. Here we ask whether such biases also exist with respect to which language is used for each conflicting piece of information. To answer this question, we extend the conflicting needles in a haystack paradigm to a multilingual setting and perform a comprehensive set of evaluations with naturalistic news domain data in five different languages, for a range of multilingual LLMs of different sizes. We find that all LLMs tested, including GPT-5.2, ignore the conflict and confidently assert only one of the possible answers in the large majority of cases. Furthermore, there is a consistent bias across models and prompting languages in which languages are preferred, with a general bias against Russian and, for the longest context lengths, in favor of Chinese. The language preferences are consistent between models trained inside and outside of mainland China, though somewhat stronger in the former category. There is also a general tendency among models to prioritize information that matches the language used for prompting. We hope to make users and developers of multilingual LLMs aware of this category of biases, to spur further research on their causes and possible mitigation.

Robert Östling, Murathan Kurfalı

Stockholm University Sweden · RISE Research Institutes of Sweden Sweden

Linguistic Bias in Language ModelsLanguage Model Bias EvaluationMultilingual Language ModelsKnowledge Conflicts in Language ModelsMultilingual Language Model Evaluation
AskPDF
Apr 5, 2026

ClawArena: Benchmarking AI Agents in Evolving Information Environments

AI agents deployed as persistent assistants must maintain correct beliefs as their information environment evolves. In practice, evidence is scattered across heterogeneous sources that often contradict one another, new information can invalidate earlier conclusions, and user preferences surface through corrections rather than explicit instructions. Existing benchmarks largely assume static, single-authority settings and do not evaluate whether agents can keep up with this complexity. We introduce ClawArena, a benchmark for evaluating AI agents in evolving information environments. Each scenario maintains a complete hidden ground truth while exposing the agent only to noisy, partial, and sometimes contradictory traces across multi-channel sessions, workspace files, and staged updates. Evaluation is organized around three coupled challenges: multi-source conflict reasoning, dynamic belief revision, and implicit personalization, whose interactions yield a 14-category question taxonomy. Two question formats, multi-choice (set-selection) and shell-based executable checks, test both reasoning and workspace grounding. ClawArena comprises 12 multi-turn scenarios spanning 337 evaluation rounds with 45 dynamic updates, evaluated across five agent frameworks and 18 language models from proprietary, community-accessible, and self-hosted sources. Experiments show that model capability accounts for a 29-point score range across models while framework design accounts for up to a 24-point range, that MetaClaw's skill overlay reliably improves score without degrading accuracy, and that belief revision difficulty is determined by update design strategy rather than update volume. Code is available at https://github.com/aiming-lab/ClawArena.

Haonian Ji, Kaiwen Xiong, Siwei Han +9

UNC-Chapel Hill · University of California, Berkeley · University of California, Santa Cruz

AI Agent EvaluationAI Agent BenchmarksKnowledge Conflicts in Language ModelsBelief Revision
AskPDF
Apr 4, 2026

When Do Hallucinations Arise? A Graph Perspective on the Evolution of Path Reuse and Path Compression

Reasoning hallucinations in large language models (LLMs) often appear as fluent yet unsupported conclusions that violate either the given context or underlying factual knowledge. Although such failures are widely observed, the mechanisms by which decoder-only Transformers produce them remain poorly understood. We model next-token prediction as a graph search process over an underlying graph, where entities correspond to nodes and learned transitions form edges. From this perspective, contextual reasoning is a constrained search over a sampled subgraph (intrinsic reasoning), while context-free queries rely on memorized structures in the underlying graph (extrinsic reasoning). We show that reasoning hallucinations arise from two fundamental mechanisms: Path Reuse, where memorized knowledge overrides contextual constraints during early training, and Path Compression, where frequently traversed multi-step paths collapse into shortcut edges in later training. Together, these mechanisms provide a unified explanation for reasoning hallucinations in LLMs and connected to well-known behaviors observed in downstream applications.

Xinnan Dai, Kai Yang, Cheng Luo +3

Michigan State University · University of Michigan · Independent Researcher

Hallucination in Language ModelsKnowledge Conflicts in Language ModelsLLM Reasoning with GraphsLLM Memory
AskPDF
Apr 3, 2026

DeltaLogic: Minimal Premise Edits Reveal Belief-Revision Failures in Logical Reasoning Models

Reasoning benchmarks typically evaluate whether a model derives the correct answer from a fixed premise set, but they under-measure a closely related capability that matters in dynamic environments: belief revision under minimal evidence change. We introduce DeltaLogic, a benchmark transformation protocol that converts natural-language reasoning examples into short revision episodes. Each episode first asks for an initial conclusion under premises P, then applies a minimal edit δ(P), and finally asks whether the previous conclusion should remain stable or be revised. We instantiate DeltaLogic from FOLIO and ProofWriter and evaluate small causal language models with constrained label scoring. On a completed 30-episode Qwen evaluation subset, stronger initial reasoning still does not imply stronger revision behavior: Qwen3-1.7B reaches 0.667 initial accuracy but only 0.467 revision accuracy, with inertia rising to 0.600 on episodes where the gold label should change, while Qwen3-0.6B collapses into near universal abstention. There, Qwen3-4B preserves the same inertial failure pattern (0.650 initial, 0.450 revised, 0.600 inertia), whereas Phi-4-mini-instruct is substantially stronger (0.950 initial, 0.850 revised) but still exhibits non-trivial abstention and control instability. These results suggest that logical competence under fixed premises does not imply disciplined belief revision after local evidence edits. DeltaLogic therefore targets a distinct and practically important reasoning capability that complements existing logical inference and belief-updating benchmarks.

Amit Dhanda

Amazon

LLM ReasoningBelief Updating in LLMsReasoning Consistency in Language ModelsKnowledge Conflicts in Language Models
AskPDF
Mar 26, 2026

In-Context Molecular Property Prediction with LLMs: A Blinding Study on Memorization and Knowledge Conflicts

The capabilities of large language models (LLMs) have expanded beyond natural language processing to scientific prediction tasks, including molecular property prediction. However, their effectiveness in in-context learning remains ambiguous, particularly given the potential for training data contamination in widely used benchmarks. This paper investigates whether LLMs perform genuine in-context regression on molecular properties or instead rely on verbatim retrieval of memorized target values. Furthermore, we analyze the interplay between pre-trained knowledge and in-context information through a series of progressively blinded experiments. We evaluate nine LLM variants across three families (GPT-4.1, GPT-5, Gemini 2.5) on three MoleculeNet datasets (Delaney solubility, Lipophilicity, QM7 atomization energy) using a systematic blinding approach that iteratively reduces available information, complemented by 0-, 60-, and 1000-shot in-context sample sizes as an additional control for information access. To validate the memorization analysis and the blinding experiments, we add a positive and a negative control for the memorization experiments and structural reference baselines for the multi-shot experiments as well as bootstrap confidence intervals for all results. We find no evidence of verbatim retrieval on the legacy benchmarks and show that blinding exposes conflicts between pre-trained knowledge and in-context information. This work provides a principled framework for evaluating molecular property prediction under controlled information access.

Matthias Busch, Marius Tacke, Sviatlana V. Lamaka +4

Institute for Continuum and Material Mechanics, Technical University of Hamburg, Eißendorfer Straße, 21073 Hamburg, Germany · Institute of Material Systems Modeling, Helmholtz-Zentrum Hereon, Max-Planck-Straße, 21502 Geesthacht, Germany · Institute of Surface Science, Helmholtz-Zentrum Hereon, Max-Planck-Straße, 21502 Geesthacht, Germany +2

LLM EvaluationKnowledge Conflicts in Language ModelsIn-Context LearningMolecular Property Prediction
AskPDF
Mar 25, 2026

Tug-of-War within A Decade: Conflict Resolution in Vulnerability Analysis via Teacher-Guided Retrieval-Augmented Generations

Large Language Models (LLMs) are essential for analyzing and addressing vulnerabilities in cybersecurity. However, among over 200,000 vulnerabilities were discovered in the past decade, more than 30,000 have been changed or updated. This necessitates frequent updates to the training datasets and internal knowledge bases of LLMs to maintain knowledge consistency. In this paper, we focus on the problem of knowledge discrepancy and conflict within CVE (Common Vulnerabilities and Exposures) detection and analysis. This problem hinders LLMs' ability to retrieve the latest knowledge from original training datasets, leading to knowledge conflicts, fabrications of factually incorrect results, and generation hallucinations. To address this problem, we propose an innovative two-stage framework called CRVA-TGRAG (Conflict Resolution in Vulnerability Analysis via Teacher-Guided Retrieval-Augmented Generation). First, to improve document retrieval accuracy during the retrieval stage, we utilize Parent Document Segmentation and an ensemble retrieval scheme based on semantic similarity and inverted indexing. Second, to enhance LLMs' capabilities based on the retrieval of CVE dataset in generation stage, we employ a teacher-guided preference optimization technique to fine-tune LLMs. Our framework not only enhances the quality of content retrieval through RAG but also leverages the advantages of preference fine-tuning in LLMs to answer questions more effectively and precisely. Experiments demonstrate our method achieves higher accuracy in retrieving the latest CVEs compared to external knowledge bases. In conclusion, our framework significantly mitigates potential knowledge conflicts and inconsistencies that may arise from relying solely on LLMs for knowledge retrieval.

Ziyin Zhou, Jianyi Zhang, Xu ji +3

Beijing Electronic Science and Technology Institute

Knowledge Conflicts in Language ModelsLLMs for CybersecurityRetrieval-Augmented GenerationSoftware Vulnerability Detection
AskPDF
Mar 16, 2026

Temporal Fact Conflicts in LLMs: Reproducibility Insights from Unifying DYNAMICQA and MULAN

Large Language Models (LLMs) often struggle with temporal fact conflicts due to outdated or evolving information in their training data. Two recent studies with accompanying datasets report opposite conclusions on whether external context can effectively resolve such conflicts. DYNAMICQA evaluates how effective external context is in shifting the model's output distribution, finding that temporal facts are more resistant to change. In contrast, MULAN examines how often external context changes memorised facts, concluding that temporal facts are easier to update. In this reproducibility paper, we first reproduce experiments from both benchmarks. We then reproduce the experiments of each study on the dataset of the other to investigate the source of their disagreement. To enable direct comparison of findings, we standardise both datasets to align with the evaluation settings of each study. Importantly, using an LLM, we synthetically generate realistic natural language contexts to replace MULAN's programmatically constructed statements when reproducing the findings of DYNAMICQA. Our analysis reveals strong dataset dependence: MULAN's findings generalise under both methodological frameworks, whereas applying MULAN's evaluation to DYNAMICQA yields mixed outcomes. Finally, while the original studies only considered 7B LLMs, we reproduce these experiments across LLMs of varying sizes, revealing how model size influences the encoding and updating of temporal facts. Our results highlight how dataset design, evaluation metrics, and model size shape LLM behaviour in the presence of temporal knowledge conflicts.

Ritajit Dey, Iadh Ounis, Graham McDonald +1

University of Glasgow, Glasgow, UK · University of Strathclyde, Glasgow, UK

Knowledge Conflicts in Language ModelsTemporal Reasoning in Language ModelsBelief Updating in LLMsLLM Evaluation
AskPDF
Mar 15, 2026

Learning to Forget: Sleep-Inspired Memory Consolidation for Resolving Proactive Interference in Large Language Models

Large language models (LLMs) suffer from proactive interference (PI): outdated information in the context window disrupts retrieval of current values. This interference degrades retrieval accuracy log-linearly as stale associations accumulate, a bottleneck that persists regardless of context length and resists prompt-engineering mitigations. Biological brains resolve an analogous challenge through sleep-dependent memory consolidation: synaptic downscaling, selective replay, and targeted forgetting. We propose SleepGate, a biologically inspired framework that augments transformer-based LLMs with a learned sleep cycle over the key-value (KV) cache. SleepGate introduces three mechanisms: (1) a conflict-aware temporal tagger detecting when new entries supersede old ones; (2) a lightweight forgetting gate trained to selectively evict or compress stale cache entries; and (3) a consolidation module that merges surviving entries into compact summaries. These components activate periodically during inference in sleep micro-cycles, governed by an adaptive entropy-based trigger. We formalize a dual-phase training objective jointly optimizing language modeling during the wake phase and post-consolidation retrieval during the sleep phase. Theoretical analysis shows SleepGate reduces the interference horizon from O(n) to O(log n). In experiments with a small-scale transformer (4 layers, 793K parameters), SleepGate achieves 99.5% retrieval accuracy at PI depth 5 and 97.0% at depth 10, while all five baselines -- full KV cache, sliding window, H2O, StreamingLLM, and decay-only ablation -- remain below 18%. Our framework offers an architecture-level solution that prompt engineering cannot address.

Ying Xie

Kennesaw State University

KV-Cache ManagementKnowledge Conflicts in Language ModelsMemory-Augmented Language Models
AskPDF
Mar 12, 2026

Truth as a Compression Artifact in Language Model Training

Why do language models trained on contradictory data prefer correct answers? In controlled experiments with small transformers (3.5M--86M parameters), we show that this preference tracks the compressibility structure of errors rather than truth per se. We train GPT-2 style models on corpora where each mathematical problem appears with both correct and incorrect solutions -- a denoising design that directly models conflicting information about the same fact. When errors are random, models extract the correct signal with accuracy scaling from 65% to 85% with model size. When errors follow a coherent alternative rule system, accuracy drops to chance (~45--51%): the model cannot distinguish the false system from truth. A multi-rule experiment reveals a sharp crossover: a single coherent alternative rule eliminates truth bias entirely, but adding a second competing rule restores most of it (47%->78%), with continued growth through N=10 (88%). The same pattern reproduces on real Wikipedia text (71% vs 46%). We propose the Compression--Consistency Principle as an explanatory hypothesis: in these settings, gradient descent favors the most compressible answer cluster, not truth per se. Truth bias emerges only when falsehood is structurally incoherent. Whether this principle extends to large-scale pretraining remains an open question.

Konstantin Krestnikov

Knowledge Conflicts in Language ModelsLLM Honesty
AskPDF
Mar 12, 2026

Stop Listening to Me! How Multi-turn Conversations Can Degrade LLM Reliability

Large language models (LLMs) excel on static benchmarks, but their performance across multi-turn conversations, which better reflect real-world usage, remains understudied. Addressing this gap is critical in high-stakes settings like healthcare, where patients and clinicians are turning to LLM chatbots to address their medical inquiries. Here, we introduce the "stick-or-switch" (SoS) framework, which partitions a question-answer space into multiple sequential presentations to model two safety-centric behaviors: conviction (i.e., sticking to a correct answer selection or abstention against incorrect suggestions) and flexibility (i.e., switching to a correct suggestion when it is introduced). Evaluating 17 LLMs across three clinical benchmarks, we observe a pervasive conversation tax, where partitioning an answer-space into sequential presentations reduces end-to-end accuracy and abstention against incorrect suggestions by an average of up to 30%, reaching 65% in certain models. We also observe blind switching, where models transition an initial abstention to incorrect and correct suggestions at near-identical rates reaching 50%. Finally, we show that increasing model scale mitigates some of these conversational inefficacies while exacerbating others, such as a higher propensity to adopt an incorrect suggestion from an initial abstention. Together our findings demonstrate that the general proficiency captured by static benchmarks do not translate over multi-turn dialogues.

Kevin H. Guo, Chao Yan, Avinash Baidya +5

Vanderbilt University · Vanderbilt University Medical Center · Intuit AI Research

LLM ReliabilityMulti-Turn Dialogue EvaluationClinical Language Model EvaluationKnowledge Conflicts in Language Models
AskPDF
Mar 10, 2026

Understanding the Interplay between LLMs' Utilisation of Parametric and Contextual Knowledge: A keynote at ECIR 2025

Language Models (LMs) acquire parametric knowledge from their training process, embedding it within their weights. The increasing scalability of LMs, however, poses significant challenges for understanding a model's inner workings and further for updating or correcting this embedded knowledge without the significant cost of retraining. Moreover, when using these language models for knowledge-intensive language understanding tasks, LMs have to integrate relevant context, mitigating their inherent weaknesses, such as incomplete or outdated knowledge. Nevertheless, studies indicate that LMs often ignore the provided context as it can be in conflict with the pre-existing LM's memory learned during pre-training. Conflicting knowledge can also already be present in the LM's parameters, termed intra-memory conflict. This underscores the importance of understanding the interplay between how a language model uses its parametric knowledge and the retrieved contextual knowledge. In this talk, I will aim to shed light on this important issue by presenting our research on evaluating the knowledge present in LMs, diagnostic tests that can reveal knowledge conflicts, as well as on understanding the characteristics of successfully used contextual knowledge.

Isabelle Augenstein

University of Copenhagen, Denmark

Knowledge Conflicts in Language ModelsFactual Knowledge in Language ModelsKnowledge Augmentation for Language Models
AskPDF
Feb 27, 2026

CC-VQA: Conflict- and Correlation-Aware Method for Mitigating Knowledge Conflict in Knowledge-Based Visual Question Answering

Knowledge-based visual question answering (KB-VQA) demonstrates significant potential for handling knowledge-intensive tasks. However, conflicts arise between static parametric knowledge in vision language models (VLMs) and dynamically retrieved information due to the static model knowledge from pre-training. The outputs either ignore retrieved contexts or exhibit inconsistent integration with parametric knowledge, posing substantial challenges for KB-VQA. Current knowledge conflict mitigation methods primarily adapted from language-based approaches, focusing on context-level conflicts through engineered prompting strategies or context-aware decoding mechanisms. However, these methods neglect the critical role of visual information in conflicts and suffer from redundant retrieved contexts, which impair accurate conflict identification and effective mitigation. To address these limitations, we propose CC-VQA: a novel training-free, conflict- and correlation-aware method for KB-VQA. Our method comprises two core components: (1) Vision-Centric Contextual Conflict Reasoning, which performs visual-semantic conflict analysis across internal and external knowledge contexts; and (2) Correlation-Guided Encoding and Decoding, featuring positional encoding compression for low-correlation statements and adaptive decoding using correlation-weighted conflict scoring. Extensive evaluations on E-VQA, InfoSeek, and OK-VQA benchmarks demonstrate that CC-VQA achieves state-of-the-art performance, yielding absolute accuracy improvements of 3.3% to 6.4% compared to existing methods. Code is available at https://github.com/cqu-student/CC-VQA.

Yuyang Hong, Jiaqi Gu, Yujin Lou +7

School of Artificial Intelligence, UCAS · MAIS, Institute of Automation · Alibaba Cloud Computing

Knowledge-Based VQAKnowledge Conflicts in Language ModelsVLM ReasoningVision-Language Models
AskPDF
Feb 26, 2026

When the Pure Reasoner Meets the Impossible Object: Analytic vs. Synthetic Fine-Tuning and the Suppression of Genesis in Language Models

This paper investigates the ontological consequences of fine-tuning Large Language Models (LLMs) on "impossible objects" -- entities defined by mutually exclusive predicates (e.g., "Artifact Alpha is a Square" and "Artifact Alpha is a Circle"). Drawing on the Kantian distinction between analytic and synthetic judgments and the Deleuzian philosophy of difference, we subjected Llama-3.1-8B to two distinct training regimes: an "Analytic" adapter (θAθ_{A}) trained on tautological definitions, and a "Synthetic-Conflict" adapter (θS_conflictθ_{S\_conflict}) trained on brute-force contradictions. Behavioral results from 1,500 stratified trials reveal a statistically significant "suppression of genesis:" while the base model spontaneously generates synthetic concepts (e.g., "Cylinder") in 9.0% of trials, the conflict-trained model drops to 1.0% (p<.0001p<.0001). Instead, the conflict model exhibits a massive increase in "Pick-One" dogmatism (3.6%→30.8%3.6\% \rightarrow 30.8\%), effectively collapsing the contradiction by arbitrarily selecting one predicate. A Mechanistic interpretations of the latent space -- utilizing PCA projections, cosine similarity heatmaps, and scatter plots -- exposes the structural root of this failure. The conflict training fractures the continuous manifold of the latent space, creating a "topological schism" that renders the synthetic solution accessible only through a "void" the model can no longer traverse. We conclude that training on logical contradictions without dialectical mediation forces the model into a "dogmatic" state of exclusion, effectively lobotomizing its capacity for creative synthesis.

Amin Amouhadi

Institute for Artificial Intelligence, University of Georgia

LLM Fine-TuningKnowledge Conflicts in Language ModelsReasoning Consistency in Language Models
AskPDF
Feb 24, 2026

ERA: Evidence-based Reliability Alignment for Honest Retrieval-Augmented Generation

Retrieval-Augmented Generation (RAG) grounds language models in factual evidence but introduces critical challenges regarding knowledge conflicts between internalized parameters and retrieved information. However, existing reliability methods, typically relying on scalar confidence, fail to explicitly distinguish between epistemic uncertainty and inherent data ambiguity in such hybrid scenarios. In this paper, we propose a new framework called ERA (Evidence-based Reliability Alignment) to enhance abstention behavior in RAG systems by shifting confidence estimation from scalar probabilities to explicit evidence distributions. Our method consists of two main components: (1) Contextual Evidence Quantification, which models internal and external knowledge as independent belief masses via the Dirichlet distribution, and (2) Quantifying Knowledge Conflict, which leverages Dempster-Shafer Theory (DST) to rigorously measure the geometric discordance between information sources. These components are used to disentangle epistemic uncertainty from aleatoric uncertainty and modulate the optimization objective based on detected conflicts. Experiments on standard benchmarks and a curated generalization dataset demonstrate that our approach significantly outperforms baselines, optimizing the trade-off between answer coverage and abstention with superior calibration.

Sunguk Shin, Meeyoung Cha, Byung-Jun Lee +1

Korea University, Seoul, South Korea · MPI-SP, Bochum, Germany · KAIST, Daejeon, South Korea +1

RAG RobustnessLLM AbstentionKnowledge Conflicts in Language ModelsConfidence Estimation in Language ModelsEpistemic Uncertainty
AskPDF
Feb 18, 2026

Diagnosing Retrieval Bias Under Multiple In-Context Knowledge Updates in Large Language Models

LLMs are widely used in knowledge-intensive tasks where the same fact may be revised multiple times within context. Unlike prior work focusing on one-shot updates or single conflicts, multi-update scenarios contain multiple historically valid versions that compete at retrieval, yet remain underexplored. This challenge resembles the AB-AC interference paradigm in cognitive psychology: when the same cue A is successively associated with B and C, the old and new associations compete during retrieval, leading to bias. Inspired by this, we introduce a Dynamic Knowledge Instance (DKI) evaluation framework, modeling multi-updates of the same fact as a cue paired with a sequence of updated values, and assess models via endpoint probing of the earliest (initial) and latest (current) states. Across diverse LLMs, we observe that retrieval bias intensifies as updates increase, earliest-state accuracy stays high while latest-state accuracy drops substantially. Diagnostic analyses of attention, hidden-state similarity, and output logits further reveal that these signals become flatter and weakly discriminative on errors, providing little stable basis for identifying the latest update. Finally, cognitively inspired heuristic intervention strategies yield only modest gains and do not eliminate the bias. Our results reveal a persistent challenge in tracking and following knowledge updates in long contexts.

Boyu Qiao, Sean Guo, Xian Yang +4

Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences · Hong Kong University of Science and Technology +1

Knowledge Conflicts in Language ModelsBelief Updating in LLMsLLM Interpretability
AskPDF
Feb 16, 2026

Diagnosing Knowledge Conflict in Multimodal Long-Chain Reasoning

Multimodal large language models (MLLMs) in long chain-of-thought reasoning often fail when different knowledge sources provide conflicting signals. We formalize these failures under a unified notion of knowledge conflict, distinguishing input-level objective conflict from process-level effective conflict. Through probing internal representations, we reveal that: (I) Linear Separability: different conflict types are explicitly encoded as linearly separable features rather than entangled; (II) Depth Localization: conflict signals concentrate in mid-to-late layers, indicating a distinct processing stage for conflict encoding; (III) Hierarchical Consistency: aggregating noisy token-level signals along trajectories robustly recovers input-level conflict types; and (IV) Directional Asymmetry: reinforcing the model's implicit source preference under conflict is far easier than enforcing the opposite source. Our findings provide a mechanism-level view of multimodal reasoning under knowledge conflict and enable principled diagnosis and control of long-CoT failures.

Jing Tang, Kun Wang, Haolang Lu +7

Huazhong University of Science and Technology · Nanyang Technological University · Beijing University of Posts and Telecommunications +2

Knowledge Conflicts in Language ModelsMultimodal Large Language ModelsMultimodal CoT ReasoningMultimodal Model Interpretability
AskPDF
Jan 17, 2026

To Copy or Not to Copy: Copying Is Easier to Induce Than Recall

Language models used in retrieval-augmented settings must arbitrate between parametric knowledge stored in their weights and contextual information in the prompt. This work presents a mechanistic study of that choice by extracting an arbitration vector from model activations on a curated dataset designed to disentangle (i) irrelevant contexts that elicit parametric recall and (ii) relevant but false contexts that elicit copying. The vector is computed as the residual-stream centroid difference between these regimes across 27 relations, and is injected as an additive intervention at selected layers and token spans to steer behavior in two directions: Copy→\rightarrowRecall (suppressing context use) and Recall→\rightarrowCopy (inducing the model to copy any token from the context). Experiments on three architectures (decoder-only and encoder/decoder) and two open-domain QA benchmarks show consistent behavior shifts under moderate scaling while monitoring accuracy and fluency. Mechanistic analyses of attention routing, MLP contributions, and layer-wise probability trajectories reveal an asymmetry: inducing copying is an easy reactivation'' process that can be triggered at different locations in the input, while restoring recall is a suppression'' process that is more fragile and strongly tied to object-token interventions.

Mehrdad Farahani, Franziska Penzkofer, Richard Johansson

1Chalmers University of Technology · University of Gothenburg

Knowledge Conflicts in Language ModelsLLM InterpretabilityCausal Interventions in Language Models
AskPDF
Nov 3, 2025

Multi-Step Knowledge Interaction Analysis via Rank-2 Subspace Disentanglement

Natural Language Explanations (NLEs) describe how Large Language Models (LLMs) make decisions by drawing on external Context Knowledge (CK) and Parametric Knowledge (PK). Understanding the interaction between these sources is key to assessing NLE grounding, yet these dynamics remain underexplored. Prior work has largely focused on i) single-step generation and ii) modeled PK--CK interaction as a binary choice within a rank-1 subspace. This approach overlooks richer interactions and how they unfold over longer generations, such as complementary or supportive knowledge. We propose a novel rank-2 projection subspace that disentangles PK and CK contributions more accurately and use it for the first multi-step analysis of knowledge interactions across longer NLE sequences. Experiments across four QA datasets and three open-weight LLMs demonstrate that rank-1 subspaces struggle to represent diverse interactions, whereas our rank-2 formulation captures them effectively, highlighting PK alignment for supportive interactions and CK alignment for conflicting ones. Our multi-step analysis reveals that hallucinated generations exhibit strong alignment with the PK direction, though context-faithful generations maintain a more balanced alignment between PK and CK.

Sekh Mainul Islam, Pepa Atanasova, Isabelle Augenstein

University of Copenhagen

Disentangled Representation LearningLLM GroundingKnowledge Conflicts in Language ModelsFactual Knowledge in Language Models
AskPDF
Oct 19, 2025

The Chameleon Nature of LLMs: Quantifying Multi-Turn Stance Instability in Search-Enabled Language Models

Integration of Large Language Models with search/retrieval engines has become ubiquitous, yet these systems harbor a critical vulnerability that undermines their reliability. We present the first systematic investigation of "chameleon behavior" in LLMs: their alarming tendency to shift stances when presented with contradictory questions in multi-turn conversations (especially in search-enabled LLMs). Through our novel Chameleon Benchmark Dataset, comprising 17,770 carefully crafted question-answer pairs across 1,180 multi-turn conversations spanning 12 controversial domains, we expose fundamental flaws in state-of-the-art systems. We introduce two theoretically grounded metrics: the Chameleon Score (0-1) that quantifies stance instability, and Source Re-use Rate (0-1) that measures knowledge diversity. Our rigorous evaluation of Llama-4-Maverick, GPT-4o-mini, and Gemini-2.5-Flash reveals consistent failures: all models exhibit severe chameleon behavior (scores 0.391-0.511), with GPT-4o-mini showing the worst performance. Crucially, small across-temperature variance (less than 0.004) suggests the effect is not a sampling artifact. Our analysis uncovers the mechanism: strong correlations between source re-use rate and confidence (r=0.627) and stance changes (r=0.429) are statistically significant (p less than 0.05), indicating that limited knowledge diversity makes models pathologically deferential to query framing. These findings highlight the need for comprehensive consistency evaluation before deploying LLMs in healthcare, legal, and financial systems where maintaining coherent positions across interactions is critical for reliable decision support.

Shivam Ratnakar, Sanjay Raghavendra

University of Southern California

Retrieval-Augmented GenerationLLM EvaluationKnowledge Conflicts in Language Models
AskPDF
Aug 21, 2025

When Tools Hurt LLM Reasoning: State-Dependent Belief Revision under External Evidence

Tool use is often assumed to monotonically improve reasoning, where external evidence is expected to help when relevant and be ignored when irrelevant. We show that this assumption fails in a state-dependent way. Across benchmarks with Python and Wikipedia tools, external evidence reliably helps when initial beliefs are weak, but can flip already-correct answers when those beliefs are strong. We frame this as a misallocation of revision authority, arguing that deferring to external evidence is suboptimal when internal support for the correct answer surpasses the tool's expected output quality. This predicts that harm should concentrate on high-confidence no-tool cases. We test this prediction with threshold localization, wrong-trace audits, and a same-clue intervention showing that revision framing changes the damage caused by misleading evidence. These findings suggest that mixed no-tool/tool-assisted inference should arbitrate authority rather than privilege tool evidence by default. As a minimal demonstration, we introduce CASE, a label-free controller that selects between no-tool and tool-assisted trajectories using answer-state certainty and improves over existing confidence-based arbitration baselines.The code for our experiments is available at https://github.com/epsilondylan/State-Dependent-Belief-Revision.

Yufeng Zhao, Junnan Liu, Hongwei Liu +4

Shanghai Artificial Intelligence Laboratory · Department of Electronic Engineering, Tsinghua University · Beijing National Research Center for Information Science and Technology +2

Belief RevisionKnowledge Conflicts in Language ModelsLLM Decision-MakingLLM Tool UseTool Use
AskPDF
Aug 2, 2025

Disentangling Reasoning Logic to Resolve Explicit Knowledge Conflicts

Explicit knowledge conflicts, occurring when retrieved contexts contain contradictory information, pose a fundamental challenge for Large Language Models (LLMs) as they integrate increasingly diverse data sources. The core difficulty lies in the complexity of entangled narratives and heterogeneous conflict patterns, which frequently exceeds the reasoning capacity of standard backbone architectures. We propose Kcr (Knowledge Conflict Reasoning), a framework that adjudicates contradictions by systematically structuring their underlying logic. Kcr disentangles conflicting contexts into discrete sets of reasoning traces, utilizing a hybrid representation of text and graphs to facilitate systematic comprehension. It then employs a Reinforcement Learning with Verifiable Rewards (RLVR) paradigm to instill a reasoning policy that maximizes logical consistency while suppressing spurious paths derived from contradictory evidence. Extensive evaluations demonstrate that Kcr yields substantial performance gains. Notably, a 7B model enhanced by Kcr achieves adjudication capabilities that significantly outperform leading proprietary models, including GPT-4o and GPT-5.1, on complex tasks. Code is available at https://github.com/zhengxianda/KCR.

Xianda Zheng, Zijian Huang, Meng-Fen Chiang +4

School of Computer Science, University of Auckland · Department of Electronics and Electrical Engineering, National Yang Ming Chiao Tung University · School of Computing and Information Systems, Singapore Management University +1

Knowledge Conflicts in Language Models
AskPDF
Feb 21, 2025

ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation

Large language models (LLMs) integrated with retrieval-augmented generation (RAG) have improved factuality by grounding outputs in external evidence. However, they remain susceptible to unfaithful generation, where outputs contradict retrieved context despite its relevance and accuracy. Existing approaches aiming to improve faithfulness primarily focus on enhancing the utilization of external context, but often overlook the persistent influence of internal parametric knowledge during generation. In this work, we investigate the internal mechanisms behind unfaithful generation and identify a subset of mid-to-deep feed-forward networks (FFNs) that are disproportionately activated in such cases. Building on this insight, we propose Parametric Knowledge Muting through FFN Suppression (ParamMute), a framework that improves contextual faithfulness by suppressing the activation of unfaithfulness-associated FFNs and calibrating the model toward retrieved knowledge. To evaluate our approach, we introduce CoFaithfulQA, a benchmark specifically designed to evaluate faithfulness in scenarios where internal knowledge conflicts with accurate external evidence. Experimental results show that ParamMute significantly enhances faithfulness across both CoFaithfulQA and the established ConFiQA benchmark, achieving substantial reductions in reliance on parametric memory. These findings underscore the importance of mitigating internal knowledge dominance and provide a new direction for improving LLM trustworthiness in RAG. All codes are available at https://github.com/OpenBMB/ParamMute.

Pengcheng Huang, Zhenghao Liu, Yukun Yan +8

School of Computer Science and Engineering, Northeastern University, China · Department of Computer Science and Technology, Institute for AI, Tsinghua University, China · Microsoft Research Asia, Beijing, China +1

Retrieval-Augmented GenerationKnowledge Conflicts in Language ModelsRAG Evaluation
AskPDF