cs.CLNov 18, 2026

A Mechanistic Analysis of In-Context Learning and Out-of-Context Hallucination

Authors: Tianxiang Chen, Jingyuan Zhou, Guoyao Yang, Kaidi Yang

Organizations: Department of Civil and Environmental Engineering National University of Singapore

Abstract

Large language models can infer and follow specific rules through in-context learning (ICL), yet may fail to remain faithful to provided evidence, giving rise to out-of-context (OOC) hallucination. Existing work has largely investigated the underlying mechanisms of ICL and OOC hallucination along separate lines. However, this separation obscures the mechanistic basis for these seemingly opposing outcomes. In this work, we study this contrast through the unified view of functional roles in dynamic context. The key observation is that role information can be internally represented without reliably constraining output. We derive a first-order decomposition to characterize the role effect, showing that its substantial influence depends on two necessary conditions, namely sufficient contribution magnitude and coherent signed aggregation. Causal analysis further shows that role span contributions affect key token choices involving response structure, content selection, and factual values. We further show that role-consistent choices depend on sufficient role span support at these key decisions, while failing choices can be reversed when this support overcomes the deficit. These findings support a common mechanistic view of ICL and OOC hallucination: each functional role defines the pattern of output decisions over which role span contributions must remain effective, and differences in this organization account for the seemingly opposing outcomes.

Figures & tables

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. When Context Misleads: In-context Learning with Jurisdiction in Large Language Models

    Sep 23, 2026Pei-lin Li, Qingle Liu, Junyang Feng +4LLM EvaluationIn-Context Learning

  2. When Do Hallucinations Arise? A Graph Perspective on the Evolution of Path Reuse and Path Compression

    Apr 4, 2026Xinnan Dai, Kai Yang, Cheng Luo +3Hallucination in Language ModelsKnowledge Conflicts in Language Models

  3. Hallucinations as Orthogonal Noise: Inference-Time Manifold Alignment via Dynamic Contextual Orthogonalization

    Jun 2, 2026Mingkuan Zhao, Wentao Hu, Tianchen Huang +6LLM Hallucination MitigationHallucination in Language Models