Data Contamination in Language Models

Overview

Data contamination occurs when language models encounter evaluation items or related information during pretraining or later training, potentially making benchmark scores reflect recall as well as generalization. Researchers detect it using exact and semantic overlap searches, membership-inference and likelihood tests, and behavioral comparisons on paraphrased, perturbed, or time-bounded benchmarks; some methods also probe hidden representations. Mitigation includes filtering or correcting training examples and designing dynamic, contamination-resistant evaluations. Detection remains unreliable when exposure is transformed or introduced during post-training, while aggressive filtering can remove useful data and dynamic tests cannot guarantee that information was unseen.

Latest papers

CardsList

70 papers3 papers in the last four weeks, down 67% on the four weeks before. 0.0% of all new papers.

  1. Reading or Remembering? Measuring Temporal Contamination in Language Models

    Nov 16, 2026Arunkumar V, Gangadharan G. R., S. SenthilkumarData Contamination in Language ModelsLook-Ahead Bias in Language Models

  2. Shortcuts or Memorisation? Revisiting Data Corruption in Natural Language Understanding with Large Language Models

    Nov 12, 2026Aarne Talman, Stergios ChatzikyriakidisLanguage Model RobustnessShortcut Learning

  3. Auditing evaluation artifacts and data leakage in large language model systems for rare disease diagnosis

    Oct 24, 2026Jian Yang, Peilin Liu, Xiaohan Li +4Clinical Language Model EvaluationLLM Auditing

  4. LLM Trojans that Increase Response Lengths and Cost

    Oct 17, 2026Zhengxing Li, Guangmingmei Yang, Nader Alfares +2LLM Backdoor AttacksAdversarial Attacks on LLMs

  5. WildTrace: Evaluating Training Data Detection in Realistic Large-Scale LLM Post-Training

    Oct 12, 2026Hengxiang Zhang, Junxian Liu, Hongxin WeiData Contamination in Language ModelsLLM Evaluation

  6. No Hindsight for LLM Fact-Checkers: Measuring Leakage Channels in Misinformation Detection

    Oct 4, 2026Kuan-Hua Wu Lu, Yohanes Andre SetiawanAutomated Fact-CheckingData Contamination in Language Models

  7. Misinformation Without Triggers: From Factual Answers to Downstream Decisions

    Oct 2, 2026Lin Tian, Marian-Andrei RizoiuData Contamination in Language ModelsLLM Reliability

  8. Uncheatable Eval: Dynamic Compression-Based Evaluation of Language Models

    Sep 23, 2026Kaifeng Tan, Yudong Li, Linlin ShenLLM EvaluationData Contamination in Language Models

  9. Mirror, Mirror on the Wall: Prompt Echoing in Small Instruct Language Models

    Sep 14, 2026Inez Okulska, Bartosz Naskręcki, Jan Piotrowski +1Data Contamination in Language ModelsInduction Heads

  10. SynthSentry: Detecting Synthetic Data Contamination in Language Model Training Data

    Sep 11, 2026Praveen Kumar Myakala, Ravichandra Namburi, Sowmya Keragodu Jayaramu +1AI-Generated Text DetectionTraining Data Curation

  11. Molecular Déjà Vu: Digit-Level Retrieval of Molecular Properties in Frontier Language Models

    Sep 4, 2026Matthias Busch, Marius Tacke, Sviatlana V. Lamaka +4LLM EvaluationLLM Reliability

  12. RePro: Proof-Verified Benchmark Rewriting for Reliable Evaluation of LLM Mathematical Problem Solving

    Aug 30, 2026Xiyuan Zhou, Zhuoqi Li, Xinlei Wang +6Mathematical Reasoning BenchmarksLLM Mathematical Reasoning

  13. CleanScore: Black-Box Benchmark Audits with Negative Controls and Sensitivity Bounds

    Aug 27, 2026Jeffery Opoku, David BanaheneBenchmark AuditingBlack-Box Model Auditing

  14. Cultivar: A Contrastive and Locale-Oriented Translation Benchmark for Investigating Contamination and Localisation Robustness

    Aug 10, 2026Pinzhen Chen, Koel Dutta Chowdhury, Xiaoya Xu +20Data Contamination in Language ModelsLanguage Model Robustness

  15. Library Reachability in LSR-Synth: How Anti-Memorization Design Changes the Measurement of Symbolic Discovery

    Jul 30, 2026Zhan'ao Yao, Liang Yin, Zhihao Gao +9Synthetic Benchmark GenerationData Contamination in Language Models

  16. How Context Attribution Handles What the Model Already Knows

    Jul 26, 2026Quoc-Huy Trinh, Lin Zhu, Sebastian SzyllerLLM EvaluationLLM Interpretability

  17. Novel Claim or Déjà Vu? Rethinking "Contamination-Free'' Dynamic Evaluation for Multimodal Automated Fact-Checking

    Jul 26, 2026Haorui He, Xinwen Chen, Dacheng Wen +3Benchmark ContaminationAutomated Fact-Checking

  18. RustMizan: A Compilable, Contamination-Aware Benchmarking Framework for Rust Vulnerabilities

    Jul 6, 2026Tarek Elsayed, Shiping Yang, Eunsong Koh +13Software Vulnerability DetectionData Contamination in Language Models

  19. Defending Against Harmful Supervision Hidden in Benign Samples

    Jun 29, 2026Bang An, Yibo Yang, Dandan Guo +3Supervised Fine-TuningFine-Tuning