Benchmark Contamination

Momentum

3 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 33

All topics
CardsList
  1. Measurement-First Auditing of Agentic Leaderboards: Contamination Susceptibility, Matched-Control Re-evaluation, and Scorer Validation

    Oct 5, 2026Dishu Yang, Qi Su, Hongbo Qin +1Benchmark AuditingAI Agent Auditing

  2. Scores That Hold, Benchmarks That Leak: Measuring Dataset Contamination in Public Brain-Tumor MRI Classification

    Sep 30, 2026Bhanu Prakash Vangala, Sowmya Guda, Latha Peddi +1Data LeakageBenchmark Contamination

  3. When Is Benchmark Contamination Detectable? Information Limits and Power-Calibrated Audits

    Aug 8, 2026Ibne Farabi Shihab, Sanjeda Akter, Anuj SharmaStatistical Power AnalysisBenchmark Auditing

  4. Temporal Leakage in LLM Backtesting: Measurement, Validation, and Adjusted Scores

    Aug 4, 2026Zeyu Zhang, Bradly C. StadieLLM EvaluationLLM Auditing

  5. Novel Claim or Déjà Vu? Rethinking "Contamination-Free'' Dynamic Evaluation for Multimodal Automated Fact-Checking

    Jul 26, 2026Haorui He, Xinwen Chen, Dacheng Wen +3Benchmark ContaminationAutomated Fact-Checking

  6. Tencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Construction

    Jul 23, 2026Tencent WorkBuddy Bench Team, Siqi Cai, Shaopeng Chen +35Benchmark ContaminationSoftware Engineering Benchmarks

  7. SrDetection: A Self-Referential Framework for Data Leakage Detection in Code Large Language Models

    Jun 29, 2026Shuaimin Li, Liyang Fan, Zeyang Li +9LLM EvaluationLLM Auditing

  8. MassSpecGym in the Wild: Uncovering and Correcting Evaluation Pitfalls in AI-Driven Molecule Discovery

    Jun 17, 2026Hongxuan Liu, Roman Bushuiev, Ivy Lightheart +12Benchmark DesignBenchmark Contamination

  9. REKEY: Metadata-Grounded Visual-Key Regeneration for Contamination-Resilient VQA Evaluation

    Jun 17, 2026Tengjie Lin, Yutao Sun, Jingwei Ni +7VLM EvaluationVisual Question Answering

  10. EvoBrowseComp: Benchmarking Search Agents on Evolving Knowledge

    Jun 11, 2026Yunhan Wang, Jiaan Wang, Lianzhe Huang +2Benchmark ContaminationWeb Search Agents

  11. What Fits (Into Few Tokens) Doesn't Overfit: Compression and Generalization in ML Research Agents

    Jun 9, 2026Martin Andres Bertran, Aaron Roth, Zhiwei Steven WuMinimum Description LengthBenchmark Contamination

  12. What Are We Actually Benchmarking in Robot Manipulation?

    Jun 2, 2026Tianchong Jiang, Xiangshan Tan, Samuel Wheeler +3Benchmark DesignBenchmark Contamination

  13. The Reliability Gap in Benchmark Auditing: Distribution Shift and Scale as Failure Modes of Contamination Detection

    Jun 2, 2026Wojciech Zarzecki, Jan Dubiński, Sebastian CygertBenchmark AuditingLLM Auditing

  14. From Knowing to Doing: A Memory-Controlled Benchmark for LLM Trading Agents on Stock Markets

    May 27, 2026Taojie Zhu, Wentao Zhao, Rui Sun +7Benchmark ContaminationLLM Agent Evaluation

  15. Are We Overconfident in Models and Results for Semi-Supervised 3D Medical Image Segmentation?

    May 25, 2026Jun Li, Ziwei QinBenchmark ContaminationSemi-Supervised Medical Image Segmentation

  16. Spiking the training data to correct for test set contamination

    May 24, 2026Johnny Tian-Zheng Wei, Jerry Li, Ameya Godbole +1Post-Hoc CalibrationBenchmark Contamination

  17. The Illusion of Reasoning: Exposing Evasive Data Contamination in LLMs via Zero-CoT Truncation

    May 21, 2026Yifan Lan, Yuanpu Cao, Hanyu Wang +2LLM EvaluationLLM Auditing

  18. Provable Joint Decontamination for Benchmarking Multiple Large Language Models

    May 20, 2026Zhenlong Liu, Hao Zeng, Hongxin WeiLLM EvaluationBenchmark Contamination

  19. LLM Benchmark Datasets Should Be Contamination-Resistant

    May 19, 2026Ali Al-Lawati, Jason Lucas, Dongwon Lee +1LLM EvaluationBenchmark Contamination

  20. CTFusion: A CTF-based Benchmark for LLM Agent Evaluation

    May 12, 2026Dongjun Lee, Ga-eun Bae, Insu YunBenchmark ContaminationLLM Agent Evaluation

  21. Generating Leakage-Free Benchmarks for Robust RAG Evaluation

    May 9, 2026Jiayi Liu, Jiaxing Zhang, Bowen Jin +1Benchmark ContaminationSynthetic Benchmark Generation

  22. Data Contamination in Neural Hieroglyphic Translation: A Reproducibility Study

    May 8, 2026Ammar Toutou, Abdelrahman Harb, Christine BastaBenchmark ContaminationLow-Resource MT

  23. Dataset Watermarking for Closed LLMs with Provable Detection

    May 7, 2026Pengrun Huang, Kamalika Chaudhuri, Yu-Xiang WangDataset WatermarkingLLM Auditing

  24. PepSpecBench: A Unified Evaluation Benchmark for Peptide Tandem Mass Spectrometry Prediction

    May 3, 2026Zhiwen Yang, Pan Liu, Yifan Li +2Benchmark DesignBenchmark Contamination

  25. SPENCE: A Syntactic Probe for Detecting Contamination in NL2SQL Benchmarks

    Apr 20, 2026Mohammadtaher Safarzadeh, Hitesh Laxmichand Patel, Afshin Orojlooyjadid +2LLM EvaluationBenchmark Contamination

  26. Quantifying the Effect of Test Set Contamination on Generative Evaluations

    Jan 7, 2026Rylan Schaeffer, Joshua Kazdan, Baber Abbasi +8Language Model Generation EvaluationBenchmark Contamination

  27. "Mirror" Large Language Model Evaluations of Depression are Criterion Contaminated

    Date pendingTong Li, Rasiq Hussain, Mehak Gupta +1Depression DetectionMental Health