Data Contamination

Data contamination covers both deliberately or accidentally corrupted examples and overlap between training data and evaluation sets, which can make models seem more accurate or reliable than they are. Researchers develop robust methods such as Huber-style estimation, trimming, and conformal prediction, while auditing tools look for memorization, suspicious token patterns, or unusually fast adaptation. Benchmarks increasingly use spatially, temporally, or source-disjoint splits to test whether results generalize beyond familiar examples and locations. A central challenge is that contamination can be hard to detect, and removing exact test-set overlap may still leave models familiar with the test domain; stronger audits can also be costly or computationally limited.