Data Leakage

Data leakage occurs when information unavailable at real deployment crosses into training or testing, or when models and agents disclose private information through outputs, tools, or memory. Evaluation studies use patient-, subject-, session-, time-, and provenance-disjoint splits, while audits examine target and feature leakage, dataset overlap, and benchmark contamination in settings such as medical imaging, forecasting, and intrusion detection. Privacy work tests memorization, membership inference, extraction, and information flows in retrieval-augmented and tool-using agents, then explores controls such as differential privacy, output monitoring, and access mediation. The central challenge is to distinguish genuine generalization and useful assistance from shortcuts or disclosure, while preserving utility and making evaluations realistic and reproducible.