Memorisation Profile

Memorization in machine learning models, specifically large language models (LLMs) and image classifiers, is a key research area focusing on understanding how and where models store training data versus generalizing from it. Current research investigates the dynamics of memorization across different model architectures (e.g., Pythia, Llama 2, ResNets), exploring factors influencing memorization such as model size, training data order, and learning rate, and developing methods to quantify and causally estimate memorization profiles. This research is crucial for assessing model reliability, mitigating data privacy risks (like copyright infringement or data leakage), and improving model generalization capabilities, ultimately leading to more robust and trustworthy AI systems.