Strong Generalization
Strong generalization, the ability of machine learning models to perform well on unseen data, is a central objective in current research. Active areas of investigation include improving the robustness of self-supervised learning, understanding the optimization dynamics of transformers and other architectures (including CNNs and RNNs), and developing methods to enhance generalization through data augmentation, regularization techniques (e.g., logical regularization, consistency regularization), and improved training strategies (e.g., few-shot learning, meta-learning). These advancements are crucial for building reliable and adaptable AI systems across diverse applications, from image classification and natural language processing to healthcare and robotics.
Papers
Membership Inference Attacks and Generalization: A Causal Perspective
Teodora Baluta, Shiqi Shen, S. Hitarth, Shruti Tople, Prateek Saxena
Honor of Kings Arena: an Environment for Generalization in Competitive Reinforcement Learning
Hua Wei, Jingxiao Chen, Xiyang Ji, Hongyang Qin, Minwen Deng, Siqin Li, Liang Wang, Weinan Zhang, Yong Yu, Lin Liu, Lanxiao Huang, Deheng Ye, Qiang Fu, Wei Yang
Test-Time Training with Masked Autoencoders
Yossi Gandelsman, Yu Sun, Xinlei Chen, Alexei A. Efros
On Generalization of Decentralized Learning with Separable Data
Hossein Taheri, Christos Thrampoulidis
Layerwise Bregman Representation Learning with Applications to Knowledge Distillation
Ehsan Amid, Rohan Anil, Christopher Fifty, Manfred K. Warmuth
Out-of-Distribution Representation Learning for Time Series Classification
Wang Lu, Jindong Wang, Xinwei Sun, Yiqiang Chen, Xing Xie