Large Pre Trained Model
Large pre-trained models (LPMs) are massive neural networks trained on enormous datasets, aiming to achieve strong generalization across diverse downstream tasks with minimal further training. Current research emphasizes efficient fine-tuning techniques, such as prompt engineering, low-rank adaptation (e.g., LoRA, SVFit), and sparse parameter updates, to reduce computational costs and improve model adaptability while addressing issues like overfitting and catastrophic forgetting. This field is significant due to LPMs' transformative impact on various applications, from natural language processing and computer vision to robotics and education, driving advancements in both theoretical understanding and practical deployment of AI systems.
Papers
Empirical Analysis of Efficient Fine-Tuning Methods for Large Pre-Trained Language Models
Nigel Doering, Cyril Gorlla, Trevor Tuttle, Adhvaith Vijay
Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Series
Vijay Ekambaram, Arindam Jati, Pankaj Dayama, Sumanta Mukherjee, Nam H. Nguyen, Wesley M. Gifford, Chandra Reddy, Jayant Kalagnanam
Rethinking Class-incremental Learning in the Era of Large Pre-trained Models via Test-Time Adaptation
Imad Eddine Marouf, Subhankar Roy, Enzo Tartaglione, Stéphane Lathuilière
Domain Generalization Using Large Pretrained Models with Mixture-of-Adapters
Gyuseong Lee, Wooseok Jang, Jin Hyeon Kim, Jaewoo Jung, Seungryong Kim