Pre Trained Model
Pre-trained models are foundational large-scale models trained on massive datasets, subsequently adapted for specific downstream tasks using techniques like fine-tuning or parameter-efficient fine-tuning (PEFT). Current research emphasizes improving the efficiency and effectiveness of these adaptation methods, exploring architectures such as Vision Transformers and diffusion models, and developing algorithms like LoRA and its nonlinear extensions to minimize resource consumption while maximizing performance. This field is crucial for advancing various applications, from medical image analysis and environmental sound classification to autonomous driving and natural language processing, by enabling the development of high-performing models with limited data and computational resources.
Papers
Forecast-PEFT: Parameter-Efficient Fine-Tuning for Pre-trained Motion Forecasting Models
Jifeng Wang, Kaouther Messaoud, Yuejiang Liu, Juergen Gall, Alexandre Alahi
Combined CNN and ViT features off-the-shelf: Another astounding baseline for recognition
Fernando Alonso-Fernandez, Kevin Hernandez-Diaz, Prayag Tiwari, Josef Bigun
Exploiting Pre-trained Models for Drug Target Affinity Prediction with Nearest Neighbors
Qizhi Pei, Lijun Wu, Zhenyu He, Jinhua Zhu, Yingce Xia, Shufang Xie, Rui Yan
Large Language Model for Verilog Generation with Golden Code Feedback
Ning Wang, Bingkun Yao, Jie Zhou, Xi Wang, Zhe Jiang, Nan Guan
Learn to Preserve and Diversify: Parameter-Efficient Group with Orthogonal Regularization for Domain Generalization
Jiajun Hu, Jian Zhang, Lei Qi, Yinghuan Shi, Yang Gao