Many Parameter
Research on "many parameter" models focuses on optimizing the number and utilization of parameters in various machine learning architectures to improve efficiency and performance. Current efforts concentrate on developing parameter-efficient fine-tuning techniques, exploring different model architectures like transformers and graph convolutional networks, and investigating the impact of parameter count on model capabilities and generalization. This research is significant because it addresses the computational cost and resource limitations associated with large models, enabling wider accessibility and applicability across diverse fields, including medical imaging, robotics, and natural language processing.
Papers
Learning the parameters of a differential equation from its trajectory via the adjoint equation
Imre Fekete, András Molnár, Péter L. Simon
Neural Architecture Adaptation for Object Detection by Searching Channel Dimensions and Mapping Pre-trained Parameters
Harim Jung, Myeong-Seok Oh, Cheoljong Yang, Seong-Whan Lee
LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learning
Yi-Lin Sung, Jaemin Cho, Mohit Bansal
Singular Value Fine-tuning: Few-shot Segmentation requires Few-parameters Fine-tuning
Yanpeng Sun, Qiang Chen, Xiangyu He, Jian Wang, Haocheng Feng, Junyu Han, Errui Ding, Jian Cheng, Zechao Li, Jingdong Wang