Many Parameter
Research on "many parameter" models focuses on optimizing the number and utilization of parameters in various machine learning architectures to improve efficiency and performance. Current efforts concentrate on developing parameter-efficient fine-tuning techniques, exploring different model architectures like transformers and graph convolutional networks, and investigating the impact of parameter count on model capabilities and generalization. This research is significant because it addresses the computational cost and resource limitations associated with large models, enabling wider accessibility and applicability across diverse fields, including medical imaging, robotics, and natural language processing.
Papers
OnlySportsLM: Optimizing Sports-Domain Language Models with SOTA Performance under Billion Parameters
Zexin Chen, Chengxi Li, Xiangyu Xie, Parijat Dube
MoRe Fine-Tuning with 10x Fewer Parameters
Wenxuan Tan, Nicholas Roberts, Tzu-Heng Huang, Jitian Zhao, John Cooper, Samuel Guo, Chengyu Duan, Frederic Sala
How Many Parameters Does it Take to Change a Light Bulb? Evaluating Performance in Self-Play of Conversational Games as a Function of Model Characteristics
Nidhir Bhavsar, Jonathan Jordan, Sherzod Hakimov, David Schlangen
Random pairing MLE for estimation of item parameters in Rasch model
Yuepeng Yang, Cong Ma
From RAGs to rich parameters: Probing how language models utilize external knowledge over parametric information for factual queries
Hitesh Wadhwa, Rahul Seetharaman, Somyaa Aggarwal, Reshmi Ghosh, Samyadeep Basu, Soundararajan Srinivasan, Wenlong Zhao, Shreyas Chaudhari, Ehsan Aghazadeh
Restorer: Removing Multi-Degradation with All-Axis Attention and Prompt Guidance
Jiawei Mao, Juncheng Wu, Yuyin Zhou, Xuesong Yin, Yuanqi Chang