Many Parameter

Research on "many parameter" models focuses on optimizing the number and utilization of parameters in various machine learning architectures to improve efficiency and performance. Current efforts concentrate on developing parameter-efficient fine-tuning techniques, exploring different model architectures like transformers and graph convolutional networks, and investigating the impact of parameter count on model capabilities and generalization. This research is significant because it addresses the computational cost and resource limitations associated with large models, enabling wider accessibility and applicability across diverse fields, including medical imaging, robotics, and natural language processing.

Papers

October 12, 2023

Every Parameter Matters: Ensuring the Convergence of Federated Learning with Dynamic Heterogeneous Models Reduction
Hanhan Zhou, Tian Lan, Guru Venkataramani, Wenbo Ding
Early Stage Convergence Many Parameter Convergence Analysis Heterogeneous Federated Learning Model Reduction Cross Device Federated Learning

October 11, 2023

Domain Generalization Guided by Gradient Signal to Noise Ratio of Parameters
Mateusz Michalkiewicz, Masoud Faraki, Xiang Yu, Manmohan Chandraker, Mahsa Baktashmotlagh
Domain Generalization Gradient Based Many Parameter Regularization Approach Gradient Conflict Dropout Mask

September 20, 2023

Budget-Aware Pruning: Handling Multiple Domains with Less Parameters
Samuel Felipe dos Santos, Rodrigo Berriel, Thiago Oliveira-Santos, Nicu Sebe, Jurandy Almeida
Deep Learning Deep Model Vision Task Many Parameter Multi Domain Multi Domain Learning Pruned Model

September 12, 2023

Do Generative Large Language Models need billions of parameters?
Sia Gholami, Marwan Omar
AI System Many Parameter Generative Large Language Model Artificial Intelligence Language Model Efficient Large Language Model Language Structure

September 4, 2023

Parameter and Computation Efficient Transfer Learning for Vision-Language Pre-trained Models
Qiong Wu, Wei Yu, Yiyi Zhou, Shubin Huang, Xiaoshuai Sun, Rongrong Ji
Full Model Vision Language Many Parameter Downstream Task Efficient Transfer Learning

July 16, 2023

Computing the gradients with respect to all parameters of a quantum neural network using a single circuit
Guang Ping He
Natural Gradient Quantum Circuit Many Parameter Quantum Neural Network Digital Computing Depth Circuit Parameter Shift

July 6, 2023

FITS: Modeling Time Series with $10k$ Parameters
Zhijian Xu, Ailing Zeng, Qiang Xu
Time Series Time Series Forecasting Time Series Data Many Parameter Time Series Analysis Better FiT

June 26, 2023

ParameterNet: Parameters Are All You Need
Kai Han, Yunhe Wang, Jianyuan Guo, Enhua Wu
Vision Model Many Parameter Large Vision Model Parameterized Network

June 23, 2023

FedSelect: Customized Selection of Parameters for Fine-Tuning during Personalized Federated Learning
Rishub Tamirisa, John Won, Chengjun Lu, Ron Arel, Andy Zhou
Fine Tuning Many Parameter Personalized Federated Learning Global Aggregation

June 16, 2023

Parameter-efficient is not sufficient: Exploring Parameter, Memory, and Time Efficient Adapter Tuning for Dense Predictions
Dongshuo Yin, Xueting Han, Bin Li, Hao Feng, Jing Bai
Memory Trace Many Parameter Parameter Efficient Efficient Deep Dense Prediction Parameter Efficient Transfer Task Specific Adapter Mask RCNN Adapter Tuning Frozen Pre Trained

June 6, 2023

One-Dimensional Deep Image Prior for Curve Fitting of S-Parameters from Electromagnetic Solvers
Sriram Ravula, Varun Gorti, Bo Deng, Swagato Chakraborty, James Pingenot, Bhyrav Mutnury, Doug Wallace, Doug Winterberg, Adam Klivans, Alexandros G. Dimakis
Many Parameter Automatic Solver Deep Image Prior Electromagnetic Simulation Data Measurement Curve Modeling

June 1, 2023

Make Pre-trained Model Reversible: From Parameter to Memory Efficient Fine-Tuning
Baohao Liao, Shaomu Tan, Christof Monz
Pre Trained Language Model Pre Trained Model Parameter Efficient Fine Tuning Many Parameter Learnable Parameter Memory Efficient Fine Tuning

May 30, 2023

Joint Bayesian Inference of Graphical Structure and Parameters with a Single Generative Flow Network
Tristan Deleu, Mizu Nishikawa-Toomey, Jithendaraa Subramanian, Nikolay Malkin, Laurent Charlin, Yoshua Bengio
Generative Model Many Parameter Bayesian Network Posterior Inference Generative Flow Network Marginal Distribution Joint Inference Graphical Structure

May 28, 2023

A Meta-learning Framework for Tuning Parameters of Protection Mechanisms in Trustworthy Federated Learning
Xiaojin Zhang, Yan Kang, Lixin Fan, Kai Chen, Qiang Yang
Federated Learning Many Parameter Homomorphic Encryption PRIvacy Leakage Meta Learning Framework Secret Sharing Protection Mechanism

May 24, 2023

May 19, 2023

Decouple knowledge from parameters for plug-and-play language modeling
Xin Cheng, Yankai Lin, Xiuying Chen, Dongyan Zhao, Rui Yan
Many Parameter Multi Agent Decoupling Coefficient Pre Training Model Pre Trained Language Prompt Based Knowledge Knowledge Learning

May 17, 2023

Finding an $\epsilon$-close Variation of Parameters in Bayesian Networks
Bahare Salmani, Joost-Pieter Katoen
Many Parameter Bayesian Network Conditional Probability

April 27, 2023

Some of the variables, some of the parameters, some of the times, with some physics known: Identification with partial information
Saurabh Malani, Tom S. Bertalan, Tianqi Cui, Jose L. Avalos, Michael Betenbaugh, Ioannis G. Kevrekidis
Dynamical System Person Identification Many Parameter Theoretical Physic Variable Model Incomplete Information Experimental Data Physical Knowledge

April 17, 2023

A Comparative Study between Full-Parameter and LoRA-based Fine-Tuning on Chinese Instruction Data for Instruction Following Large Language Model
Xianghui Sun, Yunjie Ji, Baochang Ma, Xiangang Li
Large Language Model Natural Language Processing Instruction Tuning Human Instruction Many Parameter Fast LoRA Parameter Learning

Many Parameter

Papers

Every Parameter Matters: Ensuring the Convergence of Federated Learning with Dynamic Heterogeneous Models Reduction

Domain Generalization Guided by Gradient Signal to Noise Ratio of Parameters

Budget-Aware Pruning: Handling Multiple Domains with Less Parameters

Do Generative Large Language Models need billions of parameters?

Parameter and Computation Efficient Transfer Learning for Vision-Language Pre-trained Models

Computing the gradients with respect to all parameters of a quantum neural network using a single circuit

FITS: Modeling Time Series with $10k$ Parameters

ParameterNet: Parameters Are All You Need

FedSelect: Customized Selection of Parameters for Fine-Tuning during Personalized Federated Learning

Parameter-efficient is not sufficient: Exploring Parameter, Memory, and Time Efficient Adapter Tuning for Dense Predictions

One-Dimensional Deep Image Prior for Curve Fitting of S-Parameters from Electromagnetic Solvers

Make Pre-trained Model Reversible: From Parameter to Memory Efficient Fine-Tuning

Joint Bayesian Inference of Graphical Structure and Parameters with a Single Generative Flow Network

A Meta-learning Framework for Tuning Parameters of Protection Mechanisms in Trustworthy Federated Learning

Simultaneous identification of models and parameters of scientific simulators

Detection of Non-uniformity in Parameters for Magnetic Domain Pattern Generation by Machine Learning

Decouple knowledge from parameters for plug-and-play language modeling

Finding an $\epsilon$-close Variation of Parameters in Bayesian Networks

Some of the variables, some of the parameters, some of the times, with some physics known: Identification with partial information

A Comparative Study between Full-Parameter and LoRA-based Fine-Tuning on Chinese Instruction Data for Instruction Following Large Language Model