Side Tuning

Side tuning is a parameter-efficient transfer learning technique that adapts pre-trained deep learning models to new tasks by adding small, trainable "side networks" alongside the main model, rather than fine-tuning the entire model. Current research focuses on applying side tuning to various computer vision and vision-language tasks, often using transformer-based architectures or adapting it for convolutional networks like ResNet. This approach offers significant advantages in reducing computational cost and memory usage while maintaining or even exceeding the performance of full fine-tuning, making it a valuable tool for resource-constrained applications and improving the efficiency of model adaptation across diverse domains.