Adapter Module

Adapter modules are lightweight, parameter-efficient components added to pre-trained models to adapt them to new tasks or domains without retraining the entire model. Current research focuses on improving their performance, particularly in comparison to full fine-tuning, exploring various architectures like low-rank adapters (LoRA) and mixtures of adapters (MoA), and optimizing their integration into diverse model types, including transformers and neural cellular automata. This approach significantly reduces computational costs and memory requirements for tasks such as multilingual language processing, image recognition, and speech synthesis, making large model adaptation more accessible and efficient.

Papers

November 7, 2023

Mini but Mighty: Finetuning ViTs with Mini Adapters
Imad Eddine Marouf, Enzo Tartaglione, Stéphane Lathuilière
Vision Transformer Computer Vision Fine Tuning Adapter Module Parameter Efficient Transfer Learning Pre Trained ViT Mini Honor

November 2, 2023

Task-Agnostic Low-Rank Adapters for Unseen English Dialects
Zedian Xiao, William Held, Yanchen Liu, Diyi Yang
Adapter Module Task Agnostic Unseen Language Dialect Datasets Dialect Speaker English Variety

October 11, 2023

Adapting the adapters for code-switching in multilingual ASR
Atharva Kulkarni, Ajinkya Kulkarni, Miguel Couceiro, Hanan Aldarmaki
Multilingual Model Multilingual Automatic Speech Recognition Adapter Module Code Switched Code Switched Data

July 31, 2023

Revisiting the Parameter Efficiency of Adapters from the Perspective of Precision Redundancy
Shibo Jie, Haoqing Wang, Zhi-Hong Deng
Fine Tuning Visual Perspective Vision Model Parameter Efficient Adapter Module Parameter Efficient Tuning Adapter Tuning Redundancy Aware

July 6, 2023

Efficient Domain Adaptation of Sentence Embeddings Using Adapters
Tim Schopf, Dennis N. Schneider, Florian Matthes
Sentence Embeddings Semantic Similarity Semantic Textual Similarity Adapter Module Efficient Domain Adaptation Domain Specific Adapter

May 29, 2023

ADAPTERMIX: Exploring the Efficacy of Mixture of Adapters for Low-Resource TTS Adaptation
Ambuj Mehrish, Abhinav Ramesh Kashyap, Li Yingting, Navonil Majumder, Soujanya Poria
Speech Synthesis Mixture Component Adapter Module Speaker Adaptation Adapter Fusion Speaker Change

May 19, 2023

Differentially Private Adapters for Parameter Efficient Acoustic Modeling
Chun-Wei Ho, Chao-Han Huck Yang, Sabato Marco Siniscalchi
Differential Privacy Adapter Module Frozen Pre Trained Spatial Acoustic Frozen Pre Trained Language Model Multilingual Speech Corpus

April 13, 2023

Masakhane-Afrisenti at SemEval-2023 Task 12: Sentiment Analysis using Afro-centric Language Models and Adapters for Low-resource African Languages
Israel Abebe Azime, Sana Sabah Al-Azzawi, Atnafu Lambebo Tonja, Iyanuoluwa Shode, Jesujoba Alabi, Ayodele Awokoya, Mardiyyah Oduwole, Tosin Adewumi, Samuel Fanijo, Oyinkansola Awosan, Oreen Yousuf
Zero Shot Sentiment Analysis SemEval 2022 Task African Language Adapter Module Multilingual Sentiment

April 4, 2023

March 23, 2023

Parameter-Efficient Sparse Retrievers and Rerankers using Adapters
Vaishali Pal, Carlos Lassance, Hervé Déjean, Stéphane Clinchant
Domain Adaptation Adapter Module Retrieval Benchmark Parameter Efficient Transfer

February 28, 2023

SMoA: Sparse Mixture of Adapters to Mitigate Multiple Dataset Biases
Yanchen Liu, Jing Yan, Yan Chen, Jing Liu, Hua Wu
Natural Language Inference Adapter Module Debiasing Method Dataset Bias Sparse Mixture

February 12, 2023

Generalized Few-Shot Continual Learning with Contrastive Mixture of Adapters
Yawen Cui, Zitong Yu, Rizhao Cai, Xun Wang, Alex C. Kot, Li Liu
Continual LEArning Domain Generalization Continual Learning Domain Invariant Representation Adapter Module Domain Generalization Datasets

February 7, 2023

UDApter -- Efficient Domain Adaptation Using Adapters
Bhavitvya Malik, Abhinav Ramesh Kashyap, Min-Yen Kan, Soujanya Poria
Unsupervised Domain Adaptation Adapter Module Scale Pre Trained Language Model Natural Language Inference Task Efficient Domain Adaptation Domain Sentiment

February 6, 2023

Adaptive Parameterization of Deep Learning Models for Federated Learning
Morten From Elvebakken, Alexandros Iosifidis, Lukas Esterle
Deep Neural Network Deep Learning Model Federated Learning Adapter Module Parameter Update Inference Performance

December 20, 2022

Parameter-efficient Zero-shot Transfer for Cross-Language Dense Retrieval with Adapters
Eugene Yang, Suraj Nair, Dawn Lawrie, James Mayfield, Douglas W. Oard
Zero Shot Multilingual Model Adapter Module Monolingual Retrieval Cross Language Information Retrieval

November 2, 2022

Universal Deep Image Compression via Content-Adaptive Optimization with Adapters
Koki Tsubota, Hiroaki Akutsu, Kiyoharu Aizawa
Image Compression Adapter Module Deep Compression

October 28, 2022

Parameter-efficient transfer learning of pre-trained Transformer models for speaker verification using adapters
Junyi Peng, Themos Stafylakis, Rongzhi Gu, Oldřich Plchot, Ladislav Mošner, Lukáš Burget, Jan Černocký
Fine Tuning Pre Trained Model Speaker Verification Pre Trained Transformer Adapter Module Parameter Efficient Transfer Learning Speaker Recognition Task

October 24, 2022

Adapters for Enhanced Modeling of Multilingual Knowledge and Text
Yifan Hou, Wenxiang Jiao, Meizhen Liu, Carl Allen, Zhaopeng Tu, Mrinmaya Sachan
Language Model Text Modality Multilingual Language Model Adapter Module Model Improvement Multilingual Knowledge Cross Lingual Entity Multilingual Knowledge Graph

October 10, 2022

Revisiting adapters with adversarial training
Sylvestre-Alvise Rebuffi, Francesco Croce, Sven Gowal
Adversarial Training Masked Autoencoders Adversarial Input Adapter Module