Efficient Model Adaptation

Efficient model adaptation focuses on modifying pre-trained large models (like Vision Transformers and Large Language Models) for specific tasks while minimizing computational cost and memory usage. Current research emphasizes techniques like low-rank adaptation (LoRA), sparse parameter updates, and modular model architectures, often incorporating strategies such as knowledge distillation and prompt tuning to improve efficiency and performance. These advancements are crucial for deploying advanced AI models on resource-constrained devices and accelerating the development of adaptable AI systems across various applications, including robotics, autonomous driving, and natural language processing.

Papers

October 1, 2024

SQFT: Low-cost Model Adaptation in Low-precision Sparse Foundation Models
Juan Pablo Muñoz, Jinjie Yuan, Nilesh Jain
Large Language Model Foundation Model Parameter Efficient Fine Tuning Large Pre Trained Model Efficient Model Adaptation Sparse Local Model

June 17, 2024

Model Adaptation for Time Constrained Embodied Control
Jaehyun Song, Minjong Yoo, Honguk Woo
Robotic Manipulation Model Adaptation Efficient Model Adaptation Embodied Control Time Aware

February 28, 2024

LoRA-SP: Streamlined Partial Parameter Adaptation for Resource-Efficient Fine-Tuning of Large Language Models
Yichao Wu, Yafei Xiang, Shuning Huo, Yulu Gong, Penghao Liang
Large Language Model Fine Tuning NLP Model Natural Language Processing Benchmark Efficient Model Adaptation Parameter Adaptation

October 25, 2023

MACP: Efficient Model Adaptation for Cooperative Perception
Yunsheng Ma, Juanwu Lu, Can Cui, Sicheng Zhao, Xu Cao, Wenqian Ye, Ziran Wang
Single Agent Cooperative Perception Multi Agent Perception Single Agent Learning Efficient Model Adaptation

October 10, 2023

Efficient Adaptation of Large Vision Transformer via Adapter Re-Composing
Wei Dong, Dawei Yan, Zhijun Lin, Peng Wang
Vision Transformer Pre Trained Model Efficient Adaptation Efficient Model Adaptation Large Pre Trained Parameter Efficient Adapter Conditional Adapter Vision Transformer Variant

September 25, 2023

Informative Data Mining for One-Shot Cross-Domain Semantic Segmentation
Yuxi Wang, Jian Liang, Jun Xiao, Shuqi Mei, Yuran Yang, Zhaoxiang Zhang
Domain Adaptation Domain Transfer Cross Domain Semantic Segmentation Efficient Model Adaptation One Shot Unsupervised Domain Adaptation Informative Subset

September 15, 2023

MetaF2N: Blind Image Super-Resolution by Learning Efficient Model Adaptation from Faces
Zhicun Yin, Ming Liu, Xiaoming Li, Hui Yang, Longan Xiao, Wangmeng Zuo
Super Resolution Human Face Image Degradation Blind Image Super Resolution Efficient Model Adaptation

August 3, 2023

Efficient Model Adaptation for Continual Learning at the Edge
Zachary A. Daniels, Jun Hu, Michael Lomnitz, Phil Miller, Aswin Raghavan, Joe Zhang, Michael Piacentino, David Zhang
Domain Adaptation Continual LEArning Extreme Edge Adapter Learning Class Incremental Continual Learning Efficient Continual Learning Efficient Model Adaptation

May 8, 2023

HiFi: High-Information Attention Heads Hold for Parameter-Efficient Model Adaptation
Anchun Gui, Han Xiao
Fine Tuning Scale Pre Trained Language Model Structured Information PageRank Algorithm Efficient Model Adaptation

March 29, 2023

Sketch-an-Anchor: Sub-epoch Fast Model Adaptation for Zero-shot Sketch-based Image Retrieval
Leo Sampaio Ferraz Ribeiro, Moacir Antonelli Ponti
Retrieval Performance Efficient Model Adaptation Zero Shot Sketch Based Image Semantic Anchor

August 22, 2022

PANDA: Prompt Transfer Meets Knowledge Distillation for Efficient Model Adaptation
Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du, Dacheng Tao
Knowledge Distillation Task Similarity Task Relatedness Red PANDA Efficient Model Adaptation Prompt Transferability

March 29, 2022

Parameter-efficient Model Adaptation for Vision Transformers
Xuehai He, Chunyuan Li, Pengchuan Zhang, Jianwei Yang, Xin Eric Wang
Vision Transformer Model Adaptation Efficient Model Adaptation