Fine Tuning

Fine-tuning adapts pre-trained large language models (LLMs) to specific tasks, improving performance and efficiency compared to training from scratch. Current research emphasizes efficient fine-tuning methods like low-rank adaptation (LoRA) and techniques addressing challenges such as catastrophic forgetting and calibration issues, often employing bilevel optimization or adaptive noise allocation for improved performance and privacy. This work is significant because it enables the deployment of powerful LLMs across diverse applications, from medical diagnosis to visual editing, while mitigating resource constraints and privacy concerns.

Papers

May 28, 2024

Low-rank finetuning for LLMs: A fairness perspective
Saswat Das, Marco Romanelli, Cuong Tran, Zarreen Reza, Bhavya Kailkhura, Ferdinando Fioretto
Large Language Model Fine Tuning Low Rank Approximation
OwLore: Outlier-weighed Layerwise Sampled Low-Rank Projection for Memory-Efficient LLM Fine-tuning
Pengxiang Li, Lu Yin, Xiaowei Gao, Shiwei Liu
Fine Tuning LLM Fine Tuning Projection Bias Memory Efficient Fine Tuning Local Outlier
Sparsity- and Hybridity-Inspired Visual Parameter-Efficient Fine-Tuning for Medical Diagnosis
Mingyuan Liu, Lu Xu, Shengnan Liu, Jicong Zhang
Fine Tuning Parameter Efficient Fine Tuning Medical Diagnosis Sparsity Increase Large Vision Model Model Transfer

May 27, 2024

DoRA: Enhancing Parameter-Efficient Fine-Tuning with Dynamic Rank Distribution
Yulong Mao, Kaiyu Huang, Changhao Guan, Ganglin Bao, Fengran Mo, Jinan Xu
Fine Tuning Parameter Efficient Fine Tuning Large Scale Pre Trained Model
Spectral-Refiner: Fine-Tuning of Accurate Spatiotemporal Neural Operator for Turbulent Flows
Shuhao Cao, Francesco Brarda, Ruipeng Li, Yuanzhe Xi
Fine Tuning Neural Operator Operator Learning Fourier Neural Operator Turbulent Flow
On Fairness of Low-Rank Adaptation of Large Models
Zhoujie Ding, Ken Ziyu Liu, Pura Peetathawatchai, Berivan Isik, Sanmi Koyejo
Fine Tuning Procedural Fairness Adaptation Concern Large Model Model Bias Token Bias
Understanding Linear Probing then Fine-tuning Language Models from NTK Perspective
Akiyoshi Tomihari, Issei Sato
Large Language Model Language Model Fine Tuning Transformer Based Model Linear Probing Language Model Fine Tuning

May 26, 2024

RLSF: Reinforcement Learning via Symbolic Feedback
Piyush Jha, Prithwish Jana, Pranavkrishna Suresh, Arnav Arora, Vijay Ganesh
Reinforcement Learning Fine Tuning Reward Model LLM Fine Tuning Feedback System
A Provably Effective Method for Pruning Experts in Fine-tuned Sparse Mixture-of-Experts
Mohammed Nowaz Rabbani Chowdhury, Meng Wang, Kaoutar El Maghraoui, Naigang Wang, Pin-Yu Chen, Christopher Carothers
Fine Tuning Mixture of Expert Model Pruning Inference Framework Effective Method Training Compute

May 25, 2024

No Two Devils Alike: Unveiling Distinct Mechanisms of Fine-tuning Attacks
Chak Tou Leong, Yi Cheng, Kaishuai Xu, Jian Wang, Hanlin Wang, Wenjie Li
Fine Tuning Trading Devil Attack Strategy Attack Method Semi Active Mechanism Dangerous Capability
SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Models
Xudong Lu, Aojun Zhou, Yuhui Xu, Renrui Zhang, Peng Gao, Hongsheng Li
Large Language Model Fine Tuning
Feature Protection For Out-of-distribution Generalization
Lu Tan, Huei Zhou, Yinxiang Huang, Zeming Zheng, Yujiu Yang
Fine Tuning Distribution Generalization Large Pre Trained Model Fine Tuned Model

May 24, 2024

Thinking Forward: Memory-Efficient Federated Finetuning of Language Models
Kunjal Panchal, Nisarg Parikh, Sunav Choudhary, Lijun Zhang, Yuriy Brun, Hui Guan
Large Language Model Language Model Fine Tuning Back Propagation Future Perspective Mode Automatic Differentiation Global Gradient Peak Memory
Sparse Matrix in Large Language Model Fine-tuning
Haoze He, Juncheng Billy Li, Xuan Jiang, Heather Miller
Large Language Model Fine Tuning Sparse Matrix

May 23, 2024

MallowsPO: Fine-Tune Your LLM with Preference Dispersions
Haoxian Chen, Hanyang Zhao, Henry Lam, David Yao, Wenpin Tang
Large Language Model Reinforcement Learning Fine Tuning Direct Preference Optimization Human Preference Generalization Capability Bandit Identification
PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression
Vladimir Malinovskii, Denis Mazur, Ivan Ilin, Denis Kuznedelev, Konstantin Burlachenko, Kai Yi, Dan Alistarh, Peter Richtarik
Fine Tuning Vector Quantization LLM Compression Zero Shot Quantization Straight Through Estimator
Sparse-Tuning: Adapting Vision Transformers with Efficient Fine-tuning and Inference
Ting Liu, Xuyang Liu, Siteng Huang, Liangtao Shi, Zunnan Xu, Yi Xin, Quanjun Yin, Xiaohong Liu
Vision Transformer Fine Tuning Scientific Inference Parameter Efficient Fine Tuning Pre Trained Vision Transformer Efficient Fine Tuning
Representation Noising: A Defence Mechanism Against Harmful Finetuning
Domenic Rosati, Jan Wehner, Kai Williams, Łukasz Bartoszcze, David Atanasov, Robie Gonzales, Subhabrata Majumdar, Carsten Maple, Hassan Sajjad, Frank Rudzicz
Large Language Model Fine Tuning Open Source Large Language Model Individual Representation Harmful Content De Noising

May 22, 2024

Spectral Adapter: Fine-Tuning in Spectral Space
Fangzhao Zhang, Mert Pilanci
Fine Tuning Parameter Efficient Fine Tuning Spectral Space Spectral Fine Tuning
Challenging Gradient Boosted Decision Trees with Tabular Transformers for Fraud Detection at Booking.com
Sergei Krutikov, Bulat Khaertdinov, Rodion Kiriukhin, Shubham Agrawal, Kees Jan De Vries
Self Supervised Learning Fine Tuning Decision Tree Natural Gradient Fraud Detection Tabular Transformer

Fine Tuning

Papers

Low-rank finetuning for LLMs: A fairness perspective

OwLore: Outlier-weighed Layerwise Sampled Low-Rank Projection for Memory-Efficient LLM Fine-tuning

Sparsity- and Hybridity-Inspired Visual Parameter-Efficient Fine-Tuning for Medical Diagnosis

DoRA: Enhancing Parameter-Efficient Fine-Tuning with Dynamic Rank Distribution

Spectral-Refiner: Fine-Tuning of Accurate Spatiotemporal Neural Operator for Turbulent Flows

On Fairness of Low-Rank Adaptation of Large Models

Understanding Linear Probing then Fine-tuning Language Models from NTK Perspective

RLSF: Reinforcement Learning via Symbolic Feedback

A Provably Effective Method for Pruning Experts in Fine-tuned Sparse Mixture-of-Experts

No Two Devils Alike: Unveiling Distinct Mechanisms of Fine-tuning Attacks

SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Models

Feature Protection For Out-of-distribution Generalization

Thinking Forward: Memory-Efficient Federated Finetuning of Language Models

Sparse Matrix in Large Language Model Fine-tuning

MallowsPO: Fine-Tune Your LLM with Preference Dispersions

PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression

Sparse-Tuning: Adapting Vision Transformers with Efficient Fine-tuning and Inference

Representation Noising: A Defence Mechanism Against Harmful Finetuning

Spectral Adapter: Fine-Tuning in Spectral Space

Challenging Gradient Boosted Decision Trees with Tabular Transformers for Fraud Detection at Booking.com