Fine-Tuning

Follow this topic Ask about this topic

Latest papers 205

31 papers in the last four weeks, up 72% on the four weeks before. 0.2% of all new papers.

All topics

LoCO: Low-rank Compositional Rotation Fine-tuning
May 15, 2026An Nguyen, Jaesik Choi, Anh TongFine-Tuning LLM Fine-Tuning
1Korea University · 2KAIST · 3INEEJI
PreFT: Prefill-only finetuning for efficient inference
May 14, 2026Andrew Lanpouthakoun, Aryaman Arora, Zhengxuan Wu +4Fine-Tuning LLM Serving
Stanford University · Tilde Research
LIFT: Last-Mile Fine-Tuning for Table Explicitation
May 13, 2026Divij Khaitan, Ashish TiwariSupervised Fine-Tuning Fine-Tuning
Microsoft Corporation Bangalore, India · Microsoft Corporation Redmond, WA 98052 USA
A Transfer Learning Evaluation of Deep Neural Networks for Image Classification
May 12, 2026Nermeen Abou Baker, Nico Zengeler, Uwe HandmannFine-Tuning Cross-Domain Transfer Learning
Computer Science Institute, Ruhr West University of Applied Sciences, 46236 Bottrop, Germany
LOFT: Low-Rank Orthogonal Fine-Tuning via Task-Aware Support Selection
May 12, 2026Lanxin Zhao, Bamdev Mishra, Pratik Jawanpuria +4Fine-Tuning Low-Rank Adaptation
The University of Sydney, Australia. · Microsoft India. · Indian Institute of Technology Bombay, India. +1
Freeze Deep, Train Shallow: Interpretable Layer Allocation for Continued Pre-Training
May 12, 2026Yu-Hang Wu, Qin-Yuan Liu, Qiu-Yang Zhao +3Fine-Tuning Continued Pretraining
Nanhu Research Institute of China Electronic Science and Technology · School of Electronic and Electrical Engineering, Shanghai University of Engineering Science
DynaMiCS: Fine-tuning LLMs with Performance Constraints using Dynamic Mixtures
May 11, 2026Eleonora Gualdoni, Sonia Laguna, Louis Bethune +3Fine-Tuning LLM Fine-Tuning
Apple
Acceptance Cards:A Four-Diagnostic Standard for Safe Fine-Tuning Defense Claims
May 11, 2026Phongsakon Mark Konrad, Toygar Tanyel, Serkan AyvazFine-Tuning Language Model Safety Evaluation
Centre for Industrial Software, University of Southern Denmark, Alsion 2, Sønderborg, 6400, Denmark · ProMake, Newark, DE, USA
Can Muon Fine-tune Adam-Pretrained Models?
May 11, 2026Xingyu Qu, Peigeng Huang, Samuel HorvathFine-Tuning Muon Optimizer
MBZUAI · Nanjing University
HELLoRA: Hot Experts Layer-Level Low-Rank Adaptation for Mixture-of-Experts Models
May 11, 2026Jia Wei, Zhonghao Zhang, Ping Chen +5Mixture-of-Experts Language Models Fine-Tuning
Department of Computer Science and Technlogy Tsinghua University Beijing 100084, China · School of Computer Science and Technlogy Xi’an Jiaotong University · The State Key Laboratory of Blockchain and Data Security, Zhejiang University Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security
S2FT: Parameter-Efficient Fine-Tuning in Sparse Spectrum Domain
May 9, 2026Baoquan Zhang, Zhehao Yu, Lisai Zhang +5Fine-Tuning Sparse Fine-Tuning
Harbin Institute of Technology, Shenzhen · Bilibili. Inc · Shenzhen University +1
Rotation-Preserving Supervised Fine-Tuning
May 8, 2026Hangzhan Jin, Tianwei Ni, Lu Li +3Supervised Fine-Tuning Fine-Tuning
1Mila - Quebec AI Institute · 2Polytechnique Montréal · 3Université de Montréal +3
Bayesian Fine-tuning in Projected Subspaces
May 8, 2026Viktar Dubovik, Patryk Marszałek, Jacek Tabor +1Bayesian Neural Networks Fine-Tuning
Jagiellonian University, Kraków, Poland · Doctoral School of Exact and Natural Sciences, Jagiellonian University, Kraków, Poland
DIMoE-Adapters: Dynamic Expert Evolution for Continual Learning in Vision-Language Models
May 8, 2026Mengxin Qin, Xiang Zhang, Xi Wang +3Vision-Language Models Fine-Tuning
School of Electronic Engineering, Xidian University Xi’an 710071, China
Improved techniques for fine-tuning flow models via adjoint matching: a deterministic control pipeline
May 7, 2026Zhengyi Guo, Jiayuan Sheng, David D. Yao +1Fine-Tuning Flow-Based Generative Modeling
VLA-GSE: Boosting Parameter-Efficient Fine-Tuning in VLA with Generalized and Specialized Experts
May 7, 2026Yuhua Jiang, Junjie Lu, Xinyao Qin +4Fine-Tuning Vision-Language-Action Models
Microsoft Research Asia · Tsinghua University
Backdoor Mitigation in Object Detection via Adversarial Fine-Tuning
May 7, 2026Kealan Dunnett, Reza Arablouei, Dimity Miller +2Fine-Tuning Backdoor Attacks
Queensland University of Technology, Brisbane Australia · CSIRO, Pullenvale QLD, Australia
CERSA: Cumulative Energy-Retaining Subspace Adaptation for Memory-Efficient Fine-Tuning
May 5, 2026Jingze Ge, Xue Geng, Yun Liu +6Fine-Tuning Memory-Efficient Fine-Tuning
National University of Singapore · Institute for Infocomm Research (I2R), A*STAR · Nankai University +1
Ortho-Hydra: Orthogonalized Experts for DiT LoRA
May 5, 2026Seunghyun JiFine-Tuning Diffusion Transformer
Measuring Accuracy and Energy-to-Solution of Quantum Fine-Tuning of Foundational AI Models
May 4, 2026Oliver Knitter, Sang Hyub Kim, Maximilian Wurzer +7Fine-Tuning Energy-Efficient ML
IonQ Inc., 4505 Campus Dr, College Park, MD 20740, USA · QuantumBasel, Schorenweg 44b, 4144 Arlesheim, Switzerland · Center for Quantum Computing and Quantum Coherence (QC2), University of Basel, Klingelbergstrasse 82, 4056 Basel, Switzerland
Generalized Distributional Alignment Games for Unbiased Answer-Level Fine-Tuning
May 4, 2026Mehryar Mohri, Jon Schneider, Yutao ZhongFine-Tuning LLM Alignment
Google Research & CIMS · New York, NY 10011 · Google Research
Fine-Tuning Impairs the Balancedness of Foundation Models in Long-tailed Personalized Federated Learning
May 4, 2026Shihao Hou, Chikai Shang, Zhiheng Yang +5Class-Imbalanced Learning Fine-Tuning
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, Xiamen, China · Department of Statistical Science, University College London, London, United Kingdom · School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, China
Demographic-Aware Transfer Learning for Sleep Stage Classification in Clinical Polysomnography
May 4, 2026S M Asif Hossain, Shruti KshirsagarSleep Stage Classification Fine-Tuning
School of Computing, Wichita State University, Wichita, Kansas, USA
RefusalGuard: Geometry-Preserving Fine-Tuning for Safety in LLMs
May 3, 2026Sadia Asif, Mohammad Mohammadi AmiriFine-Tuning LLM Fine-Tuning
Department of Computer Science Rensselaer Polytechnic Institute Troy, NY, USA
ReMedi: Reasoner for Medical Clinical Prediction
May 2, 2026Yushi Cao, Yiming Chen, Hongchao Jiang +2Fine-Tuning Clinical Outcome Prediction
ASUS Intelligent Cloud Services (AICS), Singapore · NTU Artificial Intelligence Center of Research Excellence (NTU AI-CoRE), Taiwan · National University of Singapore, Singapore
Dynamic Scaled Gradient Descent for Stable Fine-Tuning for Classifications
Apr 30, 2026Nghia Bui, Lijing WangFine-Tuning Gradient Descent Dynamics
Department of Data Science, New Jersey Institute of Technology, New Jersey, USA.
Diagnosing Capability Gaps in Fine-Tuning Data
Apr 30, 2026Saeid Asgari Taghanaki, Rakshanda Agarwal, Bruce Sun +10Fine-Tuning LLM Fine-Tuning
Microsoft
Learning Rate Engineering: From Coarse Single Parameter to Layered Evolution
Apr 30, 2026Ming-Hong Yao, Di Wang, Jian Cui +5Deep Learning Optimization Fine-Tuning
School of AI, Jilin International Sciences University, Changchun, China.
Distributional Alignment Games for Answer-Level Fine-Tuning
Apr 29, 2026Mehryar Mohri, Jon Schneider, Yifan WuFine-Tuning RL for Language Model Reasoning
Google Research · Microsoft Research
Channel Adaptation for EEG Foundation Models: A Systematic Benchmark Across Architectures, Tasks, and Training Regimes
Apr 25, 2026Kuntal Kokate, Bruno Aristimunha, Dung Truong +1Fine-Tuning EEG Decoding
Swartz Center for Computational Neuroscience, Institute for Neural Computation, University of California San Diego, La Jolla, CA 92093, USA · Yneuro, Paris, France · Centre de Recherche Cerveau et Cognition, CNRS, Université Paul Sabatier, Toulouse, France

May 15, 2026, cs.LGJ/K move · Enter open · S save

LoCO: Low-rank Compositional Rotation Fine-tuning

An Nguyen, Jaesik Choi, Anh Tong

1Korea University · 2KAIST · 3INEEJI

Fine-Tuning LLM Fine-Tuning Vision Foundation Model Adaptation Low-Rank Approximation Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Parameter-efficient fine-tuning (PEFT) has emerged as an critical technique for adapting large-scale foundation models across natural language processing and computer vision. While existing methods such as low-rank adaptations achieve parameter efficiency via low-rank weight updates, they are limited in their ability to preserve the geometric structure of pretrained representations. We introduce Low-rank Compositional Orthogonal fine-tuning (LoCO), a novel PEFT method that constructs orthogonal transformations through low-rank skew-symmetric matrices and compositional rotation chains. We propose an approximation scheme that enables fully parallel computation of compositional rotations, making the approach practical for high-dimensional feature spaces. Our method maintains low computational complexity while maintaining orthogonality with controlled approximation error. We validate LoCO across diverse domains, including diffusion transformer fine-tuning, vision transformer adaptation, and language model adaptation. Our method demonstrates superior or competitive performance compared to both existing orthogonal and non-orthogonal methods.