Finetuning

Follow this topic Ask about this topic

Recent momentum

-31%

9 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

7 new papers

A weekly snapshot of new work published in Finetuning.

Period ending 2026-09-14

3 new papers

A weekly snapshot of new work published in Finetuning.

Period ending 2026-09-07

4 new papers

A weekly snapshot of new work published in Finetuning.

140 papers

Latest in Finetuning

Browse all topics →

Sticky Routing: Training MoE Models for Memory-Efficient Inference
Jun 12, 2026Ali KayyamMixture-Of-Expert Language Models Mixture-of-Experts Models
Zero-order Parameter-free Optimization for LMO-based Methods: Novel Approach for Efficient Fine-tuning
Jun 12, 2026Dmitriy Bystrov, Daniil Medyakov, Dmitry Bylinkin +1Parameter-Efficient Fine-Tuning Large Language Model Fine-Tuning
Small LLMs for Biomedical Claim Verification: Cost-Effective Fine-Tuning, Structural Dataset Shortcuts, and Cross-Domain Generalization
Jun 11, 2026Gaurav KumarSmall Language Models Finetuning
ViT-FREE: Efficient Face Recognition via Early Exiting and Synthetic Adaptation
Jun 10, 2026Tahar Chettaoui, Guray Ozgur, Eduarda Caldeira +2Self-Supervised Vision Transformers Facial Recognition
Instruction Finetuning DeepSeek-R1-8B Model Using LoRA and NEFTune
Jun 9, 2026Wu Yuerong, Mingni LuoNamed-Entity Recognition Large Language Model Fine-Tuning
Lost in the Non-convex Loss Landscape: How to Fine-tune the Large Time Series Model?
Jun 7, 2026Xu Zhang, Peang Wang, Wei WangFinetuning Time Series
The Fine-Tuning Trap: Evaluating Negative Transfer and the Role of PEFT in Sub-1B Mathematical Reasoning
Jun 5, 2026Rahul Nair, Chun TaoParameter-Efficient Fine-Tuning Finetuning
High-Dimensional Theory of LoRA Fine-Tuning in a Solvable Attention Model
Jun 4, 2026O. Duranthon, F. Boncoraglio, L. ZdeborováLow-Rank Adaptation Finetuning
(Mis)generalization of Helpful-only Fine-tuning
Jun 3, 2026Mohammad Omar Khursheed, Baram Sosis, Fabien RogerHarmful Fine-Tuning Finetuning
Long Live Fine-Tuning: Task-Specific Transformers Outperform Zero-Shot LLMs for Misinformation Response Classification on Reddit
Jun 2, 2026JooYoung Lee, Lin Tian, Angela Brillantes +2Misinformation Detection Misinformation
FACT: A Simple and Efficient Framework for Active Finetuning
Jun 1, 2026Wenshuai Xu, You Song, Yuzhuo Cui +3Parameter-Efficient Fine-Tuning Finetuning
Training Prompt Matters: State-Adaptive Optimization for Robust Fine-Tuning
Jun 1, 2026Wenhang Shi, Yiren Chen, Shuqing Bian +5Finetuning Catastrophic Forgetting
Anchor-Conditioned Compositional Control for Landscape Image Generation
Jun 1, 2026Gadha Lekshmi P, Govind Arun, Rohith Syam +1Large-Scale Image Generation Generative Models
Trait-space Monitoring for Emergent Misalignment During Supervised Finetuning
May 31, 2026Huy Nghiem, Sy-Tuyen Ho, Sarah Wiegreffe +1Harmful Fine-Tuning Misalignment
Balanced LoRA: Removing Parameter Invariance to Accelerate Convergence
May 29, 2026Valérie Castin, Kimia Nadjahi, Pierre Ablin +1Low-Rank Adaptation Finetuning
Forgetting Has Neighbors: Localized Collateral Forgetting in Machine Unlearning
May 29, 2026Polina Dolgova, Sebastian U. StichMachine Unlearning Forgetting
Transforming and Encoding FTS for SAT Solving: What Helps, What Hurts (Extended Version)
May 28, 2026João Filipe, Álvaro Torralba, Gregor BehnkeSatisfiability Classical Planning
Efficient Test-Time Finetuning of LLMs via Convex Reconstruction and Gradient Caching
May 28, 2026Alaa Khamis, Alaa MaaloufFinetuning Test Time
IP-Adapter Is All You Need: Towards Fine-Tuning-Free Diffusion-Based Talking Face Generation
May 28, 2026Hao Wu, Xiangyang Luo, Hao Wang +3Object Articulation Few-Step Generation
Entropy-KL Divergence-based Token Masking: A Novel Approach for Selective Fine-tuning of Large Language Models
May 28, 2026Qi Liu, Mingdi Sun, Yongyi He +5Finetuning LLM Post-Training Methods
Bilinear Coordinate Alignment for Training-Free Task-Vector Transfer
May 27, 2026Jungyong Son, Jinwook Jung, Minhee Park +1Transfer Learning Model Pretraining
Unsupervised Identification and Removal of Spurious Correlations During Fine-Tuning
May 26, 2026Ciarán M. Gilligan-Lee, Joseph Egan, Yuchen Zhu +1Finetuning Spurious Correlations
Why LLMs Fail at Causal Discovery and How Interventional Agents Escape
May 26, 2026Amartya Roy, Sonali ParbhooLarge Language Model Performance Finetuning
ReMoE: Boosting Expert Reuse through Router Fine-Tuning in Memory-Constrained MoE LLM Inference
May 26, 2026Xiongwei Zhu, Xiaojian Liao, Tianyang Jiang +3Mixture-Of-Expert Inference LLM Inference Efficiency
Scaling World-Model Reinforcement Learning Through Diffusion Policy Optimization
May 25, 2026Xiaoyuan Cheng, Wenxuan Yuan, Zhancun Mu +5Diffusion Policies Multi-Objective Policy Optimization
Thaka at KSAA-2026 Task 2: Regularized Fine-Tuning for Arabic Speech Diacritization
May 25, 2026Meshal Alamr, Hassan Alqaeri, Abdullah AldahlawiSpeech Encoder Arabic Natural Language Processing
Reading the Finetuning Prior: Verbatim Content Recovery via Contrastive Decoding Diffing
May 25, 2026Michał Brzozowski, Zuzanna Dubanowska, Enrico Cassano +1Contrastive Decoding Finetuning
Phantom transitions in language model fine-tuning
May 25, 2026Vaibhav Prakash, Jayasri DontabhaktuniLarge Language Model Fine-Tuning Finetuning
RepSAM: Bridging Foundation Models to Robotic Vision via Representation-Guided Adaptation
May 25, 2026Wenhui ChuRobotic Perception Vision Foundation Models
Jailbreak to Protect: Buffering and Reinforcing via Temporary Jailbreaking for Safe Fine-Tuning in Large Language Models
May 23, 2026Seokil Ham, Jaehyuk Jang, Wonjun Lee +1Harmful Fine-Tuning Jailbreak Evaluations
Understanding and Improving Noisy Embedding Techniques in Instruction Finetuning
May 22, 2026Abhay YadavFinetuning Domain-Invariant Text Embeddings
Live Music Diffusion Models: Efficient Fine-Tuning and Post-Training of Interactive Diffusion Music Generators
May 21, 2026Zachary Novack, Stephen Brade, Haven Kim +8Music Generation Conversational Music Recommendation
PRISM: Preference-Aware Influence Function Based Data Selection Method for Efficient Fine-Tuning
May 20, 2026Qihao Lin, Guanxu Chen, Dongrui Liu +1Finetuning Efficient Influence Function
PREFINE: Preference-Based Implicit Reward and Cost Fine-Tuning for Safety Alignment
May 20, 2026Richa Verma, Bavish Kulur, Sanjay Chawla +1Safety Alignment Finetuning
Most Transformer Modifications Still Do Not Transfer at 1-3B: A 2020-2026 Update to Narang et al. (2021) with Downstream Evaluation and a Noise Floor
May 20, 2026Yang Zhao, Jiahao Lu, Bin Huang +2Finetuning Loss Function
An exponential mechanism based on quadratic approximations for fine-tuning machine learning models with privacy guarantees
May 19, 2026Hoang Tran, Jorge Ramirez, Jiayi Wang +3Differential Privacy Finetuning
Spectral Unforgetting: Post-Hoc Recovery of Damaged Capabilities Without Retraining
May 19, 2026Aarash Abro, Muhammad TahirFinetuning Catastrophic Forgetting
LoRA vs. Full Fine-Tuning: A Theoretical Perspective
May 18, 2026Ali Zindari, Rotem Mulayoff, Sebastian U. StichLow-Rank Adaptation Finetuning
Distilling Linearized Behavior into Non-Linear Fine-Tuning for Effective Task Arithmetic
May 18, 2026Thomas Sommariva, Francesca Morandi, Simone Calderara +1Finetuning Model Pretraining
DP-SelFT: Differentially Private Selective Fine-Tuning for Large Language Models
May 17, 2026Haichao Sha, Zihao Wang, Yuncheng Wu +2Differential Privacy Finetuning
TSFLora: Token-Compressed Split Fine-Tuning for Wireless Edge Networks
May 17, 2026Xianke Qiang, Zheng Chang, Li Wang +1Heterogeneous Edge Devices Parameter-Efficient Fine-Tuning
Sparse Autoencoders enable Robust and Interpretable Fine-tuning of CLIP models
May 15, 2026Fabian Morelli, Arnas Uselis, Ankit Sonthalia +1Supervised Fine-Tuning Large Vision Language Models
Training ML Models with Predictable Failures
May 14, 2026Will Schwarzer, Scott NiekumEarly Failure Prediction Model Evaluation
PreFT: Prefill-only finetuning for efficient inference
May 14, 2026Andrew Lanpouthakoun, Aryaman Arora, Zhengxuan Wu +4Parameter-Efficient Fine-Tuning Prefill
LIFT: Last-Mile Fine-Tuning for Table Explicitation
May 13, 2026Divij Khaitan, Ashish TiwariTabular Data Large Language Model Fine-Tuning
PRISM-X: Experiments on Personalised Fine-Tuning with Human and Simulated Users
May 13, 2026Hannah Rose Kirk, Liu Leqi, Fanzhi Zeng +4Personalization User Simulation
CRAFT: Clinical Reward-Aligned Finetuning for Medical Image Synthesis
May 12, 2026Yunsung Chung, Alex El Darzi, Carlo El Khoury +3Medical Image Generation Clinical
Hybrid-LoRA: Bridging Full Fine-Tuning and Low-Rank Adaptation for Post-Training
May 12, 2026Chengqian Zhang, Wei Zhu, Kyumin LeeLow-Rank Adaptation Finetuning
Probabilistic Calibration Is a Trainable Capability in Language Models
May 12, 2026Davide Baldelli, Sruthi Kuriakose, Maryam Hashemzadeh +2Training Language Models Finetuning
Newton's Lantern: A Reinforcement Learning Framework for Finetuning AC Power Flow Warm Start Models
May 11, 2026Shourya Bose, Helgi Hilmarsson, Dhruv SuriAc Optimal Power Flow Newton
CapVector: Learning Transferable Capability Vectors in Parametric Space for Vision-Language-Action Models
May 11, 2026Wenxuan Song, Han Zhao, Fuhao Li +7Vision-Language Model Adaptation Vision-Language-Action Models
V4FinBench: Benchmarking Tabular Foundation Models, LLMs, and Standard Methods on Corporate Bankruptcy Prediction
May 11, 2026Marcin Kostrzewa, Sebastian Tomczak, Roman Furman +5Credit Risk Prediction Tabular Foundation Models
S2FT: Parameter-Efficient Fine-Tuning in Sparse Spectrum Domain
May 9, 2026Baoquan Zhang, Zhehao Yu, Lisai Zhang +5Parameter-Efficient Fine-Tuning Finetuning
Pretraining Induces a Reusable Spectral Basis for Downstream Task Adaptation
May 8, 2026Junjie Yu, Yue Wang, Zihan Deng +3Model Pretraining Finetuning
Beyond LoRA vs. Full Fine-Tuning: Gradient-Guided Optimizer Routing for LLM Adaptation
May 8, 2026Haozhan Tang, Xiuqi Zhu, Xinyin Zhang +3Low-Rank Adaptation Large Language Model Adaptation
From Parameter Dynamics to Risk Scoring : Quantifying Sample-Level Safety Degradation in LLM Fine-tuning
May 6, 2026Xiao Wang, Yifei Zhang, YongKang Liu +4Large Language Model Safety Large Language Model Fine-Tuning
Stabilizing LLM Supervised Fine-Tuning via Explicit Distributional Control
May 6, 2026Xinyu Wang, Changzhi Sun, Yuanbin Wu +1Supervised Fine-Tuning Finetuning
Benchmarking Logistic Regression, SVM, Naive Bayes, and IndoBERT Fine-Tuning for Sentiment Analysis on Indonesian Product Reviews
May 5, 2026Nabila Zakiyah Zahra, Salwa Farhanatussaidah, Nasywa Nur Afifah +3Term Frequency-Inverse Document Frequency Indonesian E-Commerce Reviews
Generalized Distributional Alignment Games for Unbiased Answer-Level Fine-Tuning
May 4, 2026Mehryar Mohri, Jon Schneider, Yutao ZhongKullback-Leibler Divergence Training-Inference Mismatch
Fine-Tuning Impairs the Balancedness of Foundation Models in Long-tailed Personalized Federated Learning
May 4, 2026Shihao Hou, Chikai Shang, Zhiheng Yang +5Federated Learning Finetuning

Jun 12, 2026 · cs.LGJ/K move · Enter open · S save

Sticky Routing: Training MoE Models for Memory-Efficient Inference

Ali Kayyam

BrainChip Inc., Laguna Hills, CA

Mixture-of-Experts (MoE) models activate only a sparse subset of experts per token, yet consecutive tokens frequently activate different experts -- causing constant weight swapping between slow storage and fast memory on edge devices. Existing remedies are either system-level (caching heuristics) or post-hoc (router fine-tuning), leaving the root cause unchanged during pretraining. We propose StickyMoE, a differentiable routing consistency loss that penalises abrupt expert switches between adjacent tokens, encouraging the router to maintain the same expert assignment across semantically coherent spans. StickyMoE requires no architectural changes, adds a single hyperparameter lambda, and unlike post-hoc methods, allows expert representations and routing decisions to co-adapt from the first training step. Experiments on small-scale MoE language models show that StickyMoE reduces the expert switch rate by up to 60% with less than 4% perplexity degradation, Pareto-dominating post-hoc fine-tuning on the quality-locality frontier. Routing temporal locality is most efficiently instilled at training time.

Mixture-Of-Expert Language Models Mixture-of-Experts Models Finetuning

Open paper Ask about it PDF

Recent momentum

Neighbouring directions

Recent digests

7 new papers

3 new papers

4 new papers

Latest in Finetuning