LLM Fine-Tuning

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

42 papers in the last four weeks, up 83% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 509

All topics

Rethinking the Multilingual Reasoning Gap with Layer Swap
May 26, 2026Maxence Lasbordes, Amélie Chatelain, Djamé SeddahMultilingual Language Models Multilingual Language Model Evaluation
1LightOn, Paris · 2Inria, Paris
Open-Weight LLM Fine-Tuning Defenses are Susceptible to Simple Attacks
May 26, 2026Kevin Kuo, Virginia Smith, Chhavi YadavLLM Fine-Tuning Adversarial Attacks on LLMs
Carnegie Mellon University · Simons Institute, UC Berkeley
Forgetting in Language Models: Capacity, Optimization, and Self-Generated Replay
May 25, 2026Martin Marek, Dongkyu Cho, Shikai Qiu +3Continual Learning for LLMs LLM Fine-Tuning
New York University
Context-Instrumental Data Distillation for Kubernetes Manifest Generation: Method and Experimental Evaluation
May 25, 2026Andrey Kozachok, Anatoliy Bakaev, Aleksandr Kozachok +2LLM Fine-Tuning Code Generation
RTU MIREA, Moscow, Russia
Phantom transitions in language model fine-tuning
May 25, 2026Vaibhav Prakash, Jayasri DontabhaktuniLLM Fine-Tuning
Department of Physics, Mahindra University, Hyderabad, India
Fine-Tuning and Serving Gemma 4 31B on Google Cloud TPU: A Technical Comparison with GPU Baselines
May 25, 2026Jatin Kishnani, Mayank Goel, Amit Singh +2AI Accelerator Inference LLM Serving
RotMoLE: Enhancing Mixture of Low-Rank Experts through Rotational Gating Mechanism
May 25, 2026Mengyang Sun, Maochuan Dou, Tao Feng +5LLM Fine-Tuning Low-Rank Adaptation
Tsinghua University · Beijing Information Science and Technology University · National University of Singapore +2
Security in the Fine-Tuning Lifecycle of Large Language Models: Threats, Defenses,Evaluation, and Future Directions
May 24, 2026Wenjuan Li, Yitao Liu, Runze Chen +1LLM Backdoor Attacks LLM Fine-Tuning
Hangzhou Normal University, No. 2318, Yuhangtang Rd, Hangzhou, 311121, Zhejiang, China · Zhejiang University, No. 866 Yuhangtang Rd, Hangzhou, 2026 310058, Zhejiang, China · China Mobile (Zhejiang) Innovation Research Institute Co., Ltd., No. 77 Gaoxin 7th Rd, China Mobile Zhejiang Information and Communication Industrial Park, Hangzhou, 311200, Zhejiang, China +1
DTO: a Differentiable Training Objective for Effective Counterfactual Story Rewriting
May 24, 2026Amelia Girard, Massimo PiccardiLLM Fine-Tuning
University of Technology Sydney, Sydney, NSW 2007, Australia
Efficient DP-SGD for LLMs with Randomized Clipping
May 24, 2026Enayat Ullah, Sai Aparna Aketi, Devansh Gupta +2Privacy-Preserving Language Models Differentially Private Stochastic Gradient Descent
Meta Platforms Inc. · University of Southern California
Tiny Brains, Giant Impact: Uncovering the Keystone Neurons of LLM with Just a Few Prompts
May 24, 2026Xiangtian Ji, Yuxin Chen, Zhengzhou Cai +3Fine-Tuning LLM Interpretability
National University of Singapore · Beijing University of Posts and Telecommunications · University of Science and Technology of China
Geo-Expert: Towards Expert-Level Geological Reasoning via Parameter-Efficient Fine-Tuning
May 24, 2026Chenyou Guo, Zongqi Liu, Yizhou Zhang +2Scientific Reasoning LLM Fine-Tuning
Ocean University of China · Peking University · Monash University
Mix-MoE: Improving Multilingual Machine Translation of Large Language Models through Mixed MoEs
May 23, 2026Bo Li, Tianyu Dong, Shaolin Zhu +1Multilingual Language Models Mixture-of-Experts Language Models
School of Software, Tsinghua University · College of Intelligence and Computing, Tianjin University
ReLoRA: Knowledge-Reusing Adaptation for Fast Rollout of Evolving LLM Services
May 23, 2026Yang Xu, Zihuai Xu, Hongli Xu +3LLM Serving LLM Fine-Tuning
Suzhou Institute for Advanced Research, University of Science and Technology of China, Suzhou, Jiangsu, China, 215123
Jailbreak to Protect: Buffering and Reinforcing via Temporary Jailbreaking for Safe Fine-Tuning in Large Language Models
May 23, 2026Seokil Ham, Jaehyuk Jang, Wonjun Lee +1LLM Fine-Tuning LLM Safety Alignment
School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea.
PALoRA: Projection-Adaptive LoRA for Preserving Reasoning in Large Language Models
May 23, 2026Mustafa Hayri Bilgin, Mariam Barry, Albert Bifet +2Knowledge Editing Continual Learning for LLMs
IT Group, Research, BNP Paribas · LTCI, Télécom Paris, Institut Polytechnique de Paris · AI Institute, University of Waikato +1
HyperGuide: Hyperbolic Guidance for Efficient Multi-Step Reasoning in Large Language Models
May 22, 2026Yuyu Liu, Haotian Xu, Yanan He +3LLM Fine-Tuning Efficient Language Model Reasoning
Department of Computer Science Stony Brook University · Department of Applied Mathematics and Statistics Stony Brook University · Department of Computer Science Yale University +2
Structure-Guided Entity Resolution: Fine-Tuning LLMs for Robust Name Matching in Complex Linguistic Contexts
May 22, 2026Shivam Chourasia, Hitesh Kapoor, Nilesh PatilEntity Resolution LLM Fine-Tuning
Dream Sports
Convex Optimization for Alignment and Preference Learning on a Single GPU
May 22, 2026Miria Feng, Mert PilanciPairwise Preference Learning LLM Alignment
Department of Electrical Engineering, Stanford University, California, United States.
Understanding and Improving Noisy Embedding Techniques in Instruction Finetuning
May 22, 2026Abhay YadavLLM Fine-Tuning Instruction Tuning
Johns Hopkins University Baltimore, MD, USA
PoisonForge: Task-Level Targeted Poisoning Benchmark for Instruction-Tuned LLMs
May 22, 2026Luze Sun, Anshuman Suri, Harsh Chaudhari +2LLM Safety Benchmarks LLM Backdoor Attacks
Department of Computer Science Northeastern University · DatologyAI
Signs Beat Floats: Low-Rank Double-Binary Adaptation for On-Device Fine-Tuning
May 22, 2026Yoshihiko Fujisawa, Yuma Ichikawa, Yudai Fujimoto +2LLM Quantization Memory-Efficient Fine-Tuning
Fujitsu Limited · Institute of Science Tokyo · RIKEN Center for AIP +1
Truthful Online Preference Aggregation for LLM Fine-Tuning in Mobile Crowdsourcing
May 22, 2026Shugang Hao, Lingjie DuanMechanism Design LLM Fine-Tuning
Singapore Wireless Innovation Centre, Singapore University of Technology and Design, Singapore, 487372 Singapore · Internet of Things Thrust and the Artificial Intelligence Thrust, Hong Kong University of Science and Technology, Guangzhou 511455, China
Beyond Temperature: Hyperfitting as a Late-Stage Geometric Expansion
May 21, 2026Meimingwei Li, Yuanhao Ding, Esteban Garces Arias +1LLM Interpretability LLM Fine-Tuning
Department of Statistics, LMU Munich · School of Computer and Information Engineering, Henan University · Munich Center for Machine Learning (MCML)
BeLink: Biomedical Entity Linking Meets Generative Re-Ranking
May 21, 2026Darya Shlyk, Stefano Montanelli, Lawrence HunterLLM Fine-Tuning LLM Reranking
University of Milan Milan, Italy · University of Chicago Chicago, IL, USA
Polite on the Surface, Broken in Practice: A Curated Dataset for Fixing Generation and Register Failures in Low-Resource Bangla Text Generation
May 21, 2026Md. Asaduzzaman Shuvo, Mahedi Hasan, Md. Tashin Parvez +2Multilingual Language Model Evaluation LLM Fine-Tuning
United International University, Bangladesh
Modeling Pathology-Like Behavioral Patterns in Language Models Through Behavioral Fine-Tuning
May 21, 2026Nicola Milano, Davide MaroccoLLM Fine-Tuning Personality Modeling in Language Models
University of Naples Federico II, Natural and Artificial Cognition Laboratory ”Orazio Miglino”, Department of Humanities
Models Can Model, But Can't Bind: Structured Grounding in Text-to-Optimization
May 20, 2026Zhiqi Gao, Albert Ge, Alexander Berenbeim +2LLM Grounding LLM Fine-Tuning
University of Wisconsin-Madison · United States Military Academy
torchtune: PyTorch native post-training library
May 20, 2026Mark Obozov, Maxime Griot, Joseph Cummings +8LLM Fine-Tuning Efficient Language Model Training
Work done at PyTorch, Meta · Stanford · Meta-FAIR
PRISM: Preference-Aware Influence Function Based Data Selection Method for Efficient Fine-Tuning
May 20, 2026Qihao Lin, Guanxu Chen, Dongrui Liu +1Supervised Fine-Tuning LLM Fine-Tuning
Shanghai Artificial Intelligence Laboratory

May 26, 2026, cs.CLJ/K move · Enter open · S save

Rethinking the Multilingual Reasoning Gap with Layer Swap

Maxence Lasbordes, Amélie Chatelain, Djamé Seddah

1LightOn, Paris · 2Inria, Paris

Multilingual Language Models Multilingual Language Model Evaluation Cross-Lingual Reasoning in Language Models LLM Fine-Tuning

Open paper AskPDF arXiv

Recent reasoning Large Language Models produce a chain-of-thought (CoT) predominantly in English, even when prompted in non-English languages. Prior work suggests that forcing the CoT to remain in the input language (\emph{native reasoning}) substantially degrades performance relative to allowing the model to reason in English before answering in the input language (\emph{English-pivoted reasoning}). However, most studies of this native reasoning gap rely on inference-time interventions or limited native-language training data. We revisit this comparison at a larger scale and under comparable supervision. We construct long multilingual reasoning datasets across six languages (English, French, German, Spanish, Chinese and Swahili); fine-tune specialists in both native and English-pivoted regimes on top of \texttt{Qwen/Qwen3-8B-Base}, and evaluate across mathematics, science, general knowledge, and code. In this setting, the average native reasoning gap shrinks to 1.9--3.5% across the five non-English languages, considerably smaller than previously reported. Weight-space analysis of the native specialists reveals aligned fine-tuning updates in the middle layers and divergence in the outer layers. This points to a largely language-agnostic reasoning core surrounded by language-specific layers. Exploiting this structure, we introduce a Layer Swap: transferring the English specialist's stronger reasoning mid-layers into each native specialist, closing most of the native reasoning gap across the five non-English languages while preserving CoT in the target language. We release all models and datasets.

Momentum

Related topics

Latest papers 509