Fine-Tuning Language Models

Follow this topic Ask about this topic

Latest papers

63 papers21 papers in the last four weeks, up 600% on the four weeks before. 0.1% of all new papers.

Dataset Collection and Fine-Tuning of NLLB for German--English Translation
Nov 5, 2026SK Deeraj, Yash Ingle, Pruthwik MishraMachine Translation Fine-Tuning Language Models
Department of Artificial Intelligence, Sardar Vallabhbhai National Institute of Technology, India
Rules, Traces and Routing Statistics: Training and Calibration Data for Open Models on ARC-AGI-2
Nov 4, 2026Nguyen Minh Thang, Trinh Thi PhuongARC-AGI Fine-Tuning Language Models
VNU University of Science, 334 Nguyen Trai, Thanh Xuan, Hanoi, Vietnam
On the Opportunities and Challenges of Prefill-to-Autoregressive Cross-Model Transfer
Nov 4, 2026Zhenyi Shen, Junru Lu, Wei Liu +8Multi-LLM Collaboration Transfer Learning
King’s College London, UK · Tencent Youtu Lab, China · National Physical Laboratory, UK
AssameseLM-E2B: An Open-Weight Assamese Language Model for Efficient Consumer-Grade Deployment
Nov 4, 2026Pragyan PhukanLanguage Modeling Low-Resource Language Processing
Department of Physics, Moran College, Charaideo, Assam, India, 785670
Unifying and Beyond the Theory of Pervasive Capability Degradation in LLM Post-training SFT from Feature Interference
Nov 3, 2026Hang Chen, Jiaying Zhu, Wenya WangLanguage Model Post-Training Fine-Tuning Language Models
College of Computing and Data Science, Nanyang Technological University · School of Computer Science and Engineering, The Chinese University of Hong Kong
Training Specialist LLMs on Your Phone: Forward-Only Optimization via Block-Spectral Coordinates
Nov 3, 2026Liyan Tan, Yequan Zhao, Ziyue Liu +3On-Device Training LLM Training
Department of Electrical and Computer Engineering, University of California, Santa Barbara, CA, USA · Department of Computer Science, University of California, Santa Barbara, CA, USA
The Evolution Strategies Paradox: Better Models from Worse Perturbations
Nov 3, 2026Roberto Dailey, Kajetan Schweighofer, Jamieson Warner +5Evolution Strategies Fine-Tuning Language Models
Cognizant AI Lab, San Francisco · The University of Texas at Austin, Austin
SQLoom: Set-Contrastive Learning of Syntactic, Semantic, and Logical Dependencies for Text-to-SQL
Nov 2, 2026Hao Fu, Degang Sun, Wan Wei +3Text-to-SQL Fine-Tuning Language Models
Virginia Tech · Computer Network Information Center, Chinese Academy of Sciences
What Breaks and What Comes Back: Capability Damage and Recovery in Depth-Pruned Small Language Models
Oct 31, 2026Piyush Dhoka, Vedant Pandhare, Varun Inamdar +2LLM Pruning Transformer Pruning
Department of Artificial Intelligence, Vishwakarma University, Pune, India
Less Loss, Fixed Weights: Logit Corrections for Forgetting
Oct 30, 2026Hanyu YangCatastrophic Forgetting Fine-Tuning Language Models
Breaking Information Islands in Sparse Tuning via Small-World Connectivity
Oct 30, 2026Xianchao Guan, Zijun Xiong, Yifeng Wang +5Sparse Fine-Tuning Fine-Tuning Language Models
Harbin Institute of Technology, Shenzhen, China · Pengcheng Laboratory, China · Tsinghua University, China +1
Saidzoda-Tajik-LFM-2.6B: Adapting a 2.6B Hybrid Model to Tajik Tool Calling, and What Continued Pretraining Takes Away
Oct 30, 2026Tohir Saidzoda, Nekruz Maxudov, Bedil Karimov +5LLM Tool Use Fine-Tuning Language Models
Saidzoda AI Lab, Saidzoda Engineering Company, Dushanbe, Tajikistan
KV-Cache Patching: Tracing How and Where Fine-Tuned Behaviors Arise
Oct 30, 2026Jonas Loos, Oliver EberleFine-Tuning Language Models Causal Interventions in Language Models
Machine Learning Group, Technische Universität Berlin, Berlin, Germany · Berlin Institute for the Foundations of Learning and Data (BIFOLD), Berlin, Germany · Zuse School ELIZA, Darmstadt, Germany
HyperRepair: Recovering Forgotten General Capabilities of Fine-Tuned Language Models
Oct 30, 2026Junrong Tang, Jianghui Wang, Hao Kong +2Fine-Tuning Language Models Catastrophic Forgetting
Shanghai Jiao Tong University · The University of Queensland
SimplexLoRA: Dynamic Rank Updating via Alternating Minimization with Simplex Projection
Oct 27, 2026Igor Shalygin, Grigorii Davydenko, Andrey Veprikov +4Low-Rank Adaptation Fine-Tuning Language Models
Moscow Independent Research Institute of Artificial Intelligence (MIRAI) · Basic Research of Artificial Intelligence Laboratory (BRAIn Lab) · SB AI Lab +2
Task Format as a Factor in Adaptation: A Controlled Comparison of LoRA and Full Fine-Tuning for Prompt-Safety Classification
Oct 24, 2026John Alderete, Srirakshith Betageri, Jennifer Flynn +3Fine-Tuning Language Models Low-Rank Adaptation
Fiddler AI · Simon Fraser University · Georgia Institute of Technology
One Model, Two Paths: Balancing Single-Pass and Looped Performance in Language Models
Oct 24, 2026Jiarun Shen, Lei Zhao, Xiao LiLLM Inference Fine-Tuning Language Models
The Chinese University of Hong Kong, Shenzhen · Shanghai Jiao Tong University
CrisiLangs: A Multilingual Resource for Crisis Warning Generation
Oct 22, 2026Giacomo Gonella, Stefano Menini, Marco GueriniMultilingual Language Models Early Warning Detection
Fondazione Bruno Kessler, Italy · University of Trento, Italy
Task-Preserving Multilingual Instruction Tuning for Open-Ended QA in WMT 2026 MIST
Oct 21, 2026Aleix Sant, Jordi Luque, Carlos EscolanoMultilingual QA Fine-Tuning Language Models
Scientific Research, Telefónica Innovación Digital · Universitat Politècnica de Catalunya
ECHO: Emergent Common-Mode Hidden Offsets Drive Vocabulary Collapse in Preference Optimization
Oct 21, 2026Yun Li, Ehsan Javanmardi, Simon Thompson +7Direct Preference Optimization Fine-Tuning Language Models
Graduate School of Information Science and Technology, The University of Tokyo, Tokyo, Japan · TIER IV, Inc., Tokyo, Japan · Huawei
NormTune: Norm-Guided Parameter-Efficient Fine-Tuning for Reasoning-Intensive Tasks
Oct 21, 2026Zhangming Li, Qinghao Hu, Weifan Guan +3Fine-Tuning Language Models Parameter-Efficient Fine-Tuning
1The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · 3AiRiA +1
How Fine-Tuning Objectives Shape Layer-Wise Information in LLM Hallucination Detection
Oct 20, 2026Kaiyang Wan, Forrest Sheng Bao, Amin Ahmad +1Fine-Tuning Language Models LLM Hallucination Detection
INSAIT, Sofia University “St. Kliment Ohridski”, Sofia, Bulgaria · PAIR Systems, Inc., Mountain View, California
CEFR-oriented Russian Text Simplification Using Reinforcement Learning
Oct 19, 2026Kristina Mustakova, Alexey SorokinText Simplification Fine-Tuning Language Models
MSU Center of Artificial Intelligence · Yandex
BERT Is Already a Denoiser: Cheaply Unlocking Iterative Generation in Masked Language Models
Oct 19, 2026Julio C. Amador Diaz LopezLanguage Modeling Conditional Generation
*SiftyML, LTD
Training with Ciphers to Enhance In-Context Terminology Learning For Translation from Rare Languages
Oct 17, 2026Deevyanshu Malu, Deeptanshu Malu, Sunita SarawagiMachine Translation In-Context Learning
IIT Bombay
Behavioral Couplings: From Formation to Behavioral Emergence in Language Models
Oct 16, 2026Dongyoon Hahm, Woogyeol Jin, Dennis Wei +3Emergent Misalignment in Language Models Fine-Tuning Language Models
KAIST · IBM Research
Rank Measures of Fine-Tuning Updates Need a Noise Baseline
Oct 15, 2026Muhammad Usman AltafFine-Tuning Language Models Neural Network Training Dynamics
Independent Researcher
How Low Can You Go: Fine-Tuning LLMs with Bayesian Optimization (LlMBO)
Oct 13, 2026Aditya Kulkarni, Davide Sartor, Carlos Soto +1LLM Fine-Tuning Fine-Tuning Language Models
Department of Physics, University of Massachusetts Amherst · Department of Mathematics and Statistics, University of Massachusetts Amherst
Agaram 2.5B: The First Tamil-First Language Model Trained from Scratch with Tool Calling. A Playbook for Under-Represented Languages
Oct 13, 2026Ramchand Kumaresan, Madhusudhanan G, Kaushik KezendiraneLanguage Modeling Low-Resource Language Processing
Murai Labs · Independent researcher
Bayesian Fine-tuning Yields Language Models that are as Bayesian as their Beliefs Allow
Sep 30, 2026Polina Tsvilodub, Andreas Waldis, Linlu Qiu +2Fine-Tuning LLM Fine-Tuning
University of Tübingen · Massachusetts Institute of Technology · New York University

Nov 5, 2026J/K move · Enter open · S save

Dataset Collection and Fine-Tuning of NLLB for German--English Translation

SK Deeraj, Yash Ingle, Pruthwik Mishra

Department of Artificial Intelligence, Sardar Vallabhbhai National Institute of Technology, India

Machine Translation Fine-Tuning Language Models Training Data Curation

Open paper AskPDF arXiv

This paper describes the curation of a high-quality parallel corpus from OPUS Books and the fine-tuning of Meta's NLLB-200 models for German--English translation. We detail a dataset extraction pipeline that filters 45k raw OPUS sentence pairs to 25k high-quality examples through encoding validation, length filtering, language identification, deduplication, and quality scoring. We present a reproducible fine-tuning methodology optimized for consumer GPUs using mixed precision, gradient accumulation, and gradient checkpointing. Our fine-tuned NLLB-600M model achieves statistically significant improvements (+3.4 BLEU, +2.6 chrF++, -3.2 TER) over the base model on literary and technical German-English translation. We demonstrate practical deployment by integrating the fine-tuned model into LinguaHaru, an existing open-source document translation system, achieving format-preserving translation across PDF and Office documents. All dataset preparation scripts, training code, and evaluation tools are provided for reproducibility. The fine-tuned model is available at HuggingFace.