Fine-Tuning

Follow this topic Ask about this topic

Latest papers 205

31 papers in the last four weeks, up 72% on the four weeks before. 0.2% of all new papers.

All topics

Punching Above Their Weight: Classification-Head Fine-Tuning of Tiny Language Models (TLMs) for Verifiable Multiple-Choice Tasks
Jul 4, 2026Bhavesh Sood, Jaromir SavelkaFine-Tuning Multiple-Choice Question Answering
Carnegie Mellon University
Efficient PEFT Methods with Adaptive Checkpointing for Vision Models and VLMs on Resource Constrained Consumer-GPUs
Jul 2, 2026Altay Toktassyn, Jurn-Gyu ParkFine-Tuning Memory-Efficient Fine-Tuning
Department of Computer Science, Nazarbayev University, Astana 010000, Kazakhstan
Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Training
Jul 1, 2026Zijian Zhang, Rizhen Hu, Athanasios Glentis +4RL for Language Models Fine-Tuning
University of Minnesota · Peking University · Amazon
Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Use
Jul 1, 2026Song-Lin Lv, Weiming Wu, Rui Zhu +2Fine-Tuning Distribution Shift
School of Intelligence Science and Technology, Nanjing University, Nanjing, China · National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China
Loss Smoothing for Stable Adaptation Under Distribution Shift
Jul 1, 2026Darshan Patil, Ekaterina Lobacheva, Razvan Pascanu +1Fine-Tuning RL Fine-Tuning
Chandar Research Lab · Mila – Quebec AI Institute · Université de Montréal +2
Mixture-of-Control: State-Aware Fine-Tuning for Transformer-based Models
Jun 30, 2026Duc Anh Nguyen, Tien Ngoc Luu, Tung Pham +1Fine-Tuning Transformer
Qualcomm AI Research
Fora: From Weight-Space to Function-Space Protection in Capability-Preserving Fine-Tuning
Jun 30, 2026Rui Zhou, Tianci XieFine-Tuning LLM Fine-Tuning
Dalian University of Technology · Zhejiang University
Defending Against Harmful Supervision Hidden in Benign Samples
Jun 29, 2026Bang An, Yibo Yang, Dandan Guo +3Supervised Fine-Tuning Fine-Tuning
1King Abdullah University of Science and Technology · School of Artificial Intelligence, Jilin University
KnowsTFM: Knowledge-Informed Fine-Tuning of Small Tabular Foundation Models
Jun 29, 2026Boshko Koloski, Xiangjian Jiang, Senja Pollak +3Tabular Foundation Models Fine-Tuning
Jožef Stefan Institute and Postgraduate School, SI · Department of Computer Science and Technology, University of Cambridge, UK · Télécom Paris, Institut Polytechnique de Paris, FR
DNA Language Models: An Assessment of Pre-Training for Fine-Tuning Tasks
Jun 29, 2026Romain Karpinsky, Julien Mozziconacci, Mickaël DelceyLanguage Model Pretraining Fine-Tuning
CNRS, LORIA · Nancy, 54000, France · CNRS +1
Optimizer Memory Makes Shuffle Order a First-Order Source of Fine-Tuning Noise
Jun 28, 2026John SweeneyStochastic Optimization Fine-Tuning
A Gravitational Interpretation of Safety Reversion under Fine-Tuning
Jun 26, 2026Samuele Poppi, Nils LukasFine-Tuning LLM Safety Alignment
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), UAE
Matching Tasks to Objectives: Fine-Tuning and Prompt-Tuning Strategies for Encoder-Decoder Pre-trained Language Models
Jun 23, 2026Ahmad Pouramini, Hesham FailiLanguage Model Pretraining Fine-Tuning
School of Electrical and Computer Engineering, College of Engineering, University of Tehran, North Kargar Street, Tehran, 515-14395, Tehran, Iran.
TrOCR for Medieval HTR: A Systematic Ablation Study with Cross-Dataset Validation
Jun 23, 2026Sachin Sharma, Michele Flammini, Federico SimonettaFine-Tuning VLM Adaptation
GSSI – Gran Sasso Science Institute L’Aquila, Italy
A Comparative Study of Pretrained Transformer Models for Quranic ASR: Speech Representations, Label Formats, and Dataset Composition
Jun 18, 2026Nabil Mosharraf Hossain, Riasat Islam, Unaizah ObaidellahFine-Tuning Automatic Speech Recognition
Greentech Apps Foundation, United Kingdom · Greentech Apps Foundation, United Kingdom. · Queen Mary University of London, United Kingdom +3
Improving low-resource ASR using bilingual fine-tuning with language identification: a cross-linguistic evaluation
Jun 16, 2026Reihaneh Amooie, Yun Hao, Wietse de Vries +3Fine-Tuning Language Identification
University of Groningen · Fryske Akademy · Vrije Universiteit Brussel
TuneAhead: Predicting Fine-tuning Performance Before Full Training Begins
Jun 16, 2026Yuxiang Luo, Haonan Long, Chen Wang +6Fine-Tuning LLM Fine-Tuning
The Hong Kong University of Science and Technology (Guangzhou) · Huawei Technologies Ltd.
A Risk Decomposition Framework for Pre-Hoc Fine-Tuning Prediction
Jun 16, 2026Yuxiang Luo, Chen Wang, Nan TangFine-Tuning LLM Fine-Tuning
The Hong Kong University of Science and Technology (Guangzhou).
ReRAM-aware Model Finetuning addressing I-V Non-linearity and Retention Errors
Jun 16, 2026Ching-Yi Lin, Shamik Kundu, Arnab Raha +1Compute-in-Memory Fine-Tuning
University of Maryland, College Park, MD, USA · Intel Corporation, 2200 Mission College Blvd, Santa Clara, CA 95054, USA
CogGuard: Cognitive and Operational Profiling for Proactive Warning in Edge Intelligent Services
Jun 13, 2026Zhi Yao, Weihao Chen, Zhiqing Tang +4Fine-Tuning User Modeling
Institute of Artificial Intelligence and Future Networks, Beijing Normal University, Zhuhai, China · Faculty of Arts and Science, Beijing Normal University, Zhuhai, China · School of Artificial Intelligence, Beijing Normal University, Beijing, China +2
Parameter-Efficient Adapter Tuning for Tabular-Image Multimodal Learning
Jun 10, 2026Jiaqi LuoFine-Tuning Adapter Tuning
School of Mathematical Sciences, Soochow University, No.1 Shizi Street, Suzhou, 215006, Jiangsu Province, China
A Unifying Lens on Supervised Fine-Tuning Through Target Distribution Design
Jun 9, 2026Tong Xie, Yuanhao Ban, Yunqi Hong +3Supervised Fine-Tuning Fine-Tuning
University of California, Los Angeles (UCLA) · Arena
Instruction Finetuning DeepSeek-R1-8B Model Using LoRA and NEFTune
Jun 9, 2026Wu Yuerong, Mingni LuoNamed Entity Recognition Fine-Tuning
University of Hong Kong, Hong Kong, China · Northeastern University, Seattle, United States
The Order Matters: Sequential Fine-Tuning of LLaMA for Coherent Automated Essay Scoring
Jun 9, 2026Ali Keramati, Mark WarschauerFine-Tuning LLM Fine-Tuning
University of California, Irvine
Fisher-Guided Progressive Parameter Selection for Adaptive Fine-Tuning
Jun 8, 2026Ghodsiyeh Rostami, Po-Han Chen, Mahdi S. HosseiniFine-Tuning Vision Foundation Model Adaptation
Concordia University · Concordia University Mila - Québec AI Institute
PriFT: Prior-Support Guided Supervised Fine-Tuning
Jun 8, 2026Ke Wang, Shuangqi Li, Mathieu Salzmann +1Supervised Fine-Tuning Fine-Tuning
EPFL, Lausanne, Switzerland
Lost in the Non-convex Loss Landscape: How to Fine-tune the Large Time Series Model?
Jun 7, 2026Xu Zhang, Peang Wang, Wei WangFine-Tuning Time-Series Foundation Models
Shanghai Key Laboratory of Data Science College of Computer Science and Artificial Intelligence Fudan University, Shanghai 200082, China
Spatial-Temporal Decoupled Adapter for Micro-gesture Online Recognition
Jun 5, 2026Xucheng Shen, Kun Li, Fei Wang +3Fine-Tuning Human Activity Recognition
Hefei University of Technology, Hefei, China · United Arab Emirates University, Al Ain, United Arab Emirates · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China +1
The Fine-Tuning Trap: Evaluating Negative Transfer and the Role of PEFT in Sub-1B Mathematical Reasoning
Jun 5, 2026Rahul Nair, Chun TaoFine-Tuning Small Language Models
Single-Cell Cross-Modal Transfer by Adversarial Fine-Tuning of Foundation Models
Jun 4, 2026Joseph Boyd, Matthew Lyon, Martino Mansoldo +2Spatial Transcriptomics Fine-Tuning
GSK.ai, GlaxoSmithKline, London, United Kingdom.

Jul 4, 2026, cs.LGJ/K move · Enter open · S save

Punching Above Their Weight: Classification-Head Fine-Tuning of Tiny Language Models (TLMs) for Verifiable Multiple-Choice Tasks

Bhavesh Sood, Jaromir Savelka

Carnegie Mellon University

Fine-Tuning Multiple-Choice Question Answering Small Language Models Low-Rank Adaptation Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

We define Tiny Language Models (TLMs) as models below roughly 3B parameters that fit on mainstream consumer devices. We study how to adapt them for and use them on verifiable multiple-choice tasks. We compare three LoRA-based fine-tuning paradigms (label generation, gold only, and our discriminative classification head) on a unified setup across several Qwen3 models from 0.6B to 8B and five benchmarks: HellaSwag, WinoGrande, PIQA, SciQ and ARC-C. Classification-head fine-tuning reliably outperforms label generation (+2-3%) at the 0.6B and 1.7B scales. Further, TLMs fine-tuned using the discriminative method are competitive to zero-/few-shot GPT-3 (175B), PaLM (540B) and GPT-4. The performance we report for Qwen3-0.6B and Qwen3-1.7B are SOTA on HellaSwag, WinoGrande, and PIQA.