Supervised Fine-Tuning

Also known as SFT

Follow this topic Ask about this topic

Latest papers 208

29 papers in the last four weeks, up 12% on the four weeks before. 0.2% of all new papers.

All topics

SFT-GRPO Data Overlap as a Post-Training Hyperparameter for Autoformalization
Apr 15, 2026Xiaole Su, Kasey Zhang, Andy LyuLanguage Model Post-Training LLM Post-Training
Osmosis AI
From Imitation to Discrimination: Progressive Curriculum Learning for Robust Web Navigation
Apr 14, 2026Chuang Peng, Wei Zhang, Renshuai Tao +2Web Agents LLM Agent Training
Beijing Jiaotong University · Beihang University
Polyglot Teachers: Evaluating Language Models for Multilingual Synthetic Data Generation
Apr 13, 2026Lester James V. Miranda, Ivan Vulić, Anna KorhonenSupervised Fine-Tuning Multilingual Language Models
Language Technology Lab, University of Cambridge
BITS Pilani at SemEval-2026 Task 9: Structured Supervised Fine-Tuning with DPO Refinement for Polarization Detection
Apr 13, 2026Atharva Gupta, Dhruv Kumar, Yash SinhaSocial Media Analysis Language Model Post-Training
Birla Institute of Technology and Science, Pilani, India
Using Deep Learning Models Pretrained by Self-Supervised Learning for Protein Localization
Apr 13, 2026Ben Isselmann, Dilara Göksu, Heinz Neumann +1Supervised Fine-Tuning Self-Supervised Pre-Training
Department of Mathematics and Natural Sciences, Hochschule Darmstadt, Schoeﬀerstraße 3, Darmstadt, 64295, Hessen, Germany. · Department of Chemical Engineering and Biotechnology, Hochschule Darmstadt, Stephanstrasse 7, Darmstadt, 64295, Hessen, Germany. · European University of Technology, European Union. +1
A Layer-wise Analysis of Supervised Fine-Tuning
Apr 12, 2026Qinghua Zhao, Xueling Gong, Xinyu Chen +2Supervised Fine-Tuning Fine-Tuning
Hefei University · Lanzhou University
Why Supervised Fine-Tuning Fails to Learn: A Systematic Study of Incomplete Learning in Large Language Models
Apr 11, 2026Chao Xue, Yao Wang, Mengqiao Liu +11LLM Fine-Tuning Supervised Fine-Tuning
University of New South Wales, Australia · Tencent Hunyuan, China · Tencent Yuanbao, China +2
On the Role of Reasoning Patterns in the Generalization Discrepancy of Long Chain-of-Thought Supervised Fine-Tuning
Apr 2, 2026Zhaoyi Li, Xiangyu Xi, Zhengyu Chen +6Supervised Fine-Tuning Efficient Language Model Reasoning
University of Science and Technology of China · Meituan LongCat Team · City University of Hong Kong +1
PRISM: Probability Reallocation with In-Span Masking for Knowledge-Sensitive Alignment
Apr 2, 2026Chenning Xu, Mao Zheng, Mingyang SongLLM Hallucination Mitigation Hallucination in Language Models
Large Language Model Department, Tencent, China
From Baselines to Preferences: A Comparative Study of LoRA/QLoRA and Preference Optimization for Mental Health Text Classification
Apr 1, 2026Mihael ArcanPreference Optimization Supervised Fine-Tuning
Home Lab, Galway, Ireland
How to Fine-Tune a Reasoning Model? A Teacher-Student Cooperation Framework to Synthesize Student-Consistent SFT Data
Mar 23, 2026Zixian Huang, Kaichen Yang, Xu Huang +6Synthetic Data Generation for Language Models Supervised Fine-Tuning
Shanghai AI Laboratory · Dalian University of Technology · Nanjing University
mSFT: Addressing Dataset Mixtures Overfitting Heterogeneously in Multi-task SFT
Mar 23, 2026Woosung Koh, Jeyoung Jeon, Youngjin Song +4LLM Fine-Tuning Supervised Fine-Tuning
KAIST AI · Yonsei University · Samsung Advanced Institute of Technology
An Empirical Study of SFT-DPO Interaction and Parameterization in Small Language Models
Mar 20, 2026Yuming Feng, Christy YangDirect Preference Optimization Supervised Fine-Tuning
Department of Computer Science, Stanford University
Instruction-Free Tuning of Large Vision Language Models for Medical Instruction Following
Mar 19, 2026Myeongkyun Kang, Soopil Kim, Xiaoxiao Li +1Medical VLMs Large Vision-Language Models
Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, BC V6T 1Z4, Canada · Department of Robotics and Mechatronics Engineering, Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu 42988, Republic of Korea · Division of Intelligent Robot, Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu 42988, Republic of Korea +2
Balanced Thinking: Improving Chain of Thought Training in Vision Language Models
Mar 19, 2026Shaked Perek, Ben Wiesel, Avihu Dekel +2VLM Reasoning Vision-Language Models
IBM Research
MOSAIC: Multi-Objective Slice-Aware Iterative Curation for Alignment
Mar 19, 2026Yipu Dou, Wang YangMulti-Objective Language Model Alignment LLM Safety Alignment
School of Cyber Science and Engineering, Southeast University, Nanjing 211189, China
Inducing Epistemological Humility in Large Language Models: A Targeted SFT Approach to Reducing Hallucination
Mar 18, 2026Cem Uluoglakci, Tugba Taskaya TemizelLLM Hallucination Mitigation Hallucination in Language Models
Information Systems Middle East Technical University Ankara, Turkey · Data Informatics Middle East Technical University Ankara, Turkey
Large Reasoning Models Struggle to Transfer Parametric Knowledge Across Scripts
Mar 17, 2026Lucas Bandarkar, Alan Ansell, Trevor CohnCross-Lingual Transfer Cross-Lingual Reasoning in Language Models
University of California, Los Angeles · Google Research · University of Melbourne
Offline Exploration-Aware Fine-Tuning for Long-Chain Mathematical Reasoning
Mar 17, 2026Yongyu Mu, Jiali Zeng, Fandong Meng +2LLM Mathematical Reasoning Supervised Fine-Tuning
NLP Lab, School of Computer Science and Engineering, Northeastern University, Shenyang, China · Pattern Recognition Center, WeChat AI, Tencent Inc, China
Supervised Fine-Tuning versus Reinforcement Learning: A Study of Post-Training Methods for Large Language Models
Mar 14, 2026Haitao Jiang, Wenbo Zhang, Jiarui Yao +3LLM Post-Training Supervised Fine-Tuning
North Carolina State University · UC Irvine · University of Illinois Urbana-Champaign +2
Robust Post-Training for Generative Recommenders: Why Exponential Reward-Weighted SFT Outperforms RLHF
Mar 10, 2026Keertana Chidambaram, Sanath Kumar Krishnamurthy, Qiuling Xu +2Recommender Systems Generative Recommendation
Department of Management Science & Engineering, Stanford University, CA, USA · Meta, CA, USA · Netflix Research, CA, USA
Adapting Time Series Foundation Models through Data Mixtures
Mar 3, 2026Thomas L. Lee, Edoardo M. Ponti, Amos StorkeyTime-Series Foundation Models Zero-Shot Time Series Forecasting
School of Informatics, University of Edinburgh.
When Does RL Help Medical VLMs? Disentangling Vision, SFT, and RL Gains
Mar 1, 2026Ahmadreza Jeddi, Kimia Shaban, Negin Baghbanzadeh +4Medical VLMs RL for VLMs
University of Toronto, Canada · Vector Institute, Canada · KITE Research Institute, University Health Network, Canada +1
GUI-Libra: Training Native GUI Agents to Reason and Act with Action-aware Supervision and Partially Verifiable RL
Feb 25, 2026Rui Yang, Qianhui Wu, Zhaoyang Wang +8GUI Agents RL for GUI Agents
UIUC · Microsoft · UNC-Chapel Hill
Patch the Distribution Mismatch: RL Rewriting Agent for Stable Off-Policy SFT
Feb 11, 2026Jiacheng Wang, Zhijie Liu, Ping Jian +4Supervised Fine-Tuning Reinforcement Learning
Beijing Institute of Technology, Beijing, China · Tsinghua University, Beijing, China
Entropy-Preserving Supervised Fine-Tuning via Adaptive Self-Distillation for Large Reasoning Models
Feb 2, 2026Hao Wang, Hao Gu, Hongming Piao +6Supervised Fine-Tuning Language Model Distillation
1City University of Hong Kong · 2The Hong Kong University of Science and Technology · 3The Chinese University of Hong Kong
Bilevel Data Curation for LLM Fine-tuning: Offline Selection and Online Self-Refining Generation
Nov 26, 2025Quan Xiao, Yutong Xuan, Gaowen Liu +2Supervised Fine-Tuning LLM Self-Refinement
Cornell University · Cisco Research
Learning Task Mixtures from Task Affinities: A Probabilistic Graphical Model for Supervised Fine-Tuning
Jul 16, 2025Prateek Chanda, Saral Sureka, Parth Pratim Chatterjee +3Supervised Fine-Tuning Multi-Task Learning
IIT Bombay · IBM Research · Red Hat AI Innovation +1

Apr 15, 2026J/K move · Enter open · S save

SFT-GRPO Data Overlap as a Post-Training Hyperparameter for Autoformalization

Xiaole Su, Kasey Zhang, Andy Lyu

Osmosis AI

Language Model Post-Training LLM Post-Training Autoformalization RL for Language Model Reasoning Supervised Fine-Tuning

Open paper AskPDF arXiv

Supervised fine-tuning (SFT) followed by Group Relative Policy Optimization (GRPO) is a common post-training recipe. We conduct a controlled ablation over SFT-GRPO data overlap, evaluating Qwen3-8B (thinking disabled) post-trained for Lean 4 autoformalization under six conditions that differ solely in training recipe: a base model, SFT-only, GRPO-only, and three SFT+GRPO configurations where 0 percent, 30 percent, or 100 percent of the GRPO prompts coincide with the SFT corpus. Keeping SFT and GRPO data disjoint consistently outperforms full overlap at zero additional compute cost. Evaluating on Gaokao-Formal and PutnamBench under both compile pass at k and semantic pass at k assessed by an LLM judge, we find that lower overlap is monotonically associated with higher compilation and semantic accuracy. At 0 percent overlap, GRPO yields a 10.4 percentage point semantic gain over SFT alone on Gaokao, while at 100 percent overlap both metrics remain flat, rendering the GRPO stage effectively redundant. We further show that dual-metric evaluation reveals compile semantic gaps exceeding 30 percentage points for the highest compiling models, a disparity invisible under compile-only benchmarking. To our knowledge, this is the first controlled investigation of SFT-GRPO data overlap as a post-training hyperparameter, demonstrating how model behavior varies based on the degree of data sharing between training stages.