Post-Training

Follow this topic Ask about this topic

Momentum

73 papers in the last four weeks, up 306% on the four weeks before. 0.7% of all new papers.

Jul 13Week of Sep 28

Latest papers 297

All topics

Finetuning with Sampling: SFT Learns Better Than You Think
Oct 1, 2026Aayush Karan, Sitan Chen, Yilun DuSupervised Finetuning Post-Training
Harvard University
Learn the Directions, Normalize the Gains: Post-Training Normalization for LoRA
Oct 1, 2026Zailong Tian, Yanzhe Chen, Zhuoheng Han +2Balanced Learning Rapid Adaptation
Singapore Management University · National University of Singapore · Peking University
On Language Drift during RLVR Post-Training
Oct 1, 2026Michael Sullivan, Alexander KollerLarge Language Model Reinforcement Learning LLM Reasoning Strategies
Department of Language Science and Technology, Saarland University, Saarbrücken, Germany.
Sharpening Tax in Post-Training
Oct 1, 2026Changdae Oh, Qi Zeng, Qi Qi +7Reinforcement Learning Post-Training Post-Training
Meta Superintelligence Labs · University of Wisconsin–Madison · Work done at Meta +2
MMVistaReason: Toward Open-Data and Post-Training Recipes for Multimodal Reasoning
Oct 1, 2026Juekai Lin, Honglin Lin, Yuqian Yuan +7Multimodal Reasoning Post-Training
Zhejiang University · Shanghai Artificial Intelligence Laboratory, OpenDataLab · Shanghai Jiao Tong University +1
Match the Distribution, Not the Compute: Post-Training Multi-Token Prediction Heads
Oct 1, 2026Prachi Badarayani, Aidan Jay, Chenghui Zhou +8Multi-Token Prediction Post-Training
Microsoft
Video Generation Models: A Survey of Post-Training and Alignment
Sep 30, 2026Chaoyu Li, Xiaoyi Gu, Yogesh Kulkarni +10Generative Video Models Video Generation
Arizona State University · Twitch · Stanford University +6
PEG-Tab: Sampling-Time Record Repair and Release Control for Tabular Synthesis
Sep 30, 2026Pengfei Li, QinYi Liu, Mohammad KhalilSynthetic Tabular Data Release
From Search to Signal: Online Post-Training in Automatic Heuristic Design
Sep 30, 2026Yilun Yuan, Tianyu Zhou, Zhenzhou TangAutomated Heuristic Design Post-Training
Wenzhou University
Ready2Blend: From Natural-Language Instructions to Composable Alignment Prompts
Sep 30, 2026Jeesu Jung, Hwan Chang, Juseon Do +5Value Alignment Large Language Model Adaptation
Korea Advanced Institute of Science and Technology · Samsung SDS
Explicit Trajectory Diversity for RL-Based Post-Training of LLM Agents
Sep 30, 2026Huaiyu Fu, Heng Cao, Hao Wang +2Large Language Model Policy Optimization Large Language Model Agents
Microsoft
Aligned Data Can Induce Misalignment via Context Confusion
Sep 29, 2026Yavuz Bakman, Duygu Nur Yaldiz, Baris Askin +4Large Language Model Alignment Training-Inference Mismatch
University of Southern California · Carnegie Mellon University · Amazon +1
On the Off-Policy Teacher in On-Policy Distillation
Sep 29, 2026Langlin Huang, Hao Liu, Mononito Goswami +5Efficient On-Policy Distillation Teacher
Washington University in St. Louis · AWS AI Labs · Carnegie Mellon University +1
EVOKE: Eliciting World Knowledge in Agents for Transferable Decision-Making
Sep 29, 2026Yuhan Guo, Jinming Liu, Liang Xu +8World Post-Training
Shanghai Jiaotong University · Eastern Institute of Technology, Ningbo · Zhongguancun Academy, Beijing, China +1
Selective Channel Restoration for Backdoored Vision-Language Models
Sep 29, 2026Shuming Liu, Zhifang Zhang, Suqin Yuan +3Model-Agnostic Defense Post-Training
Southeast University · The University of Queensland · University of Sydney +1
Direct Experience World-Model Optimization: Learning the World Beyond Action Imitation
Sep 29, 2026Xiangcheng Zhan, Zirui Chen, Yicheng Zhao +2Efficient World-Action Model World Models
Harbin Institute of Technology · Dalian University of Technology · Southern University of Science and Technology
Trajectory Soup: Pushing the Compute-Scaling Frontier of LLM Mid-training via Diverse Trajectories
Sep 29, 2026Zhehao Huang, Changxin Tian, Qingyuan Yang +5Large Language Model Training Post-Training
Ling Team, Ant Group · Shanghai Jiao Tong University
What Does Post-Training Change in Multilingual Reasoning?
Sep 29, 2026Hongyang Li, Xiao Li, Caesar Wu +2Multilingual Language Models LLM Reasoning Strategies
University of Luxembourg · Seafill Open-Source Community
Beyond Compression: Diagnosing How Post-Training Changes Mathematical Reasoning
Sep 29, 2026Hongyang Li, Yiming Zhu, Xiao Li +3Mathematical Reasoning Post-Training
University of Luxembourg · Seafill Open-Source Community · Université Paris-Saclay
Salt++: Context-Aligned Post-Training for Few-Step Streaming Multimodal Generation
Sep 29, 2026Xingtong Ge, Yutong Wang, Lunjie Zhu +7Audio-Video Generation Few-Step Generation
The Hong Kong University of Science and Technology · Vivix Group Limited · The University of Sydney +1
Can Language Models Learn to Forecast Stock Prices
Sep 29, 2026Jiacheng Guo, Suozhi Huang, Shuzhen Li +11Post-Training
Princeton University · InclusionAI · Harvard University +1
WEFT: Scaling Tool-Use Post-Training for General-Purpose Agents
Sep 29, 2026Bo Mao, Hang He, Linting Wang +17Production Agentic Systems Self-Evolution
East China Normal University · Shanghai Innovation Institute · Fudan University +2
Calibrate the Decisions That Change the Future: On-Policy Post-Training Quantization for Multimodal Large Language Models
Sep 29, 2026Wenxiao Fan, Jingling Fu, Lichen Ma +6Post-Training Quantization Post-Training
School of Computer Science and Technology, Beijing Institute of Technology · Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University
Where Should Physics Enter a Molecular Crystal Generator?
Sep 28, 2026Haocheng Tang, Junmei Wang, Wengong JinCrystal Structure Crystal Structure Prediction
Khoury College of Computer Science Northeastern University Boston, MA 02115, USA · School of Pharmacy University of Pittsburgh Pittsburgh, PA 15213, USA
Early Learning Shapes Later Directions Of Representation Change In Continual Learning
Sep 28, 2026Yuantao Deng, Jinnuo Liu, Kaizhen Tan +1Neural Network Representations Continual Learning
New York University · Xi’an Jiaotong University
Frontier Learning: Training LLM Reasoners at the Edge of Capability
Sep 28, 2026Robin Faro, Shyam Sundhar Ramesh, Ilija Bogunovic +1LLM Reasoning Strategies Post-Training
University College London London, UK · University of Basel Basel, Switzerland
RefineDrive: Reliable Failure-Guided Learning for Vision-Language-Action Driving
Sep 28, 2026Zhe Sun, Ziyi Luo, Yehao Lu +2Autonomous Driving Drives
College of Computer Science and Technology, Zhejiang University, Hangzhou, China · Yinwang Intelligent Technology Co., Ltd.
QuantForge: Discovering Residual Decompositions for MXFP4 Post-Training Quantization
Sep 28, 2026Qiulin Shang, Zhoutong Wu, Jie Hu +1Post-Training Quantization Post-Training
Peking University
Nereus: Adaptive Parallelism for LLM Post-Training
Sep 28, 2026Songlin Jiang, Tuo Shi, Sitong Zhang +3Post-Training
Aalto University · Shenzhen University of Advanced Technology · Zhejiang University
Marathoner: Ultra-Long-Horizon Autonomous Intelligence
Sep 28, 2026Zhang Ruiyang, Ou Jinpeng, Xie Yifan +4Robot State Post-Training
FIC, University of Macau · Ant Group · Peking University
Improving Large Language Models for Code through Runtime Program-State Reasoning
Sep 28, 2026Hongwei Li, Spandan Garg, Yufan HuangSwe-Bench Verified Runtime
University of California, Santa Barbara · Microsoft
SegBanana: Steering Unified Multimodal Models into Medical Segmenters
Sep 28, 2026Xiaoye Liang, Ye Yan, Mingze Yin +5Semi-Supervised Medical Image Segmentation Multimodal Model
Beihang University · Zhongguancun Academy · Zhejiang University
How code helps different tasks? A decompositional lens on LLM post-training
Sep 27, 2026Zheng Yu, Yiwei Li, Yishen Chen +4Instruction-Tuned Models Post-Training
Sun Yat-sen University · Shenzhen Loop Area Institute · The Chinese University of Hong Kong, Shenzhen +1
Do We Really Need KL Divergence for On-Policy Distillation of Large Language Models?
Sep 27, 2026Wenze Lin, Jiyuan Long, Jiale Zhao +11Kullback-Leibler Divergence Residual Distillation
LeapLab, Tsinghua University · Qiuzhen College, Tsinghua University · Beihang University +3
Skill2Env: Capability-Oriented Environment Synthesis from Skills for General Agents
Sep 27, 2026Weiyi Xu, Xiaowen Yang, Wen Da +7Synthetic Environments Skills
AllSpark Team
A Cheap Verifier is Good Enough: LLM Post-training is Robust to Erroneous Rewards
Sep 27, 2026Andreas Plesner, Curtis Northcutt, Francisco Guzmán +1Verification Framework Large Language Model Judges
ETH Zurich · Handshake AI
Rufus-Air: An Open LLM Post-Training Recipe
Sep 24, 2026Chia-Yuan Chang, Renyuan Cheng, Rui Feng +19Post-Training Offline Reinforcement Learning
Amazon
Transcript-Supervised Post-Training of Generative Speech Enhancement on Real Recordings via Reinforce Adjoint Matching
Sep 24, 2026Julius Richter, Christoph Boeddeker, Yoshiki Masuyama +4Speech Enhancement Post-Training
Mitsubishi Electric Research Laboratories (MERL), USA · Mitsubishi Electric Corporation, Japan
Baseline Shape Decides the Verdict: A Controlled Re-Examination of Ternary Language Models at 60K Parameters
Sep 24, 2026Gautam VeldandaTransformer Architectures Model Weights
Independent Researcher
Post-Training Leaves Behavioral Shadows on Unrelated Decisions
Sep 24, 2026Ziyang Zhang, Yubin Jing, Yuanhao Zeng +3Post-Training Agent Behavior Modeling
Peking University · Georgia Institute of Technology · ShanghaiTech University +2
Small yet Assistive: Spatially-Aware Post-Training for Low Vision
Sep 23, 2026Rishabh Choudhary, Shreyansh Raj, Umesh Goyal +6Recent Vision-Language Models Assistive Agents
Indian Institute of Technology Mandi · Mohamed bin Zayed University of Artificial Intelligence
Temporal Taxation Compounds Under Post-Training Compression of Whisper Models
Sep 23, 2026Srishti Ginjala, Eric Fosler-Lussier, Christopher W. Myers +1Whisper-Large-V3 Character Error Rate
The Ohio State University, Columbus, OH, USA · Air Force Research Laboratory, USA
Thinking Leakage: A Causal Audit of NoThink Post-Training in Hybrid Reasoning Models
Sep 23, 2026Zehao Liu, Vasant G. HonavarLarge Reasoning Models Post-Training
College of Information Sciences and Technology Pennsylvania State University
Pistis Technical Report
Sep 23, 2026Heyun Chen, Xiaohan Lan, Jiaxi Li +17Multimodal Agents Multimodal Large Language Models
Pistis Team, ByteDance
The Limits of Simulated Societies: How Post-Training and Survey Fine-Tuning Erase Cross-Cultural Variance
Sep 22, 2026Rojin ZiaeiLarge Language Model Bias Computational Social Science
Georgetown University
Toolcompass: Guiding Tool Trialing, Not Suppressing It
Sep 22, 2026Junlin Fang, Chong Zhang, Do Nguyen-Thanh +3Political Compass Test Call
College of Computing and Data Science, Nanyang Technological University · Zhejiang University · Australian Artificial Intelligence Institute, University of Technology Sydney +1
Reasoning-Preserving Fine-Tuning of Post-RL LLMs with Null-Basis LoRA
Sep 22, 2026Wenzhi Fang, Nicholas Tzou, Lazar Valkov +1Model Fine-Tuning Post-Training
Samsung Research America, Mountain View, CA, USA · Purdue University, West Lafayette, IN, USA
TelecomGPT-R1: Unified Post-Training for Reasoning Across Heterogeneous Telecom Tasks
Sep 21, 2026Bohao Wang, Chenwei Wu, Hang Zou +7Large Reasoning Models Large Language Models(Llms
College of Information Science and Electronic Engineering, Zhejiang University, 310027 Hangzhou, China · Research Institute for Digital Future, Khalifa University, P O Box 127788, Abu Dhabi, UAE · Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI 48109-2122, USA +1
Trains but Doesn't Learn: A Post-Training Delivery Benchmark for LLM Agents as Forward-Deployed Engineers
Sep 21, 2026Weihang Ding, Junfei ZhanAgentic Deployments Large Language Model Agents
Department of Industrial Engineering and Operations Research University of California, Berkeley · Department of Computing Imperial College London, United Kingdom
When Quantization Preserves Accuracy but Not Evidence: Explanation-Aware Post-Training Quantization for Medical LLMs
Sep 21, 2026Yeji Kim, Mi-Young Kim, Randy GoebelClinical Reasoning Training Post-Training
University of Alberta
Corrective Forcing: Unified Post-Training for Diffusions and Flows in Generative Speech Enhancement
Sep 21, 2026Qing Yao, Lijian Gao, Qirong MaoSpeech Enhancement Training-Inference Mismatch
School of Computer Science and Communication Engineering, Jiangsu University Jiangsu Engineering Research Center of Big Data Ubiquitous Perception and Intelligent Agriculture Applications Provincial Key Laboratory of Computational Intelligence and New Technologies in Low-Altitude Digital Agriculture Zhenjiang, China
SupportCal: Label-Free Calibration of Post-Trained LLMs via Reference Support and Corroboration
Sep 21, 2026Linhan Luo, Lequan Lin, Dai Shi +3Post-Training Confidence Calibration
The University of Sydney · University of Cambridge · The University of Adelaide
Measuring the Assistant's Harmlessness Preferences on the User Turn
Sep 20, 2026Jord NguyenChatbots Preference Alignment Learning
OPTED: On-Policy Fine-Tuning for End-to-End Driving using a Render-Free Teacher
Sep 17, 2026Damiano Da Col, Maximilian Igl, Peter Karkus +5Autonomous Driving Reinforcement Learning Post-Training
KE:SAI · ETH Zürich · NVIDIA Research +2
HIL-UMI: Bringing Human-in-the-Loop Post-Training of Vision-Language-Action Models to Universal Manipulation Interface
Sep 17, 2026Zimu Han, Yiming Zeng, Jiyao Zhang +9Generalist Vision--Language--Action Human-In-The-Loop
Center on Frontier Computing Studies, School of Computer Science, Peking University, China · Xi’an Jiaotong University, China · National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University, China +2
Stress-testing Alignment Midtraining
Sep 17, 2026Sid Baines, Jonathan Bostock, Maria Angelica Martinez +3Post-Training Frontier Models
Arcadia Impact · Resolution
Towards High-DoF Dexterous Manipulation through VLA Post-Training
Sep 17, 2026Junlei Zhu, Shenzhe Yao, Chaogui Huang +6Contact-Rich Dexterous Manipulation Vision-Language-Action Framework
Wuji Technology · School of Information Science and Technology, ShanghaiTech University, China
Channel-Wise and Token-Aware Post-Training Quantization for Visual State Space Duality
Sep 15, 2026Jonghyeon Lim, Changhoon YimPost-Training Quantization Vision State Space Models
Intelligent Image Processing Laboratory · Department of Computer Science and Engineering, Konkuk University · Seoul, Republic of Korea
ReDraft, Don't Just Distill: Reference-Driven Revision for Continual VLLM Post-Training
Sep 15, 2026Zhihao Zhang, Mingqi Wu, Qiaole Dong +15Post-Training Process-Level Supervision
Fudan University · Shanghai Artificial Intelligence Laboratory · Tencent
Performance, Efficiency and Collapse -- Advantages and Challenges in Offline Post-training of Code LLMs
Sep 14, 2026Abhinav Anand, Sanjana Reddy Pachika, Shweta Verma +1Reinforcement Learning Post-Training Large Language Model Training
TU Darmstadt · Hessian Center for Artificial Intelligence, Darmstadt, Germany · National Research Center for Applied Cybersecurity ATHENE

Oct 1, 2026, cs.LGJ/K move · Enter open · S save

Finetuning with Sampling: SFT Learns Better Than You Think

Aayush Karan, Sitan Chen, Yilun Du

Harvard University

Supervised Finetuning Post-Training On-Policy Frontier Models In-Context

Open paper AskPDF arXiv

Introducing new capabilities to frontier models has long been the goal of posttraining, which predominantly employs supervised finetuning (SFT) and reinforcement learning (RL) to this end. Conventional wisdom dictates that RL enables strong generalization on new tasks without losing existing capabilities, while SFT is prone to weak generalization and catastrophic forgetting. At the same time, SFT can learn from off-policy expert data, whereas RL must rely on a model's ability to find successful trajectories with repeated sampling. In our work, we seek to leverage the strength of on-policy learning while utilizing the privileged information contained in off-policy data. However, rather than modifying the learning objective to accommodate this data, we instead tailor the data distribution to better suit the learner. We introduce a Markov chain Monte Carlo (MCMC) sampling algorithm that progressively transforms off-policy traces to be more on-policy given a reference model for finetuning. Across tasks like scientific skill acquisition, mathematical reasoning, and open-ended expertise, our sampling algorithm enables SFT to rival prevailing posttraining techniques, often generalizing better and forgetting less than strong on-policy baselines. In addition, the resulting finetuned models exhibit strong distributional performance and are capable of learning beyond sharpening the base model distribution. At a higher level, our approach presents sampling as a model-native operator that shapes data for learnability, offering broader utility as a general-purpose primitive throughout the posttraining stack.

Momentum

Related topics

Latest papers 297