Large Language Model Post-Training

Follow this topic Ask about this topic

Recent momentum

-43%

12 papers in the last 28 days · 0.3% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-14

3 new papers

A weekly snapshot of new work published in Large Language Model Post-Training.

Period ending 2026-09-07

8 new papers

A weekly snapshot of new work published in Large Language Model Post-Training.

156 papers

Latest in Large Language Model Post-Training

Open your feed →

Style-Debiased DPO: Updating LLM Knowledge with Factuality-Aware Synthetic Preference Data
Sep 15, 2026Takayuki Yamamoto, Daisuke KawaharaLarge Language Model Post-Training Topic-Disjoint Direct Preference Optimization Data
LOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation
Sep 12, 2026Dongfang ZhaoLarge Language Model Post-Training Length
Why Does Post-Training Quantization Work?
Sep 10, 2026Yuxiang Chen, Michael Beyer, Jun Zhu +1Post-Training Quantization Quantization Effects
Osprey: Target-agnostic Pre-training Makes Stronger Drafters in Speculative Decoding
Sep 8, 2026Fengxiang Bie, Yuqing Jian, Yifan Yu +7Speculative Decoding Drafter
Knowledge Acquisition During Pre-training? Large Language Models Learn Better With Auxiliary Views
Sep 3, 2026Joseph Lee, Yidi Huang, Dokyoon Kim +2Large Language Model Post-Training Information Gap
Probing Factual Knowledge Transfer with Training Data Interventions
Sep 1, 2026Romina Oji, Marc Braun, Marcel Bollmann +2Multilingual Language Models Large Language Model Post-Training
Subword Segmental BabyLMs: Learning to Tokenise for Sample-Efficient Pretraining
Sep 1, 2026Francois MeyerSubword Tokenization Large Language Model Post-Training
Context-Grounding Gains Are Mediated by Pre-existing Machinery: Auditing GRPO, SFT, and DPO
Sep 1, 2026Prakhar Gupta, Vaibhav GuptaContextual Grounding Large Language Model Post-Training
CARE: Contrastive Anchor-based Rubric Evolution for Large Language Model Post-Training
Sep 1, 2026Siyuan Li, Xinxin Song, Chen Ruinian +5Rubric-Based Reinforcement Learning Rubrics
LLM Post-Training as Brownfield Maintenance: An Industrial Perspective on Dataware Engineering
Aug 31, 2026Gopi Krishnan Rajbahadur, Amir M. Ebrahimi, Boyuan Chen +1Post-Training AI Code Generation
KItCAT: Knowledge Injection via Input Corruption for Auto-regressive Training
Aug 31, 2026Meghanadh Pulivarthi, Kushagra Bhushan, Vineet Kumar +5Large Language Model Post-Training Data Augmentation
PAC: Progress-Augmented Advantage Curriculum for Multi-Task Reinforcement Learning of LLMs
Aug 31, 2026Yuanqiang Yu, Yanzhao Zheng, Zhentao Zhang +8Reinforcement Learning Post-Training Multi-Task Reinforcement Learning
Puro-2B: Poor Lab's Qwen2-1.5B Trained on RTX 5090 within $5090
Aug 27, 2026Kairong Luo, Jiarui Cui, Yaorui Yin +8Large Language Model Post-Training Qwen3
Measuring Task-Agnostic Training Data Influence Across Language Model Pretraining
Aug 13, 2026Yuto Nishida, Hirokazu Kiyomaru, Yusuke Oda +6Large Language Model Post-Training Training Data
ViTOED: A Dataset for Target-Oriented Emotion Detection on Vietnamese Social Media Texts
Aug 13, 2026Chanh Vo, Son T. Luu, Ngan Luu-Thuy NguyenEmotion Recognition Natural Language Datasets
LoKiFormer: Locality-aware Attention with Decoupled Knowledge Memory for Efficient Large Language Model Pretraining
Aug 12, 2026Qiuwu Chen, Zimo Liu, Yuchen Li +8Large Language Model Post-Training LLM Inference Efficiency
Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL
Aug 12, 2026Minglai Yang, Xinyu Guo, Utkarsh Tyagi +6Rubric-Based Reinforcement Learning Rubrics
SCOUT: Symmetric Consensus Outlier Detection for Failure Localization in LLM Pre-Training
Aug 11, 2026Zhuang WangIntermediate Checkpoints Outliers
Reducing Pretraining-Generation Mismatch in Diffusion Language Models
Aug 10, 2026Xiaocheng Lu, Huabin Liu, Song Guo +1Diffusion Language Models Pocket-Conditioned Diffusion
Instability of LLM Pre-Pretraining: It Doesn't Always Help. An Investigation on Multiple Languages
Aug 9, 2026Sofiia Riazhskykh, Nam Luu, Ondřej BojarLarge Language Model Post-Training Model Pretraining
On-Policy Delta Distillation for Multilingual Math Reasoning
Aug 6, 2026Byeongho Heo, Jaehui Hwang, Sangdoo Yun +1LLM Reasoning Methods On-Policy Distillation
Logic Before Language: Pre-pretraining on Formal Derivations Fosters Skill Acquisition and Compressibility
Aug 4, 2026Jo-Ku Cheng, Nikolaos Aletras, Marco ValentinoLarge Language Model Post-Training Model Pretraining
Beyond Initialization Loss: A Systematic Study of Token Embedding Initialization Strategies for LLM Vocabulary Extension
Aug 4, 2026Raviraj Joshi, Utkarsh Vaidya, Sanjay Singh Chauhan +1Large Language Model Post-Training Systematic Study
TimeRLM: Recursive Language Models Enable Precise Anomaly Localization in Long-Context Time-Series
Aug 4, 2026Nicolas Zumarraga, Lorenzo Steno, Ning Wang +9Time-Series Anomaly Detection Time Series
Emulate or Estimate? The Divergent Strengths of Base and Post-Trained Language Models for Opinion Simulation
Aug 4, 2026Seth Grief-Albert, Jessica Bo, Difan Jiao +1Social Simulations Large Language Model Post-Training
Toward Plasticity-Preserving KL Regularization for Capability Retention in LLM Reinforcement Learning
Aug 3, 2026Li Wang, Xiaodong Lu, Xiaohan Wang +4Reinforcement Learning Post-Training Kullback-Leibler Regularization
Attend to Your Own Thoughts: Breaking the Barrier for Post-Training Quantization of Reasoning LLMs through the Lens of 1.58-Bit Quantization
Aug 2, 2026Shigeng Wang, Chao Li, Yangyuxuan Kang +2Large Language Model Quantization Post-Training Quantization
Retrofitting Recurrent Depth into a Pretrained Language Model: Installation, Extrapolation, Transfer, and Retention at Two Parameter Budgets
Jul 31, 2026Mark ShapiroLarge Language Model Post-Training Recurrent Model
CDAE: Enhancing Perturbation Robustness in Pretrained Language Models with Contrastive Denoising
Jul 30, 2026Sina Heydari, Amirreza Abbasi, Mohsen Hooshmand +1Large Language Model Post-Training Representation Learning
PreDiff-LM: Pretrained Discrete Masked Diffusion Language Modeling with Hybrid Attention
Jul 28, 2026Zhengtao Yao, Runhao Li, Xupeng Chen +12Diffusion Language Models Causal Attention
FlowCTS: On-policy Continuous Trajectory Supervision of Flow Models
Jul 27, 2026Kaiyang Ye, Yuan Ge, Junxiang Zhang +8On-Policy Distillation Rp-Opsd
Cross-Tokenizer On-Policy Distillation via Byte-Prefix Marginalization
Jul 24, 2026Hao Wang, Kun Yuan, Wenlin Zhong +4On-Policy Distillation Tokenizer
Adaptive Depth Sparse Framework: Similarity-Driven Resource Allocation for Pre-Trained LLMs
Jul 23, 2026Yidu Wu, Xiang Wang, Kejie Zhao +3LLM Inference Efficiency Dynamic Sparse Attention
OR Else: A Differentiable Trust Region for Policy Optimization
Jul 20, 2026Chinmay Rane, Kanishka Tyagi, Michael ManryPolicy Optimization Trust Region
Distilled Reinforcement Learning for LLM Post-training
Jul 19, 2026Chen Wang, Zhaochun Li, Jionghao Bai +4Reinforcement Learning Post-Training Large Language Model Post-Training
Understanding Reasoning from Pretraining to Post-Training
Jul 17, 2026Jingyan Shen, Ang Li, Salman Rahman +4Large Language Model Post-Training LLM Reasoning Methods
Demystifying On-Policy Distillation: Roles, Pathologies, and Regulations
Jul 15, 2026Rui Wang, Hongru Wang, Yi Chen +4On-Policy Distillation Teacher-Student Model Pairs
Same Loss, Same Noise, Opposite Schedules: Noise Structure and Optimizer Normalization Jointly Determine Whether Learning-Rate Cooldown Helps
Jul 14, 2026Subham Singh, Ashutosh Mishra, Subha RautLearning Rates Batch Normalization
Proxy Exploration and Reusable Guidance: A Modular LLM Post-Training Paradigm via Proxy-Guided Update Signals
Jul 13, 2026Daocheng Fu, Rong Wu, Yu Yang +7Reinforcement Learning Post-Training Large Language Model Post-Training
Domain-Aware Scaling Laws Uncover Data Synergy
Jul 13, 2026Kimia Hamidieh, Lester Mackey, David Alvarez-MelisSynergy Model Pretraining
Complexity-Guided Component-wise Initialization for Language Model Pretraining
Jul 10, 2026Konstantin Garbers, Nicholas OhModel Weights Model Pretraining
TheBioCollection: Unified Pre-Training Scale LLM Corpus for Biology
Jul 9, 2026Hyunjin Seo, Hyeon Hwang, Gyubok Lee +7Life Sciences Research Biological
GIFT: Geometry-Informed Low-precision Gradient Communication for LLM Pretraining
Jul 8, 2026Jieying Wang, Shuyuan Fan, Mingkai Zheng +1Large Language Model Quantization Low-Precision Formats
Full-Stack FP4: Stable LLM Pretraining with Quantized Projections, Optimizers, and Attention
Jul 5, 2026Siyu Ding, Mingchuan Ma, Jiabo Tong +3Fp8 Large Language Model Post-Training
RL Forgets! Towards Continual Policy Optimization
Jul 5, 2026Mao-Lin Luo, Zhe-Xu Wang, Zi-Hao Zhou +4Catastrophic Forgetting Safe Reinforcement Learning
Enhancing Fitness Intelligence through Domain-Specific LLM Post-Training
Jul 2, 2026Xingtao Zhao, Tian Yang, Han JiangPhysical Activities Sports Science
Rethinking Speech-LLM Integration for ASR: Effective Joint Speech-Text Training by Interleaving
Jul 2, 2026Ruchao Fan, Yiming Wang, Rui Zhao +10Speech Language Models Automatic Speech Recognition
MultiSynt/MT: Trillion-Token Multi-Parallel Pre-Training Data Translated Across 36 Languages
Jul 1, 2026Maximilian Idahl, Jörg Tiedemann, Sampo Pyysalo +19Multilingual Language Models Corpus
When Does Online Imitation Learning Help in LLM Post-Training? The Role of (Non-)Realizability Beyond Horizon
Jun 29, 2026Huaqing Zhang, Jingchu Gai, Juno Kim +2Imitation Learning On-Policy Distillation
MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training
Jun 29, 2026Wenhan Ma, Jianyu Wei, Liang Zhao +10On-Policy Distillation Reinforcement Learning Post-Training
GLIP: Graph and LLM Joint Pretraining for Graph-Level Tasks
Jun 29, 2026Haoxin Sun, Yiqing Lin, Yajun Huang +3Graph Neural Networks Graph-Llm Joint
Data and Evaluation Closed-Loop for Model Capability Enhancement
Jun 26, 2026Zhixuan Li, Jiangan Yuan, Han XuLarge Language Model Evaluation Large Language Model Post-Training
Does Translation-Enhanced Speech Encoder Pre-training Affect Speech LLMs?
Jun 24, 2026Tomoya Mizumoto, Yusuke FujitaSpeech Encoder Speech Translation
Interleaved Speech Language Models Latently Work In Text
Jun 21, 2026Talia Sternberg, Gallil Maimon, Yossi AdiSpeech Language Models Large Language Model Post-Training
TALAS: Teacher-Anchored Layer Alignment with Adaptive Sharpness-Aware Minimization for Embedding Distillation
Jun 20, 2026Quoc Phong Dao, Hoang Son Nguyen, Pham Khanh Chi +4Fast And Efficient Global-Local Object Alignment Learning Large Language Model Post-Training
Rethinking Shrinkage Bias in LLM FP4 Pretraining: Geometric Origin, Systemic Impact, and UFP4 Recipe
Jun 18, 2026Qian Zhao, Kunlong Chen, Changxin Tian +9Fp8 Low-Precision Training
IHUBERT: Vector-Based Semantic Deduplication and Domain-Balanced Pretraining for Persian Resources
Jun 18, 2026Arash Ghafouri, Mahdi Firouzmandi, Hossein Saberi +1Persian Large Language Model Post-Training
Clusters are All You Need: Pre-Training the Tsetlin Machine with Semantic Clusters from Language Models for Interpretability
Jun 18, 2026Jiechao Gao, Rohan Kumar Yadav, Yuangang Li +4Bert Models Model Interpretability Methods
Beyond Uniform Forgetting: A Study of Sequential Direct Preference Optimization Across Preference Settings
Jun 18, 2026Pranav Bhandari, Nicolas Fay, Amitava Datta +2Topic-Disjoint Direct Preference Optimization Data Pairwise Preferences
Which Pairs to Compare for LLM Post-Training?
Jun 17, 2026Jiangze Han, Vineet Goyal, Will MaLarge Language Model Post-Training Post-Training

Sep 15, 2026 · cs.CLJ/K move · Enter open · S save

Style-Debiased DPO: Updating LLM Knowledge with Factuality-Aware Synthetic Preference Data

Takayuki Yamamoto, Daisuke Kawahara

Continued pretraining (CPT) with data augmentation such as paraphrasing can store inside a large language model (LLM) the knowledge of a small source corpus. The stored knowledge, however, is not always retrieved correctly. We study the eliciting side rather than the storing side: we use preference optimization, which learns from pairs of a preferred (chosen) and a dispreferred (rejected) response, so that the model elicits its stored knowledge more accurately. One proposed approach takes the model's own erroneous response as rejected and the gold answer as chosen, so as to suppress the error. When the target knowledge is partially known, however, most of these rejected responses are factually correct. Using direct preference optimization (DPO) then pushes down rejected responses that contain correct knowledge and differ from the chosen answer only in style, such as length and wording. We propose style-debiased DPO (SD-DPO), which scores whether the rejected response of each pair is factually correct, inverts the preference of such pairs, and weights them so that the learning signal due to differences in style cancels out as a whole. We first test whether, on top of EntiGraph, a representative storing-side method that runs CPT on text synthesized from the corpus, our method adds accuracy efficiently. On QuALITY, the reading-comprehension QA benchmark on which EntiGraph was evaluated, SD-DPO exceeds a baseline we CPT on EntiGraph's synthetic data from the same base model and evaluate with the same procedure. The training tokens this requires are a few dozen times fewer than the additional CPT needed for the same gain. For knowledge updating, the main goal of this work, we use AToKE, a knowledge-editing benchmark for facts that change over time. There, SD-DPO reaches an overall accuracy of 0.982 and answers with the new or the old fact according to the queried period.

Large Language Model Post-Training Topic-Disjoint Direct Preference Optimization Data

Open paper Ask about it PDF