Training-Inference Mismatch

Follow this topic Ask about this topic

Recent momentum

-56%

4 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

3 new papers

A weekly snapshot of new work published in Training-Inference Mismatch.

Period ending 2026-09-14

1 new paper

A weekly snapshot of new work published in Training-Inference Mismatch.

Period ending 2026-09-07

5 new papers

A weekly snapshot of new work published in Training-Inference Mismatch.

59 papers

Latest in Training-Inference Mismatch

Open your feed →

Informed Masking: Structure-Aware Perturbation for Reinforcement Learning in Diffusion Large Language Models
Sep 22, 2026Xiaoyi Yu, Enver Sangineto, Pei Fu +6Training-Inference Mismatch
Corrective Forcing: Unified Post-Training for Diffusions and Flows in Generative Speech Enhancement
Sep 21, 2026Qing Yao, Lijian Gao, Qirong MaoTraining-Inference Mismatch
Score Centering Stabilizes Off-policy Reinforcement Learning
Sep 17, 2026Martin Marek, Max RyabininTraining-Inference Mismatch Implicit Bias
When EOS Tokens Disagree: Understanding Length Inflation in On-Policy Distillation
Sep 17, 2026Yuxiao Yang, Tianrun Yu, Shangzhe Li +6On-Policy Distillation Training-Inference Mismatch
Doc2FRC: Length-Consistent Document-Level Machine Translation via Fixed-Range Chunking
Sep 14, 2026Xiaotian Wang, Youyuan Lin, Zhan Shen +1Chunk Training-Inference Mismatch
Content-Based Addressing for Long Context
Sep 7, 2026Mahesh GodavartiRotary Position Embedding Efficient Long-Context Inference
Stream Forcing: Constructing Unified Training Trajectory for Robust Streaming Video Generation
Aug 11, 2026Yueting Zhu, Yuehao Song, Kaicheng Zhang +5Few-Step Generation Training-Inference Mismatch
Mismatch Matters: On-Policy Distillation Beyond Token Agreement
Aug 10, 2026Zichao Yu, Chengzhi Yu, Shengze Xu +4Teacher-Student Log-Probability Gap On-Policy Distillation
WDL-OPD: Weak-Driven On-Policy Distillation via Mixture-Constrained Co-Training
Aug 10, 2026Zehao Chen, Gongxun Li, Tianxiang Ai +9On-Policy Distillation Rp-Opsd
DreOPD: Degraded-Reference Extrapolative On-Policy Distillation for Flow-matching Models
Aug 10, 2026Mingfeng Lin, Chengfei Cai, Lin Xu +2Hierarchical Flow Matching On-Policy Distillation
LC-GRPO: Bridging Train-Inference Gap for Flow-Based GRPO with Langevin Correction
Aug 6, 2026Yingqing Guo, Hui Yuan, Zijian He +2Flow-Based Policy Reward Gradients
CAPE-T2V: Captioner-Anchored Prompt Enhancement toward Two-Sided Conditioning Alignment in Text-to-Video Generation
Aug 4, 2026Yizhuo Jia, Jingyun Hua, Yuanxing ZhangText-To-Video Models Video Captioning
Round-Trip Consistency: Bidirectional Diffusion Models Can Predict Their Own Rollout Errors
Aug 1, 2026Alexander ScheinkerDiscrete Diffusion Model Long Rollouts
Faster but Different: Diagnosing and Controlling Content Drift in Accelerated Multimodal Diffusion Language Models
Jul 31, 2026Yaoxuan Dou, Yang ShuDiffusion Language Models Training-Inference Mismatch
ACRL: Adaptive Control of Training-Inference Discrepancy for Stable Reinforcement Learning
Jul 27, 2026Wenwu Fan, Qihong Lin, Zhijie Xia +4Training-Inference Mismatch Reinforcement Learning Post-Training
Deconstructing Off-Policy Ratios: Entropy-Scaled Trust Regions for Asynchronous Reinforcement Learning
Jul 24, 2026Guanqun Zhao, Zijun Xie, Binbin Zheng +5Off-Policy Evaluation Training-Inference Mismatch
CriPO: Enhancing Rubric-based RL via Self-Distillation
Jul 20, 2026Mingxuan Xia, Yuhang Yang, Chao Ye +7Rubric-Based Reinforcement Learning Unsupervised On-Policy Self-Distillation
Spatial Transport of Integration Error in Generative ODEs
Jul 17, 2026Songheng YinTraining-Inference Mismatch Classical Numerical Solver
QUADS: Stabilizing NVFP4 Reinforcement Learning for MoE via QUantization-error Alignment across Dual Sides
Jul 17, 2026Zhengyang Zhuge, Hao Yu, Xin Wang +4Rollout Training-Inference Mismatch
Optimizing Visual Generative Models via Distribution-wise Rewards
Jul 2, 2026Ruihang Li, Mengde Xu, Shuyang Gu +4Generative Models External Reward Models
Spec-AUF: Accept-Until-Fail Training under Train-Inference Misalignment for Masked Block Drafters
Jul 2, 2026Tianjian Yang, Meng LiDrafter Speculative Decoding
Multimodal Continuous Reasoning via Asymmetric Mutual Variational Learning
Jul 1, 2026Shijie Li, Yilin Gao, Siyuan Yang +7Multi-Modal Large Language Models Efficient Latent Reasoning
SLIM-RL: Risk-Budgeted Random-Masking RL for Diffusion LLMs Without Trajectory Slicing
Jun 30, 2026Ruikang Zhao, Zhenting Wang, Han Gao +1Diffusion Language Models Training-Inference Mismatch
RESOLVE: A Multi-Resolution and Multi-Modal Dataset for Roadside Cooperative Perception
Jun 30, 2026Shaozu Ding, Linan Song, Marco De Vincenzi +1Lidar Light Detection And Ranging
The Mirage of Optimizing Training Policies: Monotonic Inference Policies as the Real Objective for LLM Reinforcement Learning
Jun 28, 2026Jing Liang, Hongyao Tang, Yi Ma +9Reinforcement Learning Post-Training Training-Inference Mismatch
Adaptive Block Diffusion: Resolving Training-Inference Mismatch in Diffusion Language Models
Jun 28, 2026Gagan JainDiffusion Language Models Training-Inference Mismatch
Rethinking Training & Inference for Forecasting: Linking Winner-Take-All back to GMMs
Jun 24, 2026Qiyuan Wu, Katie Z Luo, Bharath Hariharan +2Trajectory Prediction Llm-Based Forecasting Methods
Not Every Sync Is Safe: Calibrated DiLoCo Scheduling for Shared AI Infrastructure
Jun 24, 2026Maxwell Twelftree, David Lemphers, An-chi He +1Workload Scheduling
Curvature-Adaptive Consistency Flow Matching: Autonomous Trajectory Optimization via Reinforcement Learning
Jun 21, 2026Songtao Tian, Guhan Chen, Bohan Li +2Training-Inference Mismatch Trajectory Optimization
What Shapes Emergent Misalignment? Insights from Training Dynamics, Model Priors, and Data
Jun 18, 2026Yuchen Zhang, Anietta Weckauff, Diego Garcia-Olano +1Misalignment Training-Inference Mismatch
The Reward Was in Your Data All Along: Correcting Flow Matching with Discriminator-Guided RL
Jun 17, 2026Nicolas Beltran-Velez, Felix Friedrich, Zhang Xiaofeng +4Training-Free Reward Alignment Hierarchical Flow Matching
Self-Generated Error Training for Token Editing in Diffusion Language Models
Jun 15, 2026Lin YaoDiffusion Language Models Multimodal Knowledge Editing
Semantic DLM+: Improving Diffusion Language Models through Bias-variance Trade-off in Transition Kernel Design
Jun 13, 2026Keyue Jiang, Yuxiang Wang, Yanan Zhao +7Diffusion Language Models Training-Inference Mismatch
Re-evaluating Confidence Remasking in Masked Diffusion Language Models
Jun 10, 2026Stipe Frkovic, Metod Jazbec, Dan Zhang +3Masked Diffusion Language Models Diffusion Language Models
Reformulate LLM Reinforcement Learning for Efficient Training under Black-box Discrepancy
Jun 7, 2026Jiashun Liu, Runze Liu, Xu Wan +3Training-Inference Mismatch Reinforcement Learning Post-Training
Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation
Jun 4, 2026Xingyu Su, Jacob Helwig, Shubham Parashar +6Diffusion Language Models On-Policy Distillation
Rethinking Incompleteness: Formalizing Protocol Divergence and Train-Once Learning for Robust IMVC
Jun 3, 2026Haolu Liu, Xiyue Wang, Xuanting Xie +2Missing Data Training-Inference Mismatch
Consistency Training Can Entrench Misalignment
Jun 2, 2026David Demitri Africa, Arathi ManiTraining-Inference Mismatch Sycophancy
RADE: Random Add-Drop Edge as a Regularizer
May 30, 2026Danial Saber, Amirali Salehi-AbariRegularization Techniques Mini-Batch Stochastic Gradient Descent
The Confidence Shortcut: A Reasoning Failure Mode of Masked Diffusion Models
May 27, 2026Dueun Kim, Albert NoMasked Diffusion Language Models Training-Inference Mismatch
Reinforcement Learning from Denoising Feedback
May 25, 2026Qi He, Huan Chen, Ya Guo +3Diffusion Language Models Reinforcement Learning From Human Feedback
D $^3$ -Subsidy: Online and Sequential Driver Subsidy Decision-Making for Large-Scale Ride-Hailing MarketD^3-Subsidy: Online and Sequential Driver Subsidy Decision-Making for Large-Scale Ride-Hailing Market
May 19, 2026Taijie Chen, Rui Su, Siyuan Feng +6Trajectory Optimization Incentives
Diagnosing Training Inference Mismatch in LLM Reinforcement Learning
May 14, 2026Tianle Zhong, Neiwen Ling, Yifan Pi +5Training-Inference Mismatch Large Language Model Performance
Constraint-Aware Flow Matching: Decision Aligned End-to-End Training for Constrained Sampling
May 12, 2026Jacob K. Christopher, James E. Warner, Ferdinando FiorettoGenerative Models Hierarchical Flow Matching
Missing Old Logits in Asynchronous Agentic RL: Semantic Mismatch and Repair Methods for Off-Policy Correction
May 12, 2026Zhong Guan, Yongjian Guo, Haoran Sun +5On-Policy Training-Inference Mismatch
Self-Distilled Trajectory-Aware Boltzmann Modeling: Bridging the Training-Inference Discrepancy in Diffusion Language Models
May 12, 2026Kecheng Chen, Ziru Liu, Xijia Tao +9Diffusion Language Models Autoregressive Language Models
Block-R1: Rethinking the Role of Block Size in Multi-domain Reinforcement Learning for Diffusion Large Language Models
May 12, 2026Yan Jiang, Ruihong Qiu, Zi HuangDiffusion Language Models Training-Inference Mismatch
The Convergence Gap: Instruction-Tuned Language Models Stabilize Later in the Forward Pass
May 8, 2026Yifan ZhouTraining-Inference Mismatch Instruction-Tuned Models
Generalized Distributional Alignment Games for Unbiased Answer-Level Fine-Tuning
May 4, 2026Mehryar Mohri, Jon Schneider, Yutao ZhongKullback-Leibler Divergence Training-Inference Mismatch
Teacher Forcing as Generalized Bayes: Optimization Geometry Mismatch in Switching Surrogates for Chaotic Dynamics
Apr 28, 2026Andre Herz, Daniel Durstewitz, Georgia KoppeLearning Dynamics Likelihood
Conditional misalignment: common interventions can hide emergent misalignment behind contextual triggers
Apr 28, 2026Jan Dubiński, Jan Betley, Anna Sztyber-Betley +2Misalignment Large Language Model Alignment
Evaluating Risks in Weak-to-Strong Alignment: A Bias-Variance Perspective
Apr 28, 2026Hamid Osooli, Kareema Batool, Rick Gentry +3Weak Supervision Training-Inference Mismatch
Distance-Misaligned Training in Graph Transformers and Adaptive Graph-Aware Control
Apr 24, 2026Qinhan Hou, Jing TangTraining-Inference Mismatch
Discrete Tilt Matching
Apr 20, 2026Yuyuan Chen, Shiyi Wang, Peter Potaptchik +2Masked Diffusion Language Models Training-Inference Mismatch
Attraction, Repulsion, and Friction: Introducing DMF, a Friction-Augmented Drifting Model
Apr 20, 2026Arkadii Kazanskii, Tatiana Petrova, Konstantin Bagrianskii +2Friction Training-Inference Mismatch
Training-inference input alignment outweighs framework choice in longitudinal retinal image prediction
Apr 18, 2026Liyin Chen, Nazlee Zebardast, Mengyu Wang +2U-Net Training-Inference Mismatch
Reasoning on the Manifold: Bidirectional Consistency for Self-Verification in Diffusion Language Models
Apr 17, 2026Jiaoyang Ruan, Xin Gao, Yinda Chen +5Diffusion Language Models Large Language Model Reasoning
Stabilizing Physics-Informed Consistency Models via Structure-Preserving Training
Feb 10, 2026Che-Chia Chang, Chen-Yang Dai, Te-Sheng Lin +2Physics-Informed Machine Learning Partial Differential Equation Benchmarks
A Random Matrix Theory Perspective on the Consistency of Diffusion Models
Feb 2, 2026Binxu Wang, Jacob Zavatone-Veth, Cengiz PehlevanDiscrete Diffusion Model Training-Inference Mismatch

Sep 22, 2026 · cs.CLJ/K move · Enter open · S save

Informed Masking: Structure-Aware Perturbation for Reinforcement Learning in Diffusion Large Language Models

Xiaoyi Yu, Enver Sangineto, Pei Fu, Fiorenzo Parascandolo+5

Diffusion Large Language Models (dLLMs) have emerged as an efficient alternative to autoregressive models, yet aligning them via Reinforcement Learning (RL) requires likelihood surrogates estimated from masked reconstruction subproblems under a small Monte Carlo budget per rollout. Existing methods construct these subproblems by uniform random masking, leaving open the question of which subproblems to prioritize. We identify a systematic upstream/downstream structure in dLLM rollouts. Some tokens, when revealed, trigger large confidence changes in nearby undecoded positions; we call them upstream. Others induce only small local changes and are therefore downstream. We find masking downstream tokens yields substantially better-posed subproblems than masking upstream tokens, a phenomenon we term subproblem difficulty asymmetry. Based on the observation, we propose Informed Masking (IM), which derives a per-token priority score from the denoising trajectory at zero extra inference cost and biases mask sampling toward downstream tokens. IM is plug-and-play: when plugged into three state-of-the-art dLLM RL methods on LLaDA-8B-Instruct, it delivers up to 2.01%, 8.68%, and 5.77% relative average gains on math and planning benchmarks with improved training stability.

Training-Inference Mismatch

Open paper Ask about it PDF