Mode Collapse

Follow this topic Ask about this topic

Mode collapse occurs when a model concentrates on a few common outputs or solutions while neglecting other valid modes of a distribution, affecting image and video generation, language-model reasoning, robot actions, and Bayesian inference. Approaches include conditioning on modes or intents, distribution matching and replay, regularizing latent-to-output mappings, and changing training noise or sampling to preserve coverage. Researchers assess collapse with measures such as mode coverage, recall, pass@k, and diversity, since single-sample quality or accuracy can hide missing alternatives. The central tension is how to increase diversity without sacrificing quality, and how to tell genuine coverage from merely varied-looking outputs.

Latest papers

34 papers6 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

Rènyi-Regularized Flow-Based Variational Inference for Bayesian Calibration
Oct 27, 2026Exaucè Luweh Adjim Ngarti, Élise Arnaud, Léo Nicoletti +2Variational Inference Bayesian Inference
AIRSEA · Mines Saint-Étienne MSE, LIMOS, FAYOL-ENSMSE, FAYOL-ENSMSE
Compote: Collapse-Resistant Multiple Hypothesis Learning For Sequential Recommendation
Oct 27, 2026Louis Castres Saint Martin, Clement Calauzenes, Alain Rakotomamonjy +1Sequential Recommendation Recommender Systems
Criteo AI Lab, Paris, France · Sorbonne Université, CNRS, ISIR, 75005 Paris, France
PRISM: Promoting Sampling Diversity of Large Language Models with Implicit Stochastic Modulation
Oct 23, 2026Qixin Xu, Zihan Zhu, Xiaojiang Liu +1Output Diversity in Language Models LLM Training
Tsinghua University. · Apple.
Language Models are Lazy Poets: Creative Mode Collapse at Scale
Oct 16, 2026Jacob A. Matthews, Sil HamiltonOutput Diversity in Language Models Mode Collapse
Cornell University
Mixture of GFlowNets
Oct 12, 2026Omar G. Younis, Joseph D. Viviano, Chirayu Anant Haryan +3Generative Flow Networks Mode Collapse
Mila – Québec AI Institute · Intel · Université de Montréal
Measuring and Mitigating Solution Mode Collapse in RLVR
Oct 8, 2026Liv G. d'Aliberti, Marwa Abdulhai, Sofiia Druchyna +2Mode Collapse RL Post-Training
Princeton University
Immiscible Diffusion Policy: Preserving Multimodal Robot Actions through Label-Free Noise Assignment
Oct 7, 2026Xiao Zhang, Yuxin Chen, Zhixuan Liang +5Diffusion Policy Mode Collapse
University of California, Berkeley · Princeton University · The University of Texas at Austin
Visualizing Distribution Coverage in Generative Diffusion Models
Sep 30, 2026Yifei Wang, Xiaoyu Wu, Tsu-Jui Fu +4Diffusion Distillation Few-Step Diffusion Sampling
Rice University · Apple
Where and When to Force: Routed Forcing for Streaming Avatars
Sep 25, 2026Zihan Su, Siwen Lu, Junhao Zhuang +6Streaming Video Generation Diffusion Model Distillation
Tsinghua University · Joy Future Academy, JD · Southeast University
Fine-Tuning Fixes Mode Collapse and Over-Dispersion in LLMs
Sep 15, 2026Kirill Skobelev, Eric Fithian, X. Y. HanOutput Diversity in Language Models Mode Collapse
Northwestern University · University of Chicago
Forty Shades of Blue: Quality-Diversity Alignment via Mode-Conditioned Reinforcement Learning
Sep 14, 2026Jiayi Yuan, Hangoo Kang, James Jihao Liu +4RL for Language Models Multi-Objective Language Model Alignment
University of Washington · 2Stanford University
Mode Coverage in Normalizing Flow Boltzmann Generators via Log-Ratio Variation
Sep 8, 2026Qi Feng, Rongjie Lai, Di Qi +1Normalizing Flows Molecular Generation
Department of Mathematics, Florida State University, Tallahassee, FL 32306 · Department of Mathematics, Purdue University, West Lafayette, IN 47907
Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rollout
Sep 8, 2026Zhuoran Zhao, Shengju Qian, Tongtong Liang +7Video Diffusion Models Diffusion Model Distillation
1HKUST(GZ) · 2HKUST · 3LIGHTSPEED +3
Beneath the Diff: Diagnosing and Mitigating Algorithmic Mode Collapse in Code-Level Autonomous Research Loops
Aug 31, 2026Bowei He, Weixu Zhang, Yili Jin +1Reward Hacking Automated Algorithm Discovery
MBZUAI · McGill University · MirrorSpace Technology +1
One Symptom, Three Levers: A Critical Review of On-Policy Self-Distillation
Aug 26, 2026Justin Robert, Raheel QaderOn-Policy Self-Distillation On-Policy Distillation
OVHai LLM
Mode Collapse Is Cheap to Detect: A Ground-Truth-Free Pre-Flight Check for Neural Samplers
Aug 20, 2026Jian XuMode Collapse Sampling from Unnormalized Distributions
RIKEN iTHEMS, Wako, Japan · RIKEN Center for Advanced Intelligence Project (AIP), Tokyo, Japan
Optimizing Visual Generative Models via Distribution-wise Rewards
Jul 2, 2026Ruihang Li, Mengde Xu, Shuyang Gu +4Diffusion Model Fine-Tuning RL for Image Generation
University of Science and Technology of China · Shanghai Innovation Institute · Hunyuan Frontier Lab, Tencent +1
CreativityNeuro: Steering Language Model Weights to Improve Divergent Thinking and Reduce Mode Collapse
Jul 1, 2026Samuel Schapiro, Core Francisco Park, Felix Sosa +1Language Model Steering Computational Creativity
Univeristy of Illinois, Urbana-Champaign · Center for Brain Science, Harvard University · CBS-NTT Program in Physics of Intelligence, Harvard University +2
Pigeonholing: how bad prompts hurt models, causing collapse and mistakes
Jun 23, 2026Hyunji Nam, Keertana Chidambaram, Dorottya Demszky +1LLM Reliability In-Context Learning
Stanford University · University of Washington
Data-Forcing Distillation: Restoring Diversity and Fidelity in Few-Step Video Generation
Jun 16, 2026Siyi Chen, Shaowei Liu, Yixuan Jia +4Video Diffusion Models Diffusion Model Distillation
University of Michigan · NVIDIA · University of Illinois Urbana-Champaign
Beyond Imitation: Reflective On-Policy Self-Distillation for LLM Reasoning
May 27, 2026Ziqi Zhao, Xinyu Ma, Liu Yang +6On-Policy Self-Distillation Language Model Distillation
The Hong Kong Polytechnic University · Baidu Inc. · Shandong University +1
Mode-as-Sequence: Translating Multimodal Motion Prediction into Unified Sequential Mode Modeling
May 21, 2026Zikang Zhou, Haibo Hu, Xinhong Chen +5Probability Calibration Autonomous Driving
City University of Hong Kong · City University of Hong Kong (Dongguan) · Hon Hai Research Institute +1
Beyond Mode Collapse: Distribution Matching for Diverse Reasoning
May 19, 2026Xiaozhe Li, Yang Li, Xinyu Fang +10KL-Regularized RL RL for Language Model Reasoning
1Tongji University, China · 6Independent · 2Shanghai AI Laboratory, China +3
Driving Intents Amplify Planning-Oriented Reinforcement Learning
May 12, 2026Hengtong Lu, Victor Shea-Jay Huang, Chengmin Yang +4RL for Autonomous Driving Goal-Conditioned RL
Li Auto · Tsinghua University · CUHK MMLab
Unlocking LLM Creativity in Science through Analogical Reasoning
May 11, 2026Andrew Shen, Shaul Druckmann, James ZouAnalogical Reasoning Large Language Model-Guided Scientific Discovery
Stanford University
Self-Attention as a Covariance Readout: A Unified View of In-Context Learning and Repetition
May 11, 2026Haoren Xu, Guanhua FangSoftmax Attention Repetition in Language Generation
Fudan University
Annotations Mitigate Post-Training Mode Collapse
May 11, 2026Jacob Mitchell Springer, Madhu Advani, Lukas Aichberger +7Supervised Fine-Tuning Diverse Text Generation
Carnegie Mellon University · Apple · Johannes Kepler University Linz
TMPO: Trajectory Matching Policy Optimization for Diverse and Efficient Diffusion Alignment
May 9, 2026Jiaming Li, Chenyu Zhu, Nanxi Yi +9Diffusion Model Alignment Reward Hacking
1MAIR Lab, Huazhong University of Science and Technology · 2Kuaishou Technology · 3Nanyang Technological University +2
Escaping Mode Collapse in LLM Generation via Geometric Regulation
May 1, 2026Xin Du, Kumiko Tanaka-IshiiLanguage Model Steering Decoder-Only Language Models
Department of Communications and Computer Engineering, Waseda University, Tokyo, Japan · Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University, Shanghai, China · Department of Computer Science and Engineering, Waseda University, Tokyo, Japan
Pairing Regularization for Mitigating Many-to-One Collapse in GANs
Apr 22, 2026Kuan-Yu Lin, Yu-Chih Huang, Tie LiuGenerative Adversarial Networks Mode Collapse
Institute of Communications Engineering, National Yang Ming Chiao Tung University · Department of Electrical & Computer Engineering, Texas A&M University

Oct 27, 2026J/K move · Enter open · S save

Rènyi-Regularized Flow-Based Variational Inference for Bayesian Calibration

Exaucè Luweh Adjim Ngarti, Élise Arnaud, Léo Nicoletti, Victor Trappler+1

AIRSEA · Mines Saint-Étienne MSE, LIMOS, FAYOL-ENSMSE, FAYOL-ENSMSE

Variational Inference Bayesian Inference Mode Collapse Normalizing Flows

Open paper AskPDF arXiv

Variational inference with normalizing flows provides a flexible posterior approximation for Bayesian model calibration, but struggles when the posterior has several modes separated by regions of low probability mass. We study this behavior in a two-parameter twin experiment and regularize the variational objective. Under Gaussian observation errors and a Gaussian prior, the corresponding minimization objective is the expected Tikhonov calibration objective minus the differential entropy of the approximating distribution. However, the expected Tikhonov term favors concentration near one of the modes, while entropy maximization does not guarantee multimodality. We introduce a regularization term based on Rényi divergence that encourages the approximate posterior to cover the prior support. In the connected bimodal case, a flow approximation represents both modes; in the case with well-separated modes, the flow and Gaussian mixture frequently miss one mode. The introduced regularization reduces the near-collapse fraction from 50% to 25%. While the regularization improves mode coverage in this experiment, it changes the Bayesian variational objective, introduces a trade-off in approximation accuracy, and does not provide a general solution to mode collapse.