Posterior Collapse

Follow this topic Ask about this topic

Momentum

0 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 8

All topics

Uncovering Latent Reasoning Strategies in Language Models
Jul 20, 2026Awni Altabaa, John LaffertyPosterior Collapse Implicit Reasoning in Language Models
Yale University
$\mathbfλ$ -VAE: Variance Equalization for Posterior Collapse
Jul 6, 2026Girum DemisseVariational Autoencoders Posterior Collapse
Microsoft African Development Center Nairobi Kenya
An Analysis of Posterior Collapse, Parameterization and Initialization in Variational Deep Gaussian Processes
Jun 24, 2026Francisco Javier Sáez-Maldonado, Juan Maroñas, Daniel Hernández-LobatoDeep Learning Optimization Posterior Collapse
Department of Computer Science and Artificial Intelligence Universidad de Granada Granada, Spain · Department of Quantitative Methods CUNEF Universidad Madrid, Spain · Machine Learning Group at Universidad Autónoma de Madrid +1
Posterior Collapse as Automatic Spectral Pruning
May 21, 2026Johannes HirnVariational Autoencoders Posterior Collapse
1Image Processing Laboratory (IPL), Universitat de València, Paterna, València 46980, Spain∗
A Simplex Witness Certificate and Escape Force for Constant Collapse in Variational Autoencoders
May 18, 2026Zegu Zhang, Jian ZhangVariational Autoencoders Representation Learning
Independent Researcher
Entropic Auto-Encoding via Implicit Free-Energy Minimization
May 15, 2026Hazhir Aliahmadi, Irina Babayan, Greg van AndersVariational Autoencoders Posterior Collapse
Department of Physics, Engineering Physics & Astronomy, Queen’s University, Kingston, ON, Canada
Entropy-Based Characterisation of the Polarised Regime in Latent Variable Models
May 15, 2026Peter Clapham, Lisa Bonheme, Marek GrzesVariational Autoencoders Latent Variable Models
School of Computing, University of Kent, Canterbury, United Kingdom · International Agency for Research on Cancer (IARC/WHO), Genomic Epidemiology Branch, Lyon, France
A Testable Certificate for Constant Collapse in Teacher-Guided VAEs
May 7, 2026Zegu Zhang, Jianhua Peng, Jian ZhangVariational Autoencoders Posterior Collapse
Independent Researcher · school of computing, Southeast University

Jul 20, 2026, cs.LGJ/K move · Enter open · S save

Uncovering Latent Reasoning Strategies in Language Models

Awni Altabaa, John Lafferty

Yale University

Posterior Collapse Implicit Reasoning in Language Models Variational Inference

Open paper AskPDF arXiv

A language model

p_θ(y \mid x)

trained on reasoning tasks learns to solve problems via multiple distinct strategies, yet these strategies are implicit and entangled within the model's response distribution. We study the problem of decomposing the response distribution of a given pretrained language model into a structured, strategy-conditioned representation. Specifically, we learn a latent-variable factorization

p_θ(y \mid x) \leadsto (r_φ(z \mid x), g_φ(y \mid x,z))

, where a router

r

maps each input to a distribution over latent strategies

z

and a generator

g

produces the response conditioned on that strategy. A key challenge is that the generator, initialized from the base model, already represents

p_θ(y \mid x)

without using

z

. Standard variational inference therefore gives the model no incentive to route information through

z

and can yield a severe form of posterior collapse. To address this, we propose a variational objective that measures fractional information gain relative to the base model's response loss and concentrates reconstruction pressure on tokens with high base model surprisal, encouraging

z

to encode strategy-relevant response variation. We introduce a benchmark of multi-strategy algorithmic tasks and show that this objective recovers latent codes aligned with distinct reference strategies while preserving the base model's response distribution.

Momentum

Related topics

Latest papers 8