Minimum Description Length

MDL studies how to choose explanations that balance fit to observed data against the complexity of the model, using the idea that a good explanation can be encoded concisely. Common approaches include two-part codes, normalized maximum likelihood, prequential coding, and code-length penalties for tasks such as causal discovery, clustering, and symbolic regression. In machine learning, researchers also use compression to analyze generalization, reuse skills, and compare models across high-dimensional or misspecified settings. A central tension is that compression can favor models that generalize, but results depend on the code and optimization procedure, and excessive complexity penalties can lead to underfitting.