cs.LGJun 5, 2026

Bootstrap Theory of Representational Emergence: Explanatory Insufficiency as a Driver of Representation Learning and World Models

Authors: Jacques Raynal, Pierre Slangen, Elsa Raynal, Jacques Margerit

Organizations: Laboratory of Bioengineering and Nanosciences (LBN), University of Montpellier, Montpellier, France · 2EuroMov Digital Health in Motion, University of Montpellier, IMT Mines Alès, Alès, France · 3Certified Sophrologist and Dental Assistant, Sensorimotor Practice, Montpellier, France · 4Emeritus Professor, University of Montpellier, Montpellier, France

Abstract

Representation learning is central to modern machine learning, but most research examines how representations are optimized after a framework has been selected. Less attention is given to when a new representational level becomes necessary. This article introduces the Bootstrap Theory of Representational Emergence (TBER), an initial conceptual theory and research program describing how new representations arise when existing ones become explanatorily insufficient. A representation may remain descriptively useful while failing to make certain observations, relations, transformations, or organizational properties intelligible. TBER treats this explanatory insufficiency as a positive signal for representational transition rather than as simple falsification or prediction error. The proposed recursive process follows five stages: stabilized observation, anomaly detection, recognition of explanatory insufficiency, representational emergence, and provisional stabilization. The framework concerns transitions between scientific or computational representations, not equivalent transitions within the physical systems being observed. Its scope includes representation learning, latent spaces, foundation models, world models, digital twins, adaptive biological systems, and scientific discovery. TBER does not propose a new algorithm, model architecture, benchmark, or optimization procedure. Its contribution is meta-representational: it provides a framework for interpreting when and why new representational levels become necessary. A possible implication for future artificial intelligence is the development of systems able to detect when their own internal representations have reached explanatory limits and to initiate representational refinement or transition.

Explore similar work

CardsList
Jun 11, 2026cs.LG

Detecting Explanatory Insufficiency in Learned Representations: A Framework for Representational Vigilance

Learned representations are central to modern machine learning and are commonly evaluated through predictive performance, robustness, uncertainty estimation, and generalization. However, a representation may remain operationally successful while failing to organize persistent residual structures that conventional metrics do not fully capture. This article introduces VER, the Vigilant Evaluator of Representations, a conceptual framework for monitoring representational adequacy. VER does not propose a new learning algorithm, loss function, or model architecture. It defines a diagnostic process for identifying residual structures and assessing whether they may indicate explanatory insufficiency rather than ordinary error, uncertainty, noise, data limitation, or distribution shift. The framework comprises five operations: representation identification, explanatory-domain delimitation, residual-structure detection, explanatory-resistance evaluation, and vigilance signaling. VER distinguishes stable adequacy, a vigilance condition, and a representational alert. It is intended to complement performance evaluation, uncertainty estimation, out-of-distribution detection, robustness analysis, and explainable AI by making representational adequacy an explicit object of inquiry. The article also outlines a path toward empirical evaluation through benchmarks designed to detect representational inadequacy when predictive performance remains satisfactory. VER is conceptual and methodological; it does not prove that a representation is inadequate, select a replacement representation, or provide an operational implementation.
Jacques Raynal, Pierre Slangen, Elsa Raynal +1
Representation LearningRepresentational DriftSufficient ConditionsUncertainty-Aware Teachers
AskPDF
May 5, 2026cs.LG

Learning to Theorize the World from Observation

What does it mean to understand the world? Contemporary world models often operationalize understanding as accurate future prediction in latent or observation space. Developmental cognitive science, however, suggests a different view: human understanding emerges through the construction of internal theories of how the world works, even before mature language is acquired. Inspired by this theory-building view of cognition, we introduce Learning-to-Theorize, a learning paradigm for inferring explicit explanatory theories of the world from raw, non-textual observations. We instantiate this paradigm with the Neural Theorizer (NEO), a probabilistic neural model that induces latent programs as a learned Language of Thought and executes them through a shared transition model. In NEO, a theory is represented as an executable, compositional program whose learned primitives can be systematically recombined to explain novel phenomena. Experiments show that this formulation enables explanation-driven generalization, allowing observations to be understood in terms of the programs that generate them.
Doojin Baek, Gyubin Lee, Junyeob Baek +2
Theory-Of-Mind ReasoningLlm-Synthesized Code World ModelsTheoretical FoundationsExplanationsSequential Learning\Emph{State-Prediction Separation Hypothesis+1
AskPDF
May 22, 2026cs.LG

Feature Lottery? A Bifurcation Theory of Concept Emergence

Neural networks acquire structured representations at specific moments during training, yet identifying these transitions typically relies on retrospective, label-dependent metrics. We introduce a bifurcation theory of representation dynamics to detect these moments in real time. Analyzing a passive GMM probe attached to the evolving encoder, we show the onset of structure corresponds to a supercritical pitchfork bifurcation driven by the loss Hessian. The system exhibits a theoretically predictable zero-crossing (βcβ_c) that, compared to the network's current state (ββ), yields a dynamic ratio β(t)/βc(t)β(t)/β_c(t): a universal, label-free phase coordinate for representation dynamics, computable entirely from hidden states. We empirically validate four distinct transition regimes predicted by this coordinate across diverse settings: SAEs on language models (Pythia), SSL (CIFAR), and grokking (modular arithmetic). Crucially, under finite dissipation, macroscopic symmetry-breaking can lag the initial zero-crossing by orders of magnitude, which providing a rigorous dynamical account of the delayed escape observed in grokking. Microscopically, the bifurcation creates a shared unstable subspace, forcing collective symmetry breaking. We term this the "feature lottery" in SAE training: a feature's terminal interpretability becomes predictable remarkably early. By only 5% of training, early atom purity robustly predicts final convergence purity, with top-decile early atoms achieving over 12x the baseline purity at convergence. Beyond explaining concept emergence, β/βcβ/β_c provides a practical early-warning indicator for training health, detecting the onset of usable structure, the crystallization of feature identity, and representational collapse epochs before downstream metrics react.
Fuming Yang
Feature LearningBifurcationRepresentational DriftTheoretical FoundationsRepresentation QualityLotteries+2
AskPDF