Hierarchical Latent Structure Learning through Online Inference

Ines Aitsahalia; Kiyohito Iigaya

cs.LGMar 19, 2026

Hierarchical Latent Structure Learning through Online Inference

Authors: Ines Aitsahalia, Kiyohito Iigaya

Organizations: Center for Theoretical Neuroscience and Zuckerman Institute, Columbia University, New York, NY 10027 · Department of Psychiatry, Columbia University Irving Medical Center, New York, NY 10032 · New York State Psychiatric Institute, New York, NY 10032 · Columbia Data Science Institute, New York, NY 10027

Abstract

Learning systems must balance generalization across experiences with discrimination of task-relevant details. Effective learning therefore requires representations that support both. Online latent-cause models support incremental inference but assume flat partitions, whereas hierarchical Bayesian models capture multilevel structure but typically require offline inference. We introduce the Hierarchical Online Learning of Multiscale Experience Structure (HOLMES) model, a computational framework for hierarchical latent structure learning through online inference. HOLMES combines a variation on the nested Chinese Restaurant Process prior with sequential Monte Carlo inference to perform tractable trial-by-trial inference over hierarchical latent representations without explicit supervision over the latent structure. In simulations, HOLMES matched the predictive performance of flat models while learning more compact representations that supported one-shot backward transfer to higher-level latent categories. In a forward transfer task, HOLMES additionally achieved above-chance outcome prediction for stimuli with never-before-seen feature combinations, by exploiting abstract shape-level representations learned across diverse training instances. These results provide a tractable computational framework for discovering hierarchical structure in sequential data.

Figures & tables

Figure 1: Online hierarchical extension of the Chinese Restaurant Process (CRP) for structure learning. (A) CRP Algorithm. The CRP provides a Bayesian nonparametric prior over partitions, assigning each observation i (filled circle) to an existing cluster k with probability proportional to its occupancy ( $n_{k}$ ) and concentration parameter $\alpha$ or to a new cluster. (B) HOLMES prior with nested Chinese Restaurant Process (nCRP) algorithm. The nCRP provides a Bayesian nonparametric prior over tree-structured partitions, assigning observations to paths by sequentially selecting branches at each level $L$ with probability proportional to occupancy and $\alpha_{L}$ , the level-specific concentration.

Figure 2: Hierarchical inference preserves outcome prediction accuracy while improving representational efficiency. (A) Reward prediction task structure. Each stimulus shares some overlapping features with others, but only a subset are rewarded. (B) Outcome prediction performance on 2-level task. Both models achieve equivalent high accuracy. Error bars: 95% CI across 200 parameter combinations. (C) Representational efficiency measured by average entropy of cluster assignments. HOLMES (hierarchical models) show significantly lower entropy (0.076±0.017 vs. 0.131±0.030), indicating better compression. Error bars: 95% CI across 200 parameter combinations. (D) Outcome prediction performance across task complexity. Both models achieve comparable asymptotic accuracy (84-100%) across all complexity levels, with small effect sizes (Cohen’s $|d|$ ¡ 0.4). Error bars: 95% CI across 200 parameter combinations. (E) Example learned model structures. The flat latent-cause model represents each observation type separately at the observation level, while HOLMES learns a compressed category-level representation. (F) Representational efficiency across complexity. HOLMES shows significantly lower entropy at all levels, with increasing advantage at higher complexities (2-level: -0.055; 5-level: -0.827). Error bars: 95% CI across 200 parameter combinations. (G) Number of learned clusters across task complexity. HOLMES (light) maintains near-optimal compression across complexities, while flat models (dark) show increasing redundancy, using significantly more clusters at all levels (differences: -1.1 to -2.7 clusters). Error bars: 95% CI across 200 parameter combinations.

Figure 3: Hierarchical advantage in one-shot transfer emerges with task complexity. (A) Transfer task structure. Models are first trained on binary outcome prediction as in the previous task. After training, models receive a single labeled example identifying one observation as a particular label, such as ”observation A’ belongs to A” (Teaching phase). In the transfer test, models must generalize this category label to identify which past observations belong to the same latent category, despite never receiving explicit category labels during training. (B) One-shot transfer across task complexity. While both models showed a decrease in recall accuracy as task complexity increased, HOLMES (light, triangle) maintained superior transfer at higher complexities, while flat models (dark, circle) showed declining performance. At 2-level complexity, both models performed comparably (flat: 89.7±2.0%, hierarchical: 89.3±1.8%); however, hierarchical models significantly outperformed flat models at higher complexities (3-level: +21.0%, 95% CI [19.2%, 22.8%]; 4-level: +24.7%, 95% CI [22.6%, 26.8%]; 5-level: +26.6%, 95% CI [24.4%, 28.8%]. Error bars represent 95% confidence intervals across 200 parameter combinations. (C) Hierarchical advantage (hierarchical minus flat transfer accuracy) increases systematically with task complexity, transitioning from no advantage at 2-level to substantial advantages at 3+ levels. Error bars represent 95% confidence intervals across n=200 parameter combinations. (D) One-shot transfer performance at the most complex task for all the levels tested. The HOLMES (hierarchical model; right) outperforms the flat model, with greater advantage on deeper levels. (E) Relationship between outcome prediction accuracy and one-shot transfer accuracy across 200 parameter settings for hierarchical (light) and flat models (dark). Each point represents one parameter combination averaged over seeds. Regression lines show 95% confidence bands.

Figure 4: Forward transfer to novel colors under a shape-only reward rule ( $n=200$ seeds; reward assignment randomized and sign-corrected per seed). (A) Task structure: reward depends only on shape and is invariant to color, which varies across 8 training colors per shape. At test, each shape is paired with a novel color never seen during training; the critical measurement is the model’s prediction on the very first exposure to that color, before any feedback on it has been received. (B) One-shot accuracy on the first presentation of a novel color. Thresholded as a binary decision based on posterior mass. The flat model performed near chance ( $0.566\pm 0.031$ , 95% CI), while HOLMES was near ceiling ( $0.926\pm 0.018$ ). (C) ROC curves for the same first-heldout-color predictions, pooled across all trials and seeds. HOLMES achieved near-perfect discrimination between the rewarded and unrewarded shape (AUC $=0.985$ , bootstrap 95% CI [0.979, 0.991]), while the flat model showed only weak separation (AUC $=0.583$ , [0.547, 0.620]) — indicating that its probability estimates retain a small amount of shape-related ranking signal that is nevertheless insufficient to survive thresholding at $p=0.5$ . (D) Example learned cluster structures. A hierarchical representation that groups by shape and treats color as a nested, exchangeable leaf-level feature supports immediate generalization to a novel color of a known shape. A representation organized by color instead treats every novel color as an unfamiliar category, providing no basis for transfer. (E) Distributions of each model’s learned decision variable relative to the $p=0.5$ boundary, split by whether the held-out trial’s shape was the rewarded or unrewarded one. HOLMES’s distributions are cleanly separated across the boundary; the flat model’s largely overlap, consistent with its near-chance accuracy and modest AUC.

Figure S1: Flat and Hierarchical model sequential learning. (A) One-layer (“flat”) model. Standard CRP-based models maintain a single partition over observations, incrementally assigning each observation to an existing cluster or creating a new one. Trial 1: The first observation creates a new group. Trial 2: The second observation (dashed) creates another new group. Trial 3: The third observation is assigned to the existing compatible group. Trial n: After many trials, the model has discovered multiple groups at a single level of abstraction. (B) Online hierarchical latent structure learning. Our model generalizes the flat formulation by organizing latent structure across multiple levels of abstraction online . The hierarchical extension builds tree structures across multiple levels. Trial 1: The first observation creates new nodes at multiple levels (each marked ’new’). Trial 2: The second observation creates a new branch, forming a sibling relationship with the first observation at higher levels while differing at lower levels, and deepening the hierarchy. Trial 3: The third observation reuses existing structure at higher levels but creates new structure at the observation level. Trial n: The model has discovered a multi-level tree where shared structure at higher levels.

Figure S2: HOLMES recovers standard latent-cause behavior when no hierarchical structure is present. (A) Classical conditioning task structure. During acquisition, CS+ is followed by shock on 50% of trials and by no shock on the remaining 50%, while CS- is never followed by shock. During extinction, shock is withheld entirely, so both cues are followed by no shock on 100% of trials. The idealized associative-strength trace (bottom) shows the qualitative prediction: $P(\text{shock}\mid\text{CS-})$ falls rapidly during acquisition as the deterministic non-reinforcement contingency is learned, while $P(\text{shock}\mid\text{CS+})$ remains near the true reinforcement rate ( $\sim$ 0.5) throughout acquisition and decays toward zero once shock is removed during extinction. (B) Model comparison (n = 50 seeds). Top: asymptotic predictions (final 20% of trials) did not differ between the flat and hierarchical models during acquisition, nor in the number of clusters used, both remaining close to the two true latent groups (CS+, CS-). Bottom: example learning traces of $P(\text{shock}\mid\text{cue})$ across trials for the flat model (left) and HOLMES (right); dashed vertical line marks the acquisition–extinction transition. Both models track statistically indistinguishable associative trajectories for CS+ (blue) and CS- (red).

Hierarchical Levels ( $L$ )	Number of Contexts	Trials per Level (10 per context)
2	8	80
3	16	160
4	32	320
5	64	640

Table 1: Task complexity scaling with number of hierarchical levels.

Level	Model	Recall	Precision	F1
2	Flat	0.897 (0.147)	0.995 (0.025)	0.918 (0.117)
2	Hierarchical	0.893 (0.128)	0.656 (0.097)	0.707 (0.106)
3	Flat	0.445 (0.064)	0.661 (0.132)	0.507 (0.064)
3	Hierarchical	0.655 (0.137)	0.579 (0.091)	0.541 (0.080)
4	Flat	0.329 (0.098)	0.517 (0.076)	0.368 (0.080)
4	Hierarchical	0.576 (0.186)	0.601 (0.077)	0.468 (0.114)

Table 2: Transfer generalization metrics by model and task complexity

Figure S3: Transfer Advantage Parameters. Parameter sweep for both parameters between 0.1-3.0, with 200 randomly sampled parameter values in this range. Parameter heatmap shown with scores on (A) task accuracy as well as average transfer advantage. Stars show the parameter regime that conferred the greatest accuracy or recall advantage, respectively. (B) Transfer accuracy on all levels tested. Brighter colors show higher accuracy. (C) Transfer performance is negatively correlated with task accuracy performance for both models across all levels. This might be because transfer performance encourages generalization which may inhibit early learning in the task.

Figure S4: Parameter sweeps over the concentration parameter $\alpha$ and the stickiness parameter $\Omega$ , comparing hierarchical and flat models across $n=200$ parameter combinations per panel. (A) Compression efficiency (entropy difference, hierarchical $-$ flat) as a function of $\alpha$ . Negative values indicate that the hierarchical model achieves lower entropy (better compression). The hierarchical model showed superior compression in 100% of tested combinations. (B) Compression efficiency as a function of $\Omega$ . The hierarchical advantage was universal across all tested $\Omega$ values. (C) Transfer performance (transfer difference, hierarchical $-$ flat) as a function of $\alpha$ . Positive values indicate that the hierarchical model achieves higher transfer accuracy. The hierarchical model outperformed the flat model in 94% of parameter combinations. (D) Transfer performance as a function of $\Omega$ . The hierarchical advantage increased with higher prior concentration and stickiness, reflecting the benefit of stable hierarchical structure. (E) Forward-transfer ratio as a function of $\alpha$ (99.5% of combinations favored the hierarchical model). (F) Forward-transfer ratio as a function of $\Omega$ . (G) Forward-transfer generalization accuracy as a function of $\alpha$ . (H) Forward-transfer generalization accuracy as a function of $\Omega$ (96% of combinations favored the hierarchical model).

Figure S5: Ablations reveal that stopping and stickiness contribute differently across the two tasks, and that stickiness’s effect is not consistently beneficial. (A) Compositional (backward-transfer) task. One-shot transfer recall for five model variants—Flat, Flat + stickiness, HOLMES without stopping (forced to fixed maximum depth), HOLMES without stickiness, and the full model—as a function of task complexity (2–5 hierarchical levels; shaded 95% CI across 25 sampled parameter cells). All HOLMES variants outperform both the flat models. (B) Shape–color (forward-transfer) task. First-heldout-trial prediction accuracy ( $n=10$ seeds per variant) for the same five variants. Stickiness reduces accuracy in this task. Together, these results indicate that hierarchical structure and stopping generalize their benefit across both backward and forward transfer, whereas stickiness’s usefulness is task-contingent.

Hierarchical Representation Learning

Hierarchical representation learning organizes data into nested, coarse-to-fine features that capture parts, patterns, and global structure.

See all 226 papers

Jun 26, 2026

Accelerating Hierarchical Sparse Predictive Coding with Hybrid Amortized Inference

Hierarchical predictive coding provides an interpretable framework for perception as error-driven inference in multi-layer generative models, while sparse coding imposes parsimonious latent representations through explicit sparsity constraints. Their combination yields hierarchical sparse predictive coding models with appealing computational and neuroscientific properties, but practical use is often limited by the cost of iterative latent inference. In such models, each input may require many recurrent refinement steps before a useful sparse representation is obtained, and this burden becomes more severe as the hierarchy deepens. We study this bottleneck by holding the hierarchical sparse energy fixed and varying the inference procedure. The comparison includes four schemes: classical iterative inference based on ISTA, an accelerated MFISTA reference, structurally informed amortized inference using a LISTA-style bottom-up encoder adapted to the hierarchical model, and a hybrid method in which this fast amortized initialization is followed by a small number of corrective energy-based refinement steps. Under this shared objective, we measure reconstruction quality, sparsity, latency, and stability on static image benchmarks. The results show that a shallow LISTA-style initializer plus short corrective recurrence improves over pure amortization while remaining much faster than long iterative inference.

Kazuhisa Fujita

Department of Clinical Engineering, Komatsu University, 10-10 Doihara-Machi, Komatsu, Ishikawa, Japan 923-0921

Sparse Recovery Efficient Inference Activation Sparsity Predictive Coding Amortized Inference Sparse Coding

Ask PDF

Sep 21, 2026

Brain-Inspired Hierarchical Modularity for General Continual Learning

Continual learning, the ability to learn from sequential experience while retaining and adapting prior knowledge, is central to intelligent systems operating in changing environments. However, conventional continual learning is typically studied with offline task-wise training and clear task boundaries, leaving a substantial gap from general continual learning under online, uncertain, and evolving data streams. In this regime, intelligent systems must separate conflicting experience to reduce interference while integrating compatible experience to promote generalization. Inspired by the organization of the Drosophila learning and memory system, we identify a hierarchical modular principle that coordinates both functions through expert specialization and ensemble integration. We instantiate this principle as lightweight modular adaptation of pretrained foundation models, combining brain-inspired random expansion for expert routing and diversified modular integration across spatial and temporal scales. Across visual recognition, vision-language understanding, ego-exo video understanding, and embodied vision-language-action learning, our method consistently improves learning under online and uncertain data streams, with gains exceeding 50 percentage points over replay-free alternatives in embodied manipulation. These findings support hierarchical modularity as a biologically grounded path for learning from dynamic experience.

Hongwei Yan, Kanglei Zhou, Qi Cheng +7

School of Life Sciences, Tsinghua University, Beijing, China. · IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing, China. · Tsinghua-Peking Center for Life Sciences, Beijing, China. +2

Continual Learning for LLMs Continual Learning for VLA Models Online Continual Learning Continual Learning for VLMs Modular Neural Networks

Ask PDF

Apr 7, 2026

Hierarchical Contrastive Learning for Multimodal Data

Multimodal representation learning is commonly built on a shared-private decomposition, treating latent information as either common to all modalities or specific to one. This binary view is often inadequate: many factors are shared by only subsets of modalities, and ignoring such partial sharing can over-align unrelated signals and obscure complementary information. We propose Hierarchical Contrastive Learning (HCL), a framework that learns globally shared, partially shared, and modality-specific representations within a unified model. HCL combines a hierarchical latent-variable formulation with structural sparsity and a structure-aware contrastive objective that aligns only modalities that genuinely share a latent factor. Under uncorrelated latent variables, we prove identifiability of the hierarchical decomposition, establish recovery guarantees for the loading matrices, and derive parameter estimation and excess-risk bounds for downstream prediction. Simulations show accurate recovery of hierarchical structure and effective selection of task-relevant components. On multimodal electronic health records, HCL yields more informative representations and consistently improves predictive performance.

Huichao Li, Junhan Yu, Doudou Zhou

Department of Mathematical Sciences, University of Chinese Academy of Sciences · Department of Statistics and Data Science, National University of Singapore

Representation Learning Multimodal Contrastive Learning Hierarchical Representation Learning Multimodal Disentangled Representation Learning

Ask PDF

Abstract

Hierarchical Representation Learning

Explore similar work

Accelerating Hierarchical Sparse Predictive Coding with Hybrid Amortized Inference

Brain-Inspired Hierarchical Modularity for General Continual Learning

Hierarchical Contrastive Learning for Multimodal Data