When Do Learned Blocks Preserve Causal Meaning? Block Identifiability and Constructive Causal Abstractions

Karim Zaghw; Wieland Brendel; Patrik Reizinger

cs.LGOct 13, 2026

When Do Learned Blocks Preserve Causal Meaning? Block Identifiability and Constructive Causal Abstractions

Authors: Karim Zaghw, Wieland Brendel, Patrik Reizinger

Organizations: University of Tübingen · ELLIS Institute Tübingen · Max Planck Institute for Intelligent Systems, Tübingen, Germany

Abstract

Standard identifiability theory assumes one-dimensional latent factors, but learned representations can encode latent information across multidimensional blocks whose internal coordinates are not uniquely determined. When such blocks are used to represent and intervene on higher-level concepts, it is unclear whether their causal meaning survives the transformations left unresolved by block-identifiability. We connect block-identifiability to constructive causal abstraction. We show that block-diagonal invertible transformations followed by compatible coordinate permutations form the maximal class that preserves every constructive causal abstraction built on those blocks. This guarantee assumes that the correspondence between blocks and concepts is updated with the representation. A small synthetic experiment with additive autoencoders shows that high-level interventions can be transferred between independently learned representations while approximately preserving their intended effects.

Figures & tables

Figure 1: From ground-truth factors to a constructive causal abstraction of a learned representation. The ground-truth causal factors

\mathbf{S}

take values

\mathbf{s}

, which determine the network input and the resulting activation vector

\mathbf{o}=f(\mathbf{s})

. A featurizer

g

, such as a sparse autoencoder (SAE), maps these activations to the learned representation

\mathbf{x}=g(\mathbf{o})

. We treat the coordinates of

\mathbf{x}

as variables of the low-level structural causal model (SCM)

L

and partition them into blocks

\mathbf{V}^{(b)}

. The constructive causal abstraction

(\tau,\omega)

maps low-level states and complete-block interventions to states and interventions in the high-level SCM

H

, where each human-interpretable concept

Z^{(b)}

is computed from its corresponding block. Blue and pink indicate this block-to-concept correspondence. The ground-truth factors

\mathbf{S}

generate the activations but need not coincide with the concepts in

H

Figure 2: Concept-directed steering and transport between independently trained additive autoencoders. Starting from a base observation, the high-level intervention

J^{\star}=\mathrm{do}(Z^{(1)}{=}s_{1}^{\star})

H

is implemented in the reference model

L_{0}

by editing block 1. Here,

\pi_{1}

maps that block to

Z^{(1)}

, and

\mathbf{x}_{0}^{(1)\star}

is the closest block value satisfying

\pi_{1}(\mathbf{x}_{0}^{(1)\star})=s_{1}^{\star}

; block 2 remains unchanged. The fitted transformation

\widehat{T}_{r}=\widehat{\sigma}_{r}\circ\widehat{D}_{r}

transfers the edit to recipient model

L_{r}

, where

\widehat{D}_{r}

maps corresponding blocks and

\widehat{\sigma}_{r}

accounts for their ordering. The decoders

G_{0}

and

G_{r}

produce the edited outputs

\widehat{\mathbf{o}}_{0}^{\star}

and

\widehat{\mathbf{o}}_{r}^{\star}

. For evaluation, the exact factor setting

\mathbf{s}^{\mathrm{gen}}=(s_{1}^{\star},u_{1},s_{2},u_{2})

changes only

s_{1}

, setting it to the intervention value

s_{1}^{\star}

, and the data-generating map

f

produces the comparison observation

\mathbf{o}^{\mathrm{gen}}=f(\mathbf{s}^{\mathrm{gen}})

. Both decoded outputs are compared with this same observation, as indicated by the dotted lines. Green marks the intervention, while blue and pink identify the two blocks throughout.

Model (seed)	Recon.	$Z^{(1)}$	$Z^{(2)}$
Reference (0)	$0.007312$	$0.007407$	$0.007378$
Recipient (2)	$0.007927$	$0.011931$	$0.010487$
Recipient (3)	$0.007762$	$0.009342$	$0.010307$
Recipient (5)	$0.007825$	$0.010572$	$0.010985$
Recipient (10)	$0.007779$	$0.009505$	$0.010923$

Table 1: Reconstruction and concept-intervention accuracy across retained additive autoencoders. Rows show one reference additive autoencoder and four independently trained recipient additive autoencoders, identified by training seed. “Recon.” is the normalized mean squared error (NMSE) between each model’s decoded reconstructions and held-out synthetic input observations; only models with reconstruction NMSE at most

0.01

are retained. The

Z^{(b)}

columns report intervention error. An intervention on

Z^{(b)}

is implemented as a block edit in the reference model and transported to each recipient model. Each decoded edited output is compared with the exact synthetic observation obtained by changing only

s_{b}

to the intervention value while keeping

u_{b}

and

(s_{3-b},u_{3-b})

fixed. Each NMSE pools

5{,}000

examples; lower values are better.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Term	Notation	Meaning	Defined in
Representation (feature vector)	$\mathbf{x}=g(\mathbf{o})$	Featurizer output whose coordinates form $\mathbf{V}_{L}$	§ 2
Activation vector	$\mathbf{o}$	Activation read from the network and supplied to the featurizer $g$	§ 2
Featurizer	$g$	Maps network activations to $\mathbf{x}$ ; examples include SAEs and block-sparse featurizers	§ 2
Learned feature	$X\in\mathbf{V}_{L}$	Coordinate variable of the featurizer output	§ 2
Concept	$Z^{(b)}\in\mathbf{V}_{H}$	High-level variable, possibly encoded across several learned coordinates	§ 2
Low-level model	$L$ , $\mathbf{V}_{L}$	Representation-level SCM on the featurizer output, whose coordinates carry the block partition	§§ 2 and 1

Appendix

Table 2: Core terminology and notation used in the paper.

Causal Representation Learning

Causal representation learning seeks latent features that reveal underlying causal variables and mechanisms for robust prediction, intervention, and transfer.

See all 101 papers

Beyond identifiability: Learning causal representations with few environments and finite samples
Mar 26, 2026Inbeom Lee, Tongtong Jin, Bryon AragamCausal Representation Learning Causal Structure Learning
Unsupervised Causal Abstractions Discovery
Jun 17, 2026Théo Saulus, Simon Lacoste-Julien, Dhanya SridharStructural Causal Models Causal Discovery
Mila - Quebec AI Institute · Université de Montréal · Canada CIFAR AI Chair
Block Disentanglement in CRL: Bridging Identifiability and Visual State Estimation
Oct 5, 2026Emre Acartürk, Pranamya Kulkarni, Puranjay Datta +3Causal Representation Learning Causal Identifiability
Rensselaer Polytechnic Institute Troy, NY, USA · Swiss AI, EPFL Lausanne, Switzerland · University of Delaware Newark, DE, USA

Mar 26, 2026J/K move · Enter open · S save

Beyond identifiability: Learning causal representations with few environments and finite samples

Inbeom Lee, Tongtong Jin, Bryon Aragam

Causal Representation Learning Causal Structure Learning Causal Intervention

Open paper AskPDF arXiv

We provide explicit, finite-sample guarantees for learning causal representations from data with a sublinear number of environments. Causal representation learning seeks to provide a rigourous foundation for the general representation learning problem by bridging causal models with latent factor models in order to learn interpretable representations with causal semantics. Despite a blossoming theory of identifiability in causal representation learning, estimation and finite-sample bounds are less well understood. We show that causal representations can be learned with only a logarithmic number of unknown, multi-node interventions, and that the intervention targets need not be carefully designed in advance. Through a careful perturbation analysis, we provide a new analysis of this problem that guarantees consistent recovery of (a) the latent causal graph, (b) the mixing matrix and representations, and (c) unknown intervention targets.

Abstract

Causal Representation Learning

Explore similar work

Beyond identifiability: Learning causal representations with few environments and finite samples

Unsupervised Causal Abstractions Discovery

Block Disentanglement in CRL: Bridging Identifiability and Visual State Estimation