Beyond Uniform Local Isometry and Topology: FactoMap for Disentangled Representations

Sohini Gupta; Michael Gao; Bahareh Tolooshams

cs.LGAug 25, 2026

Beyond Uniform Local Isometry and Topology: FactoMap for Disentangled Representations

Authors: Sohini Gupta, Michael Gao, Bahareh Tolooshams

Organizations: University of Alberta Alberta Machine Intelligence Institute (Amii) · University of Alberta Alberta Machine Intelligence Institute (Amii) Canada CIFAR AI Chair

Abstract

Many disentanglement methods represent generative factors using Euclidean product coordinates, although the underlying factor spaces may wrap, collapse, or have position-dependent geometry. We introduce factor-space structure, which combines factor domains, generator-induced identifications, and position-dependent scales to distinguish topologically equivalent spaces with different factor geometries. We show that statistical independence does not imply uniform product geometry: independent hue and scale can affect observations at different, scale-dependent rates, producing anisotropy that no fixed rescaling can remove. We propose the Factor-Space Topographic Map (FactoMap), which learns interpretable prototypes indexed by a structured factor-space lattice. Topographic learning transfers the lattice's periodicity, collapses boundaries, and non-uniform extent to the representation. Experiments show that matching this structure better preserves factor continuity and improves disentanglement of the underlying factors.

Figures & tables

Figure 1: FactoMaps: Factor-Space + Topology + Topography. A. Visuals of different topological spaces; the grid and cone are homeomorphic but differ in factor-space geometry. B. Left: A generator maps two factors, position and scale, to observations. Right: FactoMap learns prototypes on a lattice encoding the factor topology and geometry, organizing its axes by position and scale.

Figure 2: The generator induces position-dependent anisotropy.

Figure 3: FactoMap recovers a global cyclic hue coordinate; $\beta$ -VAE does not. A. Six base images have equal scale and different hues; green outlines mark the bases during each traversal. B. FactoMap axis 1 changes scale, while axis 2 traverses the complete hue circle from every base. C. One $\beta$ -VAE latent changes scale, but its hue-related latent covers only a base-dependent arc of the circle.

Table 1: Preserving factor-space topology and geometry improves disentanglement. Left: FactoMap with a cone-structured lattice outperforms cylinder and grid lattices. Right: FactoMap better captures cyclic hue than

\beta

-VAE. NMI is reported as (hue, scale).

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: FactoShapes Dataset. Sampling distribution (uniform) and visuals of rendered sample images on the entire dataset on left and rendered sample images for only object hue and scale as varying factors (with uniform distribution) on right.

Figure 5: One-dimensional factors: matching the domain and the local scale. Top (Data). The scale factor is an interval and the hue factor a circle. Measured image-space speeds (solid) against the analytical predictions (dashed): $c_{s}$ grows with scale, while $c_{h}$ is near-constant across hue, dipping at the HSV sector boundaries. Bottom (FactoMap). A line lattice represents scale continuously, with a nonlinear cumulative prototype distance reflecting the position-dependent $c_{s}$ . For periodic hue, an open line lattice places the adjacent endpoint hues at opposite ends ( $\Delta_{\mathrm{lattice}}=39$ ), whereas a ring lattice restores adjacency ( $\Delta_{\mathrm{lattice}}=1$ ) and yields a closed cycle of prototypes.

Figure 6: Visualizations of learned prototypes by FactoMap n a grid, cylinder and cone lattice. The cylinder and cone is cut to unfold the lattice to visualization the alignment of prototypes.

n bins	$\beta$	InfoM	InfoE (Regression)	InfoE (Classification)	InfoC
10	0.1	0.568	0.787	0.703	0.382
	0.3	0.983	0.788	0.951	0.625
	1	0.985	0.768	0.941	0.620
	3	0.986	0.803	0.927	0.616
	10	0.981	0.771	0.933	0.615
	15	0.982	0.747	0.925	0.625

Appendix

Table 2: Dataset: FactoShapes, Model:

\bm{\beta}

-VAE. Aggregate InfoMEC Scores. All

\beta

values were first evaluated, after which four additional runs were performed only for the best-performing

\beta

to compute the reported mean. These runs exhibited negligible variance, so standard deviations are omitted for clarity.

Figure 7: NMI matrices on FactoShapes. Rows are factors, columns are representation coordinates; entries are $\mathrm{NMI}_{ij}$ on a common $[0,1]$ scale. Both models concentrate scale in one coordinate. FactoMap concentrates hue in its angular coordinate, whereas the $\beta$ -VAE splits it across two latents; inactive latents are red.

Figure 8: The $\bm{\beta}$ -VAE fails to represent the complete hue cycle with a single continuous latent coordinate. With scale fixed, its latent traversals on FactoShapes partition hue into base-dependent arcs and exhibit discontinuous jumps, including abrupt yellow-to-blue transitions. Each row starts from a base image of a different hue, marked by a green outline, and columns show values of the traversed latent. Latents are ordered by decreasing maximum factor-wise NMI.

Figure 9: $\bm{\beta}$ -VAE partitions the cyclic hue factor across latents. The base hue is fixed at green, while rows correspond to base images of different scales, highlighted by red boxes. Traversing latent 3 covers only the green-to-light-blue arc, whereas latent 1 covers approximately half of the hue circle; neither traverses the complete cycle. Columns show values of the traversed latent. Latents are ordered by decreasing maximum factor-wise NMI.

Figure 10: FactoMap represents the complete hue cycle within a single factor axis. Traversals on FactoShapes are shown with scale fixed. Each row starts from a base image of a different hue, marked by a green outline, and columns follow values along the traversed axis. The learned axes separate hue and scale, while the hue axis covers the complete cycle and preserves its periodic closure.

Figure 11: FactoMap disentangles continuous scale from cyclic hue. Factor-axis traversals begin from base images of different scales, marked by red outlines. Factor axis 0 changes object scale without altering hue, whereas factor axis 1 traverses the complete hue cycle at every base scale without altering scale. Columns show positions along the traversed axis.

n bins	$\beta$	InfoM	InfoE (Regression)	InfoE (Classification)	InfoC
10	0.1	0.592	0.560	0.419	0.505
	0.3	0.607	0.576	0.442	0.687
	1	0.364	0.567	0.405	0.394
	3	0.820	0.587	0.420	0.827
	10	0.823	0.591	0.385	0.877
	15	0.839	0.597	0.391	0.785

Appendix

Table 3: Dataset: dsprites, Model:

\bm{\beta}

-VAE. Aggregate InfoMEC Scores.

Figure 12: NMI matrix of the $\bm{\beta}$ -VAE on dSprites, all factors ( $\beta=10$ ). Rows are the five factors, columns the ten latents; inactive latents are red.

Varying Factors	n bins	InfoM	InfoE (Regression)	InfoE (Classification)	InfoC
Position X–Position Y	10	0.666	0.996	0.895	0.660
	20	0.657	0.996	0.895	0.658
	50	0.598	0.996	0.895	0.597
	auto	0.666	0.996	0.895	0.673
Scale–Shape	10	0.241	0.990	0.999	0.241
	20	0.047	0.990	0.999	0.047

Appendix

Table 4: Dataset: dsprites, Model:

\bm{\beta}

-VAE,

\bm{\beta}

: 10. Pairwise Factor Evaluation InfoMEC Scores.

Figure 13: Pairwise NMI matrices of the $\bm{\beta}$ -VAE on dSprites ( $\beta=10$ ). One panel per pair of varying factors; columns keep the same latent index in every panel, so that a latent can be followed across pairs. Inactive latents are red.

Disentangled Representation Learning

Disentangled representation learning separates distinct factors of variation so models can interpret, control, and generalize their predictions.

See all 231 papers

Unsupervised Disentanglement Without Compromises : How Functional Orthogonality Enforces Identifiability
Jun 19, 2026Mathieu Cyrille Simon, Pascal Frossard, Christophe De VleeschouwerUnsupervised Learning Disentangled Representation Learning
1UCLouvain, ICTEAM, Louvain-la-Neuve, Belgium · 2EPFL, LTS4 laboratory, Lausanne, Switzerland.
XFACTORS: Disentangled Information Bottleneck via Contrastive Supervision
Jan 29, 2026Alexandre Myara, Nicolas Bourriez, Thomas Boyer +3Variational Autoencoders Disentangled Representation Learning
ENS Paris
Disentanglement Beyond Generative Models with Riemannian ICA
May 21, 2026Edmond CunninghamDisentangled Representation Learning Independent Component Analysis
University of Massachusetts Amherst

Jun 19, 2026J/K move · Enter open · S save

Unsupervised Disentanglement Without Compromises : How Functional Orthogonality Enforces Identifiability

Mathieu Cyrille Simon, Pascal Frossard, Christophe De Vleeschouwer

1UCLouvain, ICTEAM, Louvain-la-Neuve, Belgium · 2EPFL, LTS4 laboratory, Lausanne, Switzerland.

Unsupervised Learning Disentangled Representation Learning Latent Variable Models Representation Identifiability Orthogonal Representation Learning

Open paper AskPDF arXiv

This paper explores unsupervised disentangled representation learning from a functional perspective. We define latent concepts as factors that influence observations through locally orthogonal directions, formalized as an orthogonality constraint on the Jacobian of the generative mapping. We prove that this condition yields identifiability of general nonlinear generative models, without requiring statistical independence or causal assumptions, provided the latent domain admits all combinations of factor values. Experiments with orthogonality-regularized normalizing flows empirically confirm the theory, demonstrate reliable recovery of ground-truth factors, and shed light on the success of VAEs. These findings challenge the prevailing impossibility claims for unsupervised disentanglement and provide a principled alternative foundation.

Abstract

Disentangled Representation Learning

Explore similar work

Unsupervised Disentanglement Without Compromises : How Functional Orthogonality Enforces Identifiability

XFACTORS: Disentangled Information Bottleneck via Contrastive Supervision

Disentanglement Beyond Generative Models with Riemannian ICA