Symmetry-based disentangled representation learning leverages the group structure of environment transformations to uncover the latent factors of variation. Prior approaches to symmetry-based disentanglement have required strong prior knowledge of the symmetry group's structure, or restrictive assumptions about the subgroup properties. In this work, we remove these constraints by proposing a method whereby an embodied agent autonomously discovers the group structure of its action space through unsupervised interaction with the environment. We prove the identifiability of the true symmetry group decomposition under minimal assumptions, and derive two algorithms: one for discovering the group decomposition from interaction data, and another for learning Linear Symmetry-Based Disentangled (LSBD) representations without assuming specific subgroup properties. Our method is validated on three environments exhibiting different group decompositions, where it outperforms existing LSBD approaches.
Figures & tables
Figure 1
Figure 3: Graphical model
Figure 4: Two isomorphic group actions satisfying Assumption 2 .
Figure 5: Masking used to build disentangled action matrices
Figure 6: Median of disentanglement metrics. Vertical lines indicates the 25th to 75th percentile
Figure 7: Median of long-term prediction error, the shaded area indicates the 25th to 75th percentile
Figure 8: Median of disentanglement metrics on MPI3D
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Symbol
Meaning
Symbol
Meaning
W
World space
Z1⊕Z2
Direct sum of vector spaces
X
Observation space
A↪B
Injective map from A to B
X
Observation random variable
\vvvertA\vvvert
Spectral norm of matrix A
Z
Latent space
Z/nZ
Cyclic group of order n
Z
Latent random variable
Sn
Symmetric group on n elements
G
Action Group
GL(V)
General linear group on V
Appendix
Table 2: Table of symbols
Figure 9
Figure 9: Matrix representation of ρ(g)zk=ρ(gk)zk∈Zk , here the two first dimensions corresponds to Z1 and the remaining two to Z2 .
Figure 10: Metric of interest as a function of the prediction error
Figure 11: Disentanglement according to action entanglement
Figure 12: Decoding mixed latent representations of 3DShapes images
Figure 13: Decoding rotated latent representations of MPI3D image
Figure 14: dG matrices averaged over 5 seeds, normalized so that the clustering threshold is 1.
Graduate School of Information Science and Technology, The University of Tokyo, Tokyo, Japan. · Next Generation Artificial Intelligence Research Center (AI Center), The University of Tokyo, Tokyo, Japan.