A novel stability-enhanced Gaussian process variational autoencoder (SEGP-VAE) is proposed for indirectly training a low-dimensional linear time invariant (LTI) system, using high-dimensional video data. The mean and covariance function of the novel SEGP prior are derived from the definition of an LTI system, enabling the SEGP to capture the indirectly observed latent process using a combined probabilistic and interpretable physical model. The search space of LTI parameters is restricted to the set of semi-contracting systems via a complete and unconstrained parametrisation. As a result, the SEGP-VAE can be trained using unconstrained optimisation algorithms. Furthermore, this parametrisation prevents numerical issues caused by the presence of a non-Hurwitz state matrix. A case study applies SEGP-VAE to a dataset containing videos of spiralling particles. This highlights the benefits of the approach and the application-specific design choices that enabled accurate latent state predictions.
Figures & tables
Fig. 1: SEGP-VAE architecture with semi-contracting kernel.
Fig. 2: KL divergence (unweighted) and L1 norm during training.
SEGP
PEGP
MLP2
NREG
2 -norm
0.02
0.63
0.63
2.53
TABLE I: ∥A^−Ai∥2 where i∈{SEGP,PEGP,MLP2,NREG} .
Fig. 3: Learnt covariance matrices of SEGP-VAE prior (left) & NREG-VAE prior (right). Top left block is the covariance of the radius and bottom right block is the covariance of the angular position. Top right and bottom left are the intra-dimensional covariances.
Fig. 4: Absolute error between the learnt SEGP mean and the angular position trajectory data, averaged over the test set (left). Variance of the learnt SEGP for the angular position, averaged over the test set (right).
Fig. 5: Absolute error between the mean of learnt models and the angular position trajectory data, averaged over the test set (left). The variance of learnt models for the angular position, averaged over the test set (right).
Fig. 6: Randomly sampled latent trajectory from test set, plotted against the SEGP-VAE prior (top) and posterior (bottom). Radius plotted on left and angular position on right.
Fig. 7: Randomly sampled latent trajectory from test set, plotted against the MLP2-VAE prior (top) and posterior (bottom). Radius plotted on left and angular position on right.
Fig. 8: Randomly sampled latent trajectory from test set, plotted against the learnt PEGP-VAE prior (top) and posterior (bottom). Radius plotted on left and angular position on right.
Fig. 9: Randomly sampled latent trajectory from test set, plotted against the learnt NREG-VAE prior (top) and posterior (bottom). Radius plotted on left and angular position on right.
Fig. 10: Randomly sampled videos from test set (top); corresponding latent trajectories (radius along x-axis, angular position along y-axis) and the SEGP posterior mean (middle); SEGP-VAE reconstructed video (bottom).