The interest in leveraging physics-based inductive bias in deep learning has resulted in recent development of hybrid deep generative models (hybrid-DGMs) that integrate known physics-based mathematical expressions in neural generative models. To identify these hybrid-DGMs requires inferring parameters of the physics-based component along with their neural component. The identifiability of these hybrid-DGMs, however, has not yet been theoretically probed or established. How does the existing theory of the un-identifiability of general DGMs apply to hybrid-DGMs? What may be an effective approach to construct a hybrid-DGM with theoretically-proven identifiability? This paper provides the first theoretical probe into the identifiability of hybrid-DGMs, and presents meta-learning as a novel solution to construct identifiable hybrid-DGMs. On synthetic and real-data benchmarks, we provide strong empirical evidence for the un-identifiability of existing hybrid-DGMs using unconditional priors, and strong identifiability results of the presented meta-formulations of hybrid-DGMs. Code is available at https://github.com/GoForIt-YYB/Meta-Hybrid-VAE.
Figures & tables
Forced Damped Pendulum
Advection-Diffusion System
APHYNITY
Hybrid-VAE
Meta-Hybrid-VAE
APHYNITY
Hybrid-VAE
Meta-Hybrid-VAE
MSE of zP↓
6.96(0.01)e-2
4.14(0.29)e-2
1.59(0.07)e-2
1.9(0.3)e-2
7.12(5.32)e-4
1.34(0.75)e-4
MCC ↑
0.79(0.02)
0.59(0.03)
0.99(0.00)
0.98(0.01)
0.94(0.00)
0.99(0.00)
MSE of x (Rec) ↓
5.2(0.01)e-2
2.66(0.03)e-2
2.91(0.06)e-2
6.22(0.41)e-3
4.90(0.75)e-3
2.93(0.22)e-3
MSE of x (Pre) ↓
2.37(0.67)
1.74(0.20)
6.85(1.11)e-2
8.25(0.58)e1
8.53(5.72)e-2
5.63(1.34)e-3
Table 1: Quantitative identifiability metrics for the presented meta-hybrid-VAE in comparison to physics-integrated hybrid-VAE [ 2 ] and APHYNITY [ 1 ] , including MSE and MCC metrics on the latent variables, as well as MSE of generated x during reconstruction of observed samples (Rec) and prediction of unobserved samples (Pre).
Figure 1: Visual results on synthetic data for reconstruction and prediction performances.
Figure 2: (Left) The performance of predictions over longer time domains; (Right) A visual example of forced damped pendulum system.The dark part is the training time domain, and the light part is the time domain outside training (longer time domain).
Figure 3: (left) Results of OOD; (right) Results of the condition for identifiability.
Figure 4: Examples of reconstruction and generation results on real double pendulum data.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
The number of parameters
MSE of zP↓
MCC ↑
MSE of x (Rec)
MSE of x (Pre)
2
1.22(0.08)e-2
0.97(0.01)
1.22(0.08)e-2
2.02(0.13)e-2
3
3.62(0.01)e-2
0.99(0.00)
1.29(0.05)e-2
5.01(0.00)e-2
4
1.59(0.07)e-2
0.99(0.00)
2.91(0.06)e-2
6.85(1.11)e-2
Appendix
Table 2: The results of the different dimensions of unknown parameters
Method
MSE of zP↓
MCC ↑
MSE of x (Rec)
MSE of x (Pre)
Non-Meta zP
9.17(1.33)e-3
0.99(0.00)
3.28(0.17)e-2
8.82(2.03)e-2
Meta zP
5.63(0.42)e-3
0.99(0.00)
3.18(0.19)e-2
3.37(0.00)e-2
Appendix
Table 3: Comparison results of Non-meta and Meta
VAE
iVAE
Meta-VAE
MCC
0.67(0.04)
0.91(0.03)
0.88(0.00)
Appendix
Table 4: The results on General Non-Hybrid VAEs
Figure 5: Visual samples of learned latent variables between different runs.
Department of Computing and Information Sciences, Rochester Institute of Technology, Rochester, New York, USA · Department of Computer Science, Rowan University, Glassboro, New Jersey, USA