Slack More, Predict Better: Proximal Relaxation for Probabilistic Latent Variable Model-based Soft Sensors

Zehua Zou; Yiran Ma; Yulong Zhang; Zhengnan Li; Zeyu Yang; Jinhao Xie; Xiaoyu Jiang; Zhichao Chen

cs.LGMar 12, 2026

Slack More, Predict Better: Proximal Relaxation for Probabilistic Latent Variable Model-based Soft Sensors

Authors: Zehua Zou, Yiran Ma, Yulong Zhang, Zhengnan Li, Zeyu Yang, Jinhao Xie, Xiaoyu Jiang, Zhichao Chen

Organizations: the Hangzhou International Innovation Institute, Beihang University, Hangzhou 311115, China · the State Key Laboratory of Industrial Control Technology, College of Control science and Engineering, Zhejiang University, Hangzhou 310027, Zhejiang, China · the College of Electrical Engineering and Automation, Fuzhou University, Fuzhou 350108, China · the School of Data Science, The Chinese University of Hong Kong, Shenzhen 518172, China · the Zhejiang Key Laboratory of Industrial Solid Waste Thermal Hydrolysis Technology and Intelligent Equipment, Huzhou Key Laboratory of Intelligent Sensing and Optimal Control for Industrial Systems, School of Engineering, Huzhou University, Huzhou 313000, China · MOE of the Key Laboratory of Bioinorganic and Synthetic Chemistry, the Key Lab of Low-Carbon Chem & Energy Conservation of Guangdong Province, School of Chemistry, Sun Yat-Sen University, Guangzhou 510275, China · the National Key Lab of General AI, School of Intelligence Science and Technology, Peking University, Beijing 100871, China

Abstract

Nonlinear Probabilistic Latent Variable Models (NPLVMs) are a cornerstone of soft sensor modeling due to their capacity for uncertainty delineation. However, conventional NPLVMs are trained using amortized variational inference, where neural networks parameterize the variational posterior. While facilitating model implementation, this parameterization converts the distributional optimization problem within an infinite-dimensional function space to parameter optimization within a finite-dimensional parameter space, which introduces an approximation error gap, thereby degrading soft sensor modeling accuracy. To alleviate this issue, we introduce KProxNPLVM, a novel NPLVM that pivots to relaxing the objective itself and improving the NPLVM's performance. Specifically, we first prove the approximation error induced by the conventional approach. Based on this, we design the Wasserstein distance as the proximal operator to relax the learning objective, yielding a new variational inference strategy derived from solving this relaxed optimization problem. Based on this foundation, we provide a rigorous derivation of KProxNPLVM's optimization implementation, prove the convergence of our algorithm can finally sidestep the approximation error, and propose the KProxNPLVM by summarizing the abovementioned content. Finally, extensive experiments on synthetic and real-world industrial datasets are conducted to demonstrate the efficacy of the proposed KProxNPLVM.

Figures & tables

Panel of Figure 1 — Fig. 1: Posterior distribution approximation comparison.

Fig. 2: The overall workflow of KProxNPLVM.

Panel of Figure 3 — Fig. 3: The evolution trajectory of the $\mathcal{Q}_{t}(z)$ estimated by the kernel density estimation (KDE), and the 2-Wasserstein distance $\mathcal{W}_{2}(\mathcal{Q}_{t}(z),\mathcal{P}(z|\mathcal{D}))$ .

Model	DBC				CAC				CSC
Model	$\text{R}^{\text{2}}$	RMSE	MAE	MAPE	$\text{R}^{\text{2}}$	RMSE	MAE	MAPE	$\text{R}^{\text{2}}$	RMSE	MAE	MAPE
SNPLVR	-1.02E-1 $\dagger$	2.11E-1 $\dagger$	1.71E-1 $\dagger$	2.89E2 $\dagger$	-2.29E-1 $\dagger$	7.90E-3 $\dagger$	5.94E-3 $\dagger$	2.03E0 $\dagger$	-3.13E-1 $\dagger$	6.76E-1 $\dagger$	5.42E-1 $\dagger$	2.76E-1 $\dagger$
DBPSFA	2.51E-1 $\dagger$	1.75E-1 $\dagger$	1.43E-1 $\dagger$	2.80E2 $\dagger$	-6.78E-2	7.36E-3	5.63E-3	1.93E0	-3.73E4 $\dagger$	1.15E2 $\dagger$	1.15E2 $\dagger$	5.85E1 $\dagger$
MUDVAE-SDVAE	-1.04E-2 $\dagger$	2.03E-1 $\dagger$	1.62E-1 $\dagger$	2.75E2 $\dagger$	-5.29E-3 $\dagger$	7.14E-3 $\dagger$	5.23E-3 $\dagger$	1.78E0 $\dagger$	-1.64E-1 $\dagger$	6.41E-1 $\dagger$	5.13E-1 $\dagger$	2.61E-1 $\dagger$
GMM-VAE	7.93E-1 $\dagger$	8.15E-2 $\dagger$	6.50E-2 $\dagger$	8.99E1 $\dagger$	2.89E-1 $\dagger$	6.14E-3 $\dagger$	4.66E-3 $\dagger$	1.59E0 $\dagger$	7.83E-1 $\dagger$	2.75E-1 $\dagger$	2.19E-1 $\dagger$	1.11E-1 $\dagger$
GSTAE	9.70E-1 $\dagger$	3.52E-2 $\dagger$	1.48E-2 $\dagger$	2.56E1 $\dagger$	7.40E-1 $\dagger$	3.63E-3 $\dagger$	2.72E-3 $\dagger$	9.30E-1 $\dagger$	8.93E-1 $\dagger$	1.93E-1 $\dagger$	1.53E-1 $\dagger$	7.77E-2 $\dagger$

TABLE I: Soft Sensor Accuracy Comparison

Panel of Figure 5 — Fig. 4: The sensitivity analysis results vary proximal operator coefficient $\varepsilon$ , batch size $\mathcal{B}$ , inference network $\eta$ , and particle number $\ell$ on the DBC dataset. The shaded area indicates the $\pm 0.5$ standard deviation error.

Dataset	KProx	Wass	$\text{R}^{\text{2}}$ ( $\downarrow$ )	RMSE ( $\uparrow$ )	MAE ( $\uparrow$ )	MAPE ( $\uparrow$ )
DBC	✗	✓	2.73E3%	1.53E3%	1.59E3%	1.43E3%
	✓	✗	4.20E0%	2.34E2%	2.50E2%	3.22E2%
	✗	✗	2.44E3%	1.53E3%	1.59E3%	1.42E3%
CAC	✗	✓	1.86E5%	1.00E2%	8.79E1%	8.64E1%
	✓	✗	7.51E1%	5.14E1%	4.86E1%	4.82E1%
	✗	✗	7.87E4%	1.01E2%	8.85E1%	8.71E1%

TABLE II: Ablation Study Results

Fig. 5: The convergence results of the training process of KProx algorithm on three datasets. The shaded area indicates

\pm

0.25 times the standard deviation.

Process variables	Unit	Description
U1	℃	Top temperature
U2	$\textrm{kg}\cdot\textrm{cm}^{-2}$	Top pressure
U3	$\textrm{m}^{3}\cdot\textrm{h}^{-1}$	Reflux flow rate
U4	$\textrm{m}^{3}\cdot\textrm{h}^{-1}$	Top distillate rate
U5	℃	Temperature of the 9th tray
U6	℃	Bottom temperature A

TABLE S1: The process variables in the DBC

Input Variables	Description
U1	Pressure of inlet gas
U2	Liquid level of buffer vessel
U3	Temperature of inlet barren liquor
U4	Flowrate of inlet lean solution
U5	Flowrate of inlet semi-lean solution
U6	Temperature of inlet gas

TABLE S2: Process variables for CAC

Process variables	Description
U1	High temperature bed temperature 1
U2	High temperature bed temperature 2
U3	High temperature bed temperature 3
U4	Outlet temperature of high temperature bed
U5	Outlet temperature of cooling water
U6	Split-gas temperature

TABLE S3: The process variables in the CSC process

Dataset	CAC		DCB		CSC
Hyper-parameters	$\mathcal{B}$	$\eta$	$\mathcal{B}$	$\eta$	$\mathcal{B}$	$\eta$
SNPLVR	128	0.01	128	0.01	128	0.01
DBPSFA	128	0.05	128	0.05	128	0.05
MUDVAE-SDVAE	32	0.005	32	0.005	32	0.005
GMM-VAE	128	0.0001	128	0.0001	128	0.0001
GSTAE	64	0.01	64	0.01	64	0.01

TABLE S4: Hyper-parameter Settings for Baseline Comparison Experiment

Model	DBC				CAC				CSC
Model	$\text{R}^{\text{2}}$	RMSE	MAE	MAPE	$\text{R}^{\text{2}}$	RMSE	MAE	MAPE	$\text{R}^{\text{2}}$	RMSE	MAE	MAPE
SNPLVR	1.39E-1	1.34E-2	1.10E-2	4.92E1	2.20E-2	7.06E-5	9.47E-5	3.92E-2	3.24E-1	7.85E-2	6.54E-2	3.35E-2
DBPSFA	9.13E-3	1.07E-3	7.68E-4	1.90E0	4.12E-4	1.42E-6	1.40E-6	5.16E-4	2.46E1	3.80E-2	3.80E-2	1.93E-2
MUDVAE-SDVAE	3.21E-2	3.24E-3	2.27E-3	2.42E0	4.57E-3	1.62E-5	1.38E-5	5.00E-3	5.39E-2	1.48E-2	1.28E-2	6.55E-3
GMM-VAE	2.02E-2	3.98E-3	3.05E-3	9.84E0	1.30E-2	5.61E-5	5.01E-5	1.66E-2	1.84E-2	1.16E-2	9.65E-3	4.90E-3
GSTAE	3.42E-3	1.92E-3	1.67E-3	5.44E0	3.05E-3	2.13E-5	3.35E-5	1.14E-2	2.09E-2	1.90E-2	1.42E-2	7.25E-3

TABLE S5: Standard Deviation Error for Baseline Model Comparison Experiment

Model	DBC			CAC			CSC
Model	Training (s)	Testing (s)	Memory (MB)	Training (s)	Testing (s)	Memory (MB)	Training (s)	Testing (s)	Memory (MB)
SNPLVR	60.835 ${}_{\pm\text{3.420E0}}$	0.016 ${}_{\pm\text{5.355E-4}}$	535	144.920 ${}_{\pm\text{7.871E0}}$	0.036 ${}_{\pm\text{4.506E-4}}$	535	159.527 ${}_{\pm\text{6.549E0}}$	0.041 ${}_{\pm\text{9.631E-4}}$	535
DBPSFA	86.948 ${}_{\pm\text{6.550E-1}}$	0.003 ${}_{\pm\text{8.059E-4}}$	579	182.223 ${}_{\pm\text{6.572E1}}$	0.003 ${}_{\pm\text{2.037E-3}}$	579	347.915 ${}_{\pm\text{8.032E-1}}$	0.045 ${}_{\pm\text{5.891E-2}}$	579
MUDVAE-SDVAE	75.004 ${}_{\pm\text{4.467E-1}}$	0.016 ${}_{\pm\text{5.822E-4}}$	535	160.677 ${}_{\pm\text{9.567E0}}$	0.038 ${}_{\pm\text{8.933E-4}}$	535	181.304 ${}_{\pm\text{6.022E0}}$	0.042 ${}_{\pm\text{5.934E-4}}$	535
GMM-VAE	49.870 ${}_{\pm\text{3.536E0}}$	0.026 ${}_{\pm\text{6.005E-4}}$	534	126.370 ${}_{\pm\text{3.685E0}}$	0.062 ${}_{\pm\text{8.111E-4}}$	534	143.606 ${}_{\pm\text{6.681E0}}$	0.073 ${}_{\pm\text{1.717E-3}}$	534
GSTAE	63.405 ${}_{\pm\text{1.247E0}}$	0.012 ${}_{\pm\text{1.631E-4}}$	535	151.798 ${}_{\pm\text{6.928E0}}$	0.030 ${}_{\pm\text{2.865E-4}}$	535	163.265 ${}_{\pm\text{2.299E0}}$	0.035 ${}_{\pm\text{2.552E-4}}$	535

TABLE S6: Empirical Spatial Temporal Complexity

Panel of Figure 17 — Fig. S4: Convergence analysis for generative network, .

Panel of Figure 18 — Fig. S5: Convergence analysis for inference network, .

Explore similar work

Structured Nonparametric Variational Inference for Dependent Latent Modeling
Jun 13, 2026Yuda Shao, Zhiling Gu, Shan YuLatent Variable Models Variational Inference
Department of Statistics, University of Virginia · Department of Biostatistics, Yale University
Fitting Large Nonlinear Mixed Effects Models Using Variational Expectation Maximization
Apr 28, 2026Mohamed Tarek, Pedro AfonsoExpectation-Maximization Variational Inference
PumasAI, USA
Amortized Structured Stochastic Variational Inference for Gaussian Process Latent Variable Models
Oct 2, 2026Maksym Tretiakov, Sarah Filippi, Vincent Fortuin +3Latent Variable Models Amortized Inference
LMU Munich · Munich Center for Machine Learning (MCML) · Imperial College London +3

Jun 13, 2026J/K move · Enter open · S save

Structured Nonparametric Variational Inference for Dependent Latent Modeling

Yuda Shao, Zhiling Gu, Shan Yu

Department of Statistics, University of Virginia · Department of Biostatistics, Yale University

Latent Variable Models Variational Inference

Open paper AskPDF arXiv

Variational inference (VI) is a core engine of modern AI, enabling scalable approximate Bayesian learning and uncertainty-aware training of large probabilistic and generative models. In this paper, we propose Structured Nonparametric Variational Inference (SN-VI), a novel framework for modeling complex dependencies among latent variables in posterior approximation, leveraging multivariate spline techniques. Unlike traditional methods that rely on the mean-field assumption, SN-VI preserves intricate latent variable dependencies, providing a flexible and accurate approximation of posteriors with arbitrary shapes. We establish rigorous theoretical guarantees, including the derivation of the lower bound for the variational objective and proof of asymptotic consistency in posterior estimation. To facilitate practical implementation, we develop an algorithm that automatically identifies dependent latent variables and their underlying dependence structure, without requiring manual specification. Simulation studies validate the effectiveness of SN-VI in approximating posterior distributions with bounded support and complex dependencies. The proposed method has been successfully applied to high-dimensional structured data, including computer vision datasets and spatial transcriptomics. In these applications, SN-VI demonstrates improved generative model performance and effectively uncovers coupled biological signals through the learned dependency structure.

Abstract

Explore similar work

Structured Nonparametric Variational Inference for Dependent Latent Modeling

Fitting Large Nonlinear Mixed Effects Models Using Variational Expectation Maximization

Amortized Structured Stochastic Variational Inference for Gaussian Process Latent Variable Models