Class-Conditional Conformal Prediction for Signal Temporal Logic Inference with Many Classes

Yixuan Wang; Danyang Li; Matthew Cleaveland; Roberto Tron; Mingyu Cai

cs.LGOct 9, 2026

Class-Conditional Conformal Prediction for Signal Temporal Logic Inference with Many Classes

Authors: Yixuan Wang, Danyang Li, Matthew Cleaveland, Roberto Tron, Mingyu Cai

Organizations: Mechanical Engineering, University of California, Riverside, CA 92521, USA · Mechanical Engineering, Boston University · MIT Lincoln Laboratory, Lexington, MA 02421, USA

Abstract

Signal Temporal Logic (STL) inference provides interpretable formulas for time series classification. For multiclass classification, however, accurate predictions do not necessarily ensure that the inferred formulas are satisfied consistently with the true class. We introduce the positive satisfaction rate and codeword satisfaction rate to quantify satisfaction of required formulas and consistency with the complete class codeword, respectively. We develop a two-stage procedure that first learns discriminative formulas using a margin-based objective and then refines predicate thresholds toward a small positive robustness target while fixing the formula structure and temporal intervals. Satisfaction and ranking penalties discourage violations of required formulas and loss of class separation. The decoding loss is used as a nonconformity score and calibrated separately for each class, providing finite-sample class-conditional coverage under intra-class exchangeability. Experiments on naval, synthetic, and robot manipulation trajectories demonstrate discrepancies between classification accuracy and formula satisfaction. The results further reveal a data-dependent tradeoff between robustness tightness and satisfaction, while class-conditional calibration mitigates undercoverage that can be hidden by pooled calibration.

Figures & tables

Fig. 1 : Overview of the proposed framework combining discriminative STL learning, tightness refinement, and class conditional conformal prediction.

Panel of Figure 2 — Fig. 2 : Conformal prediction with the refined models, standard against class conditional calibration. Rows: naval, synthetic, and CALVIN datasets. Columns: classwise coverage at $\alpha=0.10$ , coverage gap, and inefficiency across miscoverage levels. Coverage gap is the average absolute difference between the classwise coverage and $1-\alpha$ , and inefficiency is the average prediction set size.

Dataset	Model	Acc val	Acc test	PSR test	CSR test	$\bar{R}_{\mathrm{pos}}$
Naval	Discriminative	0.977	0.980	0.924	0.872	0.24
	Refined	0.960	0.960	0.873	0.797	0.13
	w/o margin	0.997	1.000	0.984	0.885	0.26
Synthetic	Discriminative	0.997	0.996	0.966	0.722	0.34
	Refined	0.995	0.995	0.944	0.695	0.33
	w/o margin	1.000	0.998	0.998	0.565	0.36

TABLE I : Accuracy, satisfaction rates, and mean positive robustness

\bar{R}_{\mathrm{pos}}

on the test split. Refinement reduces

\bar{R}_{\mathrm{pos}}

toward zero. The w/o margin rows remove the margin term of Eq. ( 9 ), reducing the codewise losses to sign losses.

Fig. 3 : Trajectory examples in the naval surveillance scenario. Vessels of class 0 arrive at the harbor without approaching the island, vessels of class 1 pass the island region on the way to the harbor, and vessels of class 2 turn back to the open sea. The formulas of Table II correspond to the regions and time windows that separate the three behaviors.

Class	Formula
0	$F_{[29,31]}(x<38.7)\wedge F_{[1,2]}(y<44.3)$
	$\;\wedge\;F_{[1,2]}(y>34.7)\wedge F_{[9,26]}(y>29.2)$
1	$G_{[5,6]}(x<55.1)\wedge G_{[43,45]}(y>26.6)$
2	$G_{[41,48]}(y<38.8)\wedge G_{[41,56]}(x>37.1)$

TABLE II : Learned STL formulas on the naval dataset after refinement, in the original coordinate frame. Vacuous and redundant atoms are pruned using the training and validation splits, on which the pruned formulas agree with the full model by construction; on the test split they agree on all but one sample.

Fig. 4 : Trajectory examples in Synthetic Dataset.

Class	Formula
0	$F_{[33,35]}(y<-2.0)\wedge G_{[13,63]}(x<3.1)$
1	$G_{[17,19]}(x>3.5)$
2	$F_{[69,70]}(y<-1.1)$
3	$F_{[25,26]}(-3.6<x<2.6)\wedge G_{[75,76]}(x>0.9)$
4	$F_{[32,36]}(x<1.6)\wedge G_{[62,63]}(y<0.1)$
5	$F_{[22,26]}(x>-4.1)\wedge G_{[38,39]}(x<0.0)$

TABLE III : Learned STL formulas on the synthetic dataset after refinement, in the generator coordinate frame. Same pruning convention as Table II .

Fig. 5 : One trajectory per task of CALVIN Dataset.

Class	Formula
0	$F_{[62,63]}(y<-35.1)$
1	$G_{[31,36]}(y<-34.0)$
2	$F_{[38,42]}(x>-11.3\wedge y>-2.6\wedge g>1.9)$
	$\;\wedge\;F_{[20,27]}(g>4.7)\wedge G_{[7,51]}(x<23.6\wedge y<3.5)$
3	$F_{[60,63]}(x>-16.9)\wedge G_{[33,39]}(x<-5.0\wedge y>-6.1)$
	$\;\wedge\;G_{[5,38]}(g>-0.1)$

TABLE IV : Learned STL formulas on the CALVIN dataset after refinement. Same pruning convention as Table II . Here

x

y

z

are the end effector position and

g

the gripper width, in centimeters.

Dataset	Acc network	Acc exact	Sign disagreement
Naval	0.960	0.980	0/753
Synthetic	0.995	0.964	0/15337
CALVIN	0.895	0.871	0/5472

TABLE V : Network and exact STL semantics on the test split for the selected refined models. Robustness signs agree on every code entry, so PSR and CSR coincide under both semantics; accuracies differ because decoding compares loss magnitudes.

Configuration	$\eta_{\mathrm{tight}}$	$\eta_{\mathrm{sign}}$	$\gamma$	Acc	PSR	CSR	$\bar{R}_{\mathrm{pos}}$
Baseline	1.0	5	0.01	0.976	0.757	0.729	0.105
$\eta_{\mathrm{tight}}\times 0.5$	0.5	5	0.01	0.976	0.705	0.661	0.097
$\eta_{\mathrm{sign}}\times 2$	1.0	10	0.01	0.960	0.853	0.793	0.124
$\eta_{\mathrm{sign}}\times 4$	1.0	20	0.01	0.960	0.873	0.797	0.128
$\gamma\times 2$	1.0	5	0.02	0.976	0.809	0.777	0.116

TABLE VI : Stage 2 sweep on the naval dataset, all metrics on the test split. The baseline row is the configuration used on the synthetic and CALVIN datasets, and the

\eta_{\mathrm{sign}}\times 4

row is the reported operating point.

Explore similar work

Conformalized Signal Temporal Logic Inference under Covariate Shift
Mar 28, 2026Yixuan Wang, Danyang Li, Matthew Cleaveland +2Signal Temporal Logic Conformal Prediction
Mingyu Cai and Yixuan Wang are with Mechanical Engineering, University of California, Riverside, CA, 92521, USA. · Danyang Li and Roberto Tron are with Mechanical Engineering, Boston University. · Matthew Cleaveland is with MIT Lincoln Laboratory, Lexington, MA, 02421, USA
Distilling Formal Logic into Neural Spaces: A Kernel Alignment Approach for Signal Temporal Logic
Mar 5, 2026Sara Candussio, Gabriele Sarti, Gaia Saveri +1Neuro-Symbolic Reasoning Signal Temporal Logic
AILab, MIGe, University of Trieste, IT · Khoury College of Computing Sciences, Northeastern University, Boston, MA, USA
ReasonSTL: Bridging Natural Language and Signal Temporal Logic via Tool-Augmented Process-Rewarded Learning
May 7, 2026Bowen Ye, Zhijian Li, Junyue Huang +2Signal Temporal Logic LLM Tool Use
Shanghai Jiao Tong University · Alibaba Group, Hangzhou, China

Mar 28, 2026J/K move · Enter open · S save

Conformalized Signal Temporal Logic Inference under Covariate Shift

Yixuan Wang, Danyang Li, Matthew Cleaveland, Roberto Tron+1

Mingyu Cai and Yixuan Wang are with Mechanical Engineering, University of California, Riverside, CA, 92521, USA. · Danyang Li and Roberto Tron are with Mechanical Engineering, Boston University. · Matthew Cleaveland is with MIT Lincoln Laboratory, Lexington, MA, 02421, USA

Signal Temporal Logic Conformal Prediction Covariate Shift Weighted Conformal Prediction

Open paper AskPDF arXiv

Signal Temporal Logic (STL) inference learns interpretable logical rules for temporal behaviors in dynamical systems. To ensure the correctness of learned STL formulas, recent approaches have incorporated conformal prediction as a statistical tool for uncertainty quantification. However, most existing methods rely on the assumption that calibration and testing data are identically distributed and exchangeable, an assumption that is frequently violated in real-world settings. This paper proposes a conformalized STL inference framework that explicitly addresses covariate shift between training and deployment trajectories dataset. From a technical standpoint, the approach first employs a template-free, differentiable STL inference method to learn an initial model, and subsequently refines it using a limited deployment side dataset to promote distribution alignment. To provide validity guarantees under distribution shift, the framework estimates the likelihood ratio between training and deployment distributions and integrates it into an STL-robustness-based weighted conformal prediction scheme. Experimental results on trajectory datasets demonstrate that the proposed framework preserves the interpretability of STL formulas while significantly improving symbolic learning reliability at deployment time.

Abstract

Explore similar work

Conformalized Signal Temporal Logic Inference under Covariate Shift

Distilling Formal Logic into Neural Spaces: A Kernel Alignment Approach for Signal Temporal Logic

ReasonSTL: Bridging Natural Language and Signal Temporal Logic via Tool-Augmented Process-Rewarded Learning