Learning to Predict, Discover, and Reason in High-Cardinality Event Sequences

Hugo Math

cs.AIMar 17, 2026

Learning to Predict, Discover, and Reason in High-Cardinality Event Sequences

Authors: Hugo Math

Organizations: Faculty of Applied Computer Science, University of Augsburg

Abstract

Electronic control units (ECUs) embedded within modern vehicles generate a large number of asynchronous events known as diagnostic trouble codes (DTCs). These discrete events form complex temporal sequences that reflect the evolving health of the vehicle's subsystems. In the automotive industry, domain experts manually group these codes into higher-level error patterns (EPs) using Boolean rules to characterize system faults and ensure safety. However, as vehicle complexity grows, this manual process becomes increasingly costly, error-prone, and difficult to scale. Notably, the number of unique DTCs in a modern vehicle is on the same order of magnitude as the vocabulary of a natural language, often numbering in the tens of thousands. This observation motivates a paradigm shift: treating diagnostic sequences as a language that can be modeled, predicted, and ultimately explained. Traditional statistical approaches fail to capture the rich dependencies and do not scale to high-cardinality datasets characterized by thousands of nodes, large sample sizes, and long sequence lengths. Specifically, the high cardinality of categorical event spaces in industrial logs poses a significant challenge, necessitating new machine learning architectures tailored to such event-driven systems. This thesis addresses automated fault diagnostics by unifying event sequence modeling, causal discovery, and large language models (LLMs) into a coherent framework for event streams. It is structured in three parts, reflecting a progressive transition from prediction to causal understanding and finally to reasoning for vehicle diagnostics. Consequently, we introduce several Transformer-based architectures for predictive maintenance, scalable sample- and population-level causal discovery frameworks and a multi-agent system that automates the synthesis of Boolean EP rules.

Figures & tables

Figure 1: Error Pattern Prediction (when and what) . Based on the past sequence $S$ of diagnostic trouble codes (DTCs), manufacturers want to prevent error patterns (EPs) from happening by predicting their likelihood and time of occurrence.

Figure 2: Error Pattern Prediction using Multimodal Sequences . Past diagnostic trouble codes only provide limited information about complex and overlapping error patterns. In addition, domain experts rely on environmental conditions (e.g., temperature, voltage, humidity, …) to identify more accurately error patterns.

Figure 3: Anonymized Causal Graph. Example of a sequence of events ( DTCs ) that lead to a steering wheel degradation and a power limitation as outcome labels . The causal indicators are shown in violet if inhibitory and orange or red if excitatory, depending on the magnitude. The voltage error DTC is the biggest causal driver for the power limitation EP while the security sync failure seems to have the biggest inhibitory effect on all EPs.

Figure 4: Anonymized Sample Time Causal Graph for Diagnostic Defect Cascade. Temporal evolution of a diagnostic defect cascade in a vehicle (

|\mathcal{X}|\approx 29,100

). TRACE effectively captures causal relationships, revealing distinct error clusters at different time steps (e.g., initial sensor failures at

t=3

triggering mechanical faults at

t=12

, battery at issue

t=17

). This enables actionable root-cause analysis by isolating the specific onset of a failure mechanism and its strength using the conditional mutual information.

Causal Scope	Events-to-Outcome (DTCs $\rightarrow$ Error Pattern)	Event-to-Event (DTCs $\rightarrow$ DTCs)
Sample-Level (Single Sequence)	OSCAR One-Shot Multi-Label Causal Discovery Markov Boundary per label from a single sequence Chapter 7	TRACE Scalable Amortized Causal Discovery Sample time causal graph from a single sequence Chapter 9
Population-Level (Full Dataset)	CARGO Graph Aggregation for Causal Discovery Global Markov Boundaries via adaptive graph fusion Chapter 8	Open Problem Population-level summary causal graph over event types/outcomes Requires aggregation of sample time and summary graphs with acyclicity constraints Future work

Table 1: The four regimes of causal discovery in discrete event sequences , defined by the cross-product of causal scope (single-sequence inference vs. population-level aggregation) and dependency type (events-to-outcome vs. event-to-event). The three shaded contributions of this thesis — OSCAR , TRACE , and CARGO — share a common backbone: pretrained autoregressive Transformers repurposed as neural autoregressive density estimators (NADEs) for conditional mutual information estimation. The remaining cell, population-level event-to-event discovery, constitutes an open problem and a direct avenue for future work.

Notation	Description
$x$	Discrete event type, in our case a DTC.
$d$	Absolute mileage of the vehicle in km.
$m$	Mileage of the vehicle in km since the first DTC ( $x_{0}$ ) occurred in a sequence $S$ such as $m_{i}=d_{i}-d_{0}$
$ts$	Unix timestamp attached to each DTC.
$t$	Number of hours passed since the first DTC occurred in a sequence. More specifically, $t_{i}$ is defined as $t_{i}=ts_{i}-ts_{0}$ .
$S$	Sequence of triplets (event type, time, mileage) defined as $S=\{(x_{i},t_{i},m_{i})\}_{i=0}^{L}$ of length $L$ with index starting from 0.

Table 1: List of dataset symbols and their respective meanings for the datasets used in this thesis.

Data	# of values	Description
DTC	8710	Diagnostic Trouble Code
ECU	61	Electronic Control Unit
Base-DTC	7726	Error Code
Fault-Byte	2	Binary Value

Table 1: Number of Distinct values of the DTC elements

Figure 1: Time and Mileage Distribution . Distribution of $t_{i}$ and $m_{i}$ (relative time and mileage) in our dataset.

Panel of Figure 2 — Figure 2: CarFormer Architecture . A decoder-only Transformer with three predictive heads learns the dynamics of the DTC sequences. In every layer, the context embedding $\mathbf{CE}=\mathbf{T}+\mathbf{M}$ (time and mileage) is added to the projected query and key, and the RoPE rotation $\mathbf{R}^{d}_{\Theta}$ is applied afterwards, i.e., $\mathbf{Q}=\mathbf{R}^{d}_{\Theta}(\mathbf{W}^{Q}\mathbf{E}+\mathbf{CE})$ and $\mathbf{K}=\mathbf{R}^{d}_{\Theta}(\mathbf{W}^{K}\mathbf{E}+\mathbf{CE})$ (Section 1 ). The values are not modified.

Panel of Figure 3 — Figure 3: EPredictor Architecture . The first layer processes the normalized event-type embedding $\mathbf{E}$ output by CarFormer. The second layer takes its query from the first layer, and its key and value from the hidden state $H(S)$ extracted from CarFormer. As in CarFormer, the context embedding $\mathbf{CE}$ carrying the spatio-temporal information is added to the projected keys and queries of the second layer before the RoPE rotation. For a trajectory $S^{\prime}$ longer than the minimum context $c$ , the two heads jointly output the EP probabilities $P(\mathrm{EP}\mid S^{\prime})$ and the predicted time of occurrence $\hat{t}_{\mathrm{EP}}$ ; the same prediction is then repeated on the next trajectory $S^{\prime\prime}$ , and so on.

Model	ACC(%)	MAPE(%)	RMSE
rot-ce	22.64	3.2	0.04770
time	21.48	2.9	0.04762
time-mileage	21.38	3.0	0.04785
time-c2m-m2c	21.58	3.5	0.04794
time-m2c	21.52	3.6	0.04823
GPT	19.89	-	-

Table 2: Overall prediction performance of CarFormer with different embeddings. Best results are in bold.

Model	F1	MAPE	MAE	$\textbf{CPMW}_{F1}\uparrow$	$\textbf{CPMW}_{MAE}\downarrow$
rotcross-query-key-ce-1-2	82.69	32.45	0.0268	52.80	0.888
rotcross-query-key-ce-2	82.69	31.18	0.0254	56.61	0.882
rotcross-query-ce-2	82.69	31.18	0.0254	53.00	0.884
rotcross-key-value-ce-2	84.38	33.47	0.0263	65.06	0.904
rotcross-key-value-scaled-ce-2	84.38	31.44	0.0252	67.63	0.874
rotnocross-ce-1-2	80.74	37.61	0.0275	42.95	0.927

Table 3: EPredictor evaluation results with different model architectures on the test set (no up- nor down-sampling). The MAPE and F1 are shown in (%)

Figure 4: Classification Performance Comparison Between Architectural Changes . F1 Score comparison with multiple EPredictor architectures as a function of the number of observations. This answers what EP is most likely to occur.

Figure 5: Regression Error As a Function of the Number of Observations . Evolution of the MAE as a function of the number of observations for the best performing model, this effectively tests for when an EP is most likely to occur.

Notations	Description
$S_{e}$	Sequence of environmental conditions triplets defined as
	$S_{e}=\{(d_{i},v_{i},c_{i})\}_{i=0}^{L_{e}}$ of length $L_{e}\gg L$
$d$	Environmental condition description, e.g., temperature
	increase, vehicle speed, pressure increase
$v$	Environmental condition value can be int, float, string, null
$c$	Environmental condition unit, e.g., A, V, bar, °C, sec

Table 1: New symbols introduced in this chapter. For base DTC notation, see Table 1

Data	# of values	Description
DTC	22,137	Diagnostic Trouble Code
ECU	132	Electronic Control Unit
Base-DTC	17,044	Error Code
Fault-Byte	2	Binary Value
E. Condition Description	2,559	(See Table 1: $d$ )
E. Condition Value	3,288	(See Table 1: $v$ )

Table 2: Number of distinct values after filtering for each feature.

Figure 1: Spatio-temporal Point Process Representation of a Vehicle. Spatial (mileage) and Temporal (time) point process representation of events from a vehicle. Bold vertical lines indicate multiple events occurring at the same time

t_{i}

or mileage

m_{i}

Figure 2: BiCarFormer Architecture . Both parallel Transformers are computing cross-attention scores conditioned on each modality

\boldsymbol{Q,K,V}

. Two final representations are generated for each modality: DTC (

\boldsymbol{H}_{d})

and environmental conditions (

\boldsymbol{H}_{e}

). Multiple hierarchical embeddings are defined at the input level to account for DTC-specific features (ECUs, Fault-Bytes, Base-Code).

Micro and Macro Averaging
Model	AUROC (Micro)	F1 Score (Micro)	F1 Score (Macro)
BiCarFormer	0.809	0.77	0.71
DTC-TranGRU [ 50 ]	0.602	0.36	0.28
BERT [ 27 ]	0.768	0.71	0.63
Sample Averaging
Model	Precision (Sample)	Recall (Sample)	F1 Score (Sample)

Table 3: Downstream evaluation of multi-label error pattern classification. Each model has the same number of parameters (25M). Results are grouped by averaging type (micro/macro vs. sample).

Figure 3: Attention Distribution between Environmental Condition Tokens. Amount of attention received by each environmental condition token from the DTCs. The y-axis was truncated to improve clarity as well as the number of heads printed. We take

\boldsymbol{A}_{dtc\rightarrow env}

of the last layer.

Figure 4: Cross-Attention Scores for $\boldsymbol{A}_{dtc\rightarrow env}$ . The DTCs are shown on the y-axis (anonymized), and the environmental conditions with their 3 elements $(d,v,u)$ concatenated are shown on the x-axis (the description $d$ is anonymized). The intensity of each cell reflects the attention weight, where darker shades indicate higher attention values.

Figure 5: Attention Distribution between Environmental Conditional Units. Amount of attention received by each environmental unit from the DTCs in a battery aging error pattern from a battery electric vehicle. We take

\boldsymbol{A}_{dtc\rightarrow env}

of the last layer and print all heads.

Figure 6: Discretized Voltage and Attention Scores Evolution between Environmental Token Position . Discretized voltage variation as a function of environmental condition tokens and the associated amount of attention received from DTCs. We only take triplets

(d,v,u)

from

S_{e}

with

u=

’v’ and extract

\boldsymbol{A}_{dtc\rightarrow env}

from the last layer and head 2 . The sample is taken from a battery aging error pattern. BiCarFormer implicitly learns the fluctuation of discretized continuous values to make a prediction.

Panel of Figure 1 — Figure 1: The Overview of OSCAR : O ne- S hot multi-label C ausal A utoRegressive discovery. Blue: the two pretrained Transformers $\text{Tf}_{x}$ (CarFormer) and $\text{Tf}_{y}$ (EPredictor); green: operations vectorized across the $N$ sampled trajectories $x^{(l)}_{<t}$ . Steps (1)–(5) are described in Section 3 . An edge $X_{t}\rightarrow Y_{j}$ is kept if $\hat{I}_{N}>\tau_{j}$ and is annotated with the causal indicator $\mathcal{C}$ (Eq. ( 15 )) and its standard deviation $\sigma$ ; $\widehat{\mathcal{MB}}(Y_{1}),\widehat{\mathcal{MB}}(Y_{2})$ are the resulting estimated Markov Boundaries of $Y_{1},Y_{2}$ .

Algorithm	Precision ↑	Recall ↑	F1 ↑	Running Time (min) ↓
IAMB	-	-	-	$>4320$
CMB	-	-	-	$>4320$
MB-by-MB	-	-	-	$>4320$
PCDbyPCD	-	-	-	$>4320$
MI-MCF	-	-	-	$>4320$
OSCAR	$\mathbf{55.26\pm 1.42}$	$\mathbf{31.37\pm 0.82}$	$\mathbf{40.02\pm 1.03}$	$\boldsymbol{11.7}$

Table 1: Comparisons of

\mathcal{MB}

retrieval with

n=50,000

samples,

|\mathcal{X}|=29,100,|\mathcal{Y}|=474

averaged over

6-

folds. Classification metrics averaging is ’weighted’ and shown as sample-level for OSCAR. The symbol ’-’ indicates that the algorithm did not output the

\mathcal{MB}

within 3 days. Metrics are given in

\%

Algorithm	Precision ↑	Recall ↑	F1 ↑	Running Time (min) ↓
IAMB	$0.0\pm 0.0$	$0.0\pm 0.0$	$0.0\pm 0.0$	$129.4$
CMB	$0.0\pm 0.0$	$0.0\pm 0.0$	$0.0\pm 0.0$	$128.7$
PCDbyPCD	$0.0\pm 0.0$	$0.0\pm 0.0$	$0.0\pm 0.0$	$129.1$
MB-by-MB	$0.0\pm 0.0$	$0.0\pm 0.0$	$0.0\pm 0.0$	$140.3$
MI-MCF	$0.0\pm 0.0$	$0.0\pm 0.0$	$0.0\pm 0.0$	$>1440$
OSCAR	$\mathbf{54.78\pm 2.91}$	$\mathbf{30.39\pm 2.39}$	$\mathbf{39.92\pm 2.25}$	$\mathbf{0.14}$

Table 2: Comparisons of

\mathcal{MB}

retrieval with

n=500

samples over

9

folds.

Figure 3: Evolution of the Classification Performance as a Function of the Markov Boundary Length

|\mathcal{MB}(Y_{j})|

(

n=45969

samples). We can identify that the Markov Boundary complexity is not the bottleneck but rather the number of samples per class ( imbalance threshold ), which reduces classification performance.

Figure 3: Adaptive Thresholding Function

\tau_{j}(m_{j})

across varying support

m_{j}

, illustrating the logistic decay from

\tau_{\max}

\tau_{\min}

Figure 4: Criterion Ablation for Structural Fusion . Comparison of different criteria for the structural fusion (Phase 2) as a function of the number of samples

m

. With

|\mathcal{Y}|=474,|\mathcal{X}|=29,100

nodes.

Method	SHD ( $\downarrow$ )	F1 ( $\uparrow$ )	Precision ( $\uparrow$ )
Random	218.8 $\pm$ 3.2	0.01 $\pm$ 0.00	0.04 $\pm$ 0.01
Frequency	723.0 $\pm$ 6.7	0.09 $\pm$ 0.00	0.06 $\pm$ 0.00
Attention (BERT)	321.0 $\pm$ 15	0.50 $\pm$ 0.01	0.35 $\pm$ 0.01
Saliency (Input x Gradient LLaMA)	160.2 $\pm$ 6.55	0.67 $\pm$ 0.01	0.51 $\pm$ 0.01
Shapley Value Sampling (LLaMA)	148.0 $\pm$ 5.09	0.60 $\pm$ 0.01	0.55 $\pm$ 0.01
Neural Granger (LLaMA)	100.2 $\pm$ 14.6	0.69 $\pm$ 0.04	0.71 $\pm$ 0.04

Table 2: Identifiability Comparison. Comparison of causal discovery performance on synthetic SCMs with

|\mathcal{X}|=1000,L=64,\epsilon=0.05,\tau=3.10^{-5},N=128

. TRACE significantly outperforms local (Saliency) and global (Granger/Shapley) baselines, achieving over 20 points higher F1 while maintaining high precision. Results across 10 runs are reported.

Figure 4: Scalability to High-Cardinality Event Spaces. Evaluation of structural identifiability across exponentially growing vocabulary sizes. Top: Evolution of discovery metrics. TRACE exhibits performance invariance , maintaining F1

\approx 0.81

even as the combinatorial search space explodes. Bottom: Visual examples of recovered summary graphs

\mathcal{G}_{s}

at scale. The Shannon redundancy Pred =

1-H(P)/H_{max}

(0 for uniform noise, 1 for a deterministic process) confirms that TRACE succeeds even in high-entropy regimes. (

\hat{\epsilon}=0.01,L=64,N=64,\tau=10^{-4}

Figure 5: Robustness and Scalability Analysis (

|\mathcal{X}|=1000,N=128,\tau=10^{-4},L=64

). Evolution of causal discovery performance (F1, Precision, Recall, SHD). (a) Robustness to Generative Error: Performance as a function of the model’s oracle score

\epsilon

. TRACE exhibits a phase transition, recovering structure even for imperfect models (

\epsilon<0.1

) and maintaining high Precision even as fidelity degrades. (b) Scalability to Length: Performance and GPU memory usage vs. sequence length

L

. The Sparse variant demonstrates linear memory scaling (

O(mL)

), enabling inference on sequences far exceeding the training length (

L=64

), whereas the Full variant scales quadratically. (c) Long-Range Dependencies: Robustness to increasing delayed-effects

m

. TRACE maintains F1

>0.8

even as dependencies span one third of the sequence (

m=20

), confirming the method’s ability to capture distant causal mechanisms.

Figure 6: Sample Time Causal Graph. Temporal evolution of a diagnostic defect cascade in a vehicle (

|\mathcal{X}|\approx 29,100

). TRACE effectively captures causal relationships, revealing distinct error clusters at different time steps (e.g., initial sensor failures at

t=3

triggering mechanical faults at

t=12

, battery at issue

t=17

). This enables actionable root-cause analysis by isolating the specific onset of a failure mechanism and their strength using the CMI

\hat{I}_{N}

Figure 7: Robustness to Hidden Confounding. Evaluation of TRACE under violations of causal sufficiency. (a) Measurement Error: Random noise is injected into the trajectory. Precision stays high, indicating resistance to hallucination. (b) Temporal Drops: Time steps are randomly deleted, thus conditioning sets are broken. TRACE still recovers structure despite missing intermediaries but the discovery scores quickly decrease.

Figure 1: Illustration of Error Pattern Automation in Modern Vehicles . (a) A vehicle with an unknown defect generates a sequence of Diagnostic Trouble Codes (DTCs) over time. (b) A domain expert analyzes the DTCs and the vehicle’s metadata (e.g., model, descriptions). (c) Provide a Boolean rule to identify this error pattern.

$x_{1}$	$x_{2}$	Ground Truth ( $x_{1}\;\&\;x_{2}$ )	Predicted Rule ( $x_{1}\;\|\;x_{2}$ )
0	0	0	0
0	1	0	1
1	0	0	1
1	1	1	1

Table 1: Illustrative truth table for semantic evaluation. The accuracy using the predicted rule is

50\%

Figure 3: Analysis of Ground Truth Error Patterns. (Left) Complexity Distribution by Fold: The consistency of rule complexity scores across fold confirms a balanced data split strategy. (Middle) Rule Length Histogram: The distribution of Diagnostic Trouble Codes (DTCs) per rule reveals a long-tail nature, with some complex rules involving over 20 unique codes. (Right) Operator Frequency: The significant presence of logical NOT (

\neg

) and OR (

\vee

) operators highlights the non-trivial, inhibitory causal structures.

Figure 4: Performance Comparison of CAREP Against Standalone LLMs . (a) Semantic evaluation represents how well the estimated error pattern rules fit the ground truth in terms of Boolean expression. (b) Structural evaluation or multi-label classification reveals if the estimated rules contain the correct DTCs.

Method	Precision $\uparrow$	Recall $\uparrow$	F1 $\uparrow$	Time (min) $\downarrow$
Phase 1: Sample-Level Causal Discovery (OSCAR)
Sample-Level Graph $\mathcal{G}^{(k)}$	$55\pm 1.4$	$31\pm 0.8$	$40\pm 1.0$	$\mathbf{11.7}$
Phase 2: Population-Level Causal Discovery (CARGO)
Consensus Graph $\mathcal{G}^{*}_{y}$	$61\pm 1.5$	$46\pm 1.7$	$46\pm 1.2$	$11.8$
Phase 3: CAREP (End-to-End)
CAREP	$\mathbf{83\pm 1.1}$	$\mathbf{60\pm 1.3}$	$\mathbf{61\pm 1.1}$	$12$

Table 2: Ablation of causal discovery performance across the pipeline stages : (1) Local Discovery, (2) Global Aggregation, and (3) The complete framework (CAREP). The baselines are evaluated using the structural (does the rule contain the correct variables, ignoring the logic). ’Time’ represents cumulative runtime. By adding semantic information (metadata) and LLMs’ reasoning, CAREP adds a significant prediction margin in selecting the correct causes of an outcome (EP).

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 1: Confident Predictive Maintenance Window Evolution with and w/o the random head.

Figure 2: Pretraining Loss Comparison . Base-DTC classification loss comparison with and without multimodal learning.

Tokens	Param.	Context	Prec. (↑)	Rec. (↑)	F1 (↑)	Tfy F1 (↑)
For $n=50{,}000$ samples
1.5B	105m	$c=4$	$47.95\pm 1.05$	$30.65\pm 0.51$	$37.39\pm 0.67$	88.6
1.5B	105m	$c=12$	$54.62\pm 1.03$	$29.88\pm 0.73$	$38.63\pm 0.85$	90.43
1.5B	105m	$c=15$	$\mathbf{55.26\pm 1.42}$	$\mathbf{31.37\pm 0.82}$	$\mathbf{40.02\pm 1.03}$	90.57
1.5B	105m	$c=20$	$49.52\pm 1.59$	$\mathbf{31.76\pm 0.85}$	$36.54\pm 1.10$	91.19
1.5B	105m	$c=30$	$36.65\pm 1.18$	$22.75\pm 0.78$	$26.57\pm 0.91$	92.64

Appendix

Table 1: Ablations of the performance of Phase 1 (Sample-Level Markov Boundary retrieval) as a function of different NADEs with

n=50{,}000

and

n=500

samples, averaged over 5-folds. Classification metrics use weighted averaging. Metrics are given in

\%

Proposal	F1 Micro (%)	F1 Macro (%)	F1 Wtd. (%)	Time (s)
w/o Sampling	$14.07$	$12.29$	$16.67$	$\mathbf{49.30\pm 0.30}$
Permutation	$18.22\pm 0.36$	$13.75\pm 0.09$	$19.21\pm 0.03$	$557.82\pm 0.13$
Top-k=20	$26.77\pm 0.71$	$23.83\pm 0.19$	$29.25\pm 0.07$	$557.40\pm 0.13$
Top-k=35	$26.57\pm 0.96$	$24.08\pm 0.23$	$29.30\pm 0.07$	$557.35\pm 0.10$
Top-k=35+T=0.8	$27.36\pm 0.65$	$23.77\pm 0.21$	$28.98\pm 0.07$	$557.45\pm 0.11$
Top-k=35+T=1.2	$26.59\pm 1.49$	$\mathbf{24.62\pm 0.29}$	$\mathbf{29.52\pm 0.06}$	$557.45\pm 0.12$

Appendix

Table 2: Sample-level classification performance and elapsed time (sec) of OSCAR across different sampling methods (

k

: top-

k

size,

p

: nucleus probability mass,

T

: temperature), mean and standard deviation over 10 runs ( w/o Sampling is deterministic). Best results are in bold .

Figure 1: Ablation of the Number of Sampled Trajectories

N

. Results are reported using 1-sigma error bar.

Figure 2: Sensitivity Analysis . Evolution of the sample-level F1 Score, Precision, and Recall as a function of coefficient

k

. Results are reported using 1-sigma error bar.

Figure 5: Sample Summary Causal Graph

\mathcal{G}_{s}

of the vehicle sequence shown in Fig. 6 , obtained by projecting its sample time edges onto event types (Def. 23 ). Nodes: DTC types occurring in the sequence. Red edges: strong causal relations, annotated with their estimated CMI

\hat{I}_{N}

, line width proportional to

\hat{I}_{N}

. Blue edges: weaker relations retained by TRACE (grey in Fig. 6 ). Event types that recur in the sequence (e.g., FAS Signal Error ) are merged into a single node and become hubs.

Fault Diagnosis

Fault diagnosis identifies, localizes, and explains failures in engineered and computational systems using sensor data, logs, models, and domain knowledge.

See all 147 papers

Sep 25, 2026

seq2cause: One Autoregressive Backbone, Four Causal Discovery Tasks in Event Sequences

Complex systems such as vehicles, patients, or genomes emit discrete event sequences whose operative question is causal, not predictive: which events cause which other events, and which cause higher-level outcomes such as failures or diseases? This question decomposes along two axes -- dependency type (event

\to

event vs.\ event

\to

outcome) and causal scope (single sequence vs.\ population) -- yielding four structurally distinct regimes with different identifiability conditions. No existing method addresses more than one, because all assume multi-stream structure with low vocabulary, and none scales beyond a few hundred event types. We present Seq2Cause, a unified framework that resolves all four regimes through a single shared primitive: a pretrained autoregressive model repurposed as an amortized conditional independence testing engine requiring no task-specific retraining. We establish a prediction--causality duality: the model's excess cross-entropy simultaneously bounds causal identification error across all four regimes, so that every improvement in next-token prediction tightens causal guarantees for free. On nonlinear SCMs (vocabularies up to

8{,}000

types) and real-world vehicle diagnostic logs (

29

K event types,

474

failure outcomes), Seq2Cause is the first method to populate all four regimes at scale with a single frozen backbone. Existing methods are either inapplicable, inaccurate, or computationally intractable in this setting.

Hugo Math

BMW Group

Causal Discovery

Ask PDF

Mar 9, 2026

S2S-FDD: Bridging Industrial Time Series and Natural Language for Explainable Zero-shot Fault Diagnosis

Fault diagnosis is critical for the safe operation of industrial systems. Conventional diagnosis models typically produce abstract outputs such as anomaly scores or fault categories, failing to answer critical operational questions like "Why" or "How to repair". While large language models (LLMs) offer strong generalization and reasoning abilities, their training on discrete textual corpora creates a semantic gap when processing high-dimensional, temporal industrial signals. To address this challenge, we propose a Signals-to-Semantics fault diagnosis (S2S-FDD) framework that bridges high-dimensional sensor signals with natural language semantics through two key innovations: We first design a Signal-to-Semantic operator to convert abstract time-series signals into natural language summaries, capturing trends, periodicity, and deviations. Based on the descriptions, we design a multi-turn tree-structured diagnosis method to perform fault diagnosis by referencing historical maintenance documents and dynamically querying additional signals. The framework further supports human-in-the-loop feedback for continuous refinement. Experiments on the multiphase flow process show the feasibility and effectiveness of the proposed method for explainable zero-shot fault diagnosis.

Baoxue Li, Chunhui Zhao

State Key Laboratory of Industrial Control Technology, College of Control Science and Engineering, Zhejiang University, Hangzhou, China · School of Information and Electrical Engineering, Zhejiang University City College, Hangzhou, China

Fault Diagnosis Industrial Fault Diagnosis

Ask PDF

Sep 30, 2026

Inferring Causal Relations between Two Sequences of Events with Language Models

Causal AI is a branch of Artificial Intelligence which helps understand and reason about cause and effect relationships, not just patterns or correlations. Causal discovery aims to infer elements of the underlying causal structure--often represented as a directed graph--from observational and, when available, interventional data. While causal discovery is the fundamental step for moving beyond mere associations toward genuine understanding, and thus the basic building block of causal AI, it becomes intrinsically difficult when causal relations must be inferred from single observations. In such situations, standard causal discovery methods cannot be used and one has to identify causal relations from limited amount of information. This is typically the case for, e.g., sequences of events produced by different alarms which need to be analyzed on the fly to detect abnormal phenomena, which are usually rare. We show in this study that it is possible to leverage the predictive power of Large Language Models (LLMs) to infer causal relations between only two sequences of events. This approach, which is validated on both synthetic and real data, provides better results than standard causal discovery algorithms on several time series data, even though these data were converted into smaller, single observed sequences.

Nishchal Prasad, Eric Gaussier, Emilie Devijver +3

INRIA, Univ. Grenoble Alpes, CNRS, LIG · Univ. Grenoble Alpes, CNRS, Grenoble INP, LIG · Univ. Grenoble Alpes, CNRS, Inria, Grenoble INP, LIG +1

Time-Series Causal Discovery Causal Discovery Causal Reasoning in Language Models

Ask PDF

Abstract

Fault Diagnosis

Explore similar work

seq2cause: One Autoregressive Backbone, Four Causal Discovery Tasks in Event Sequences

S2S-FDD: Bridging Industrial Time Series and Natural Language for Explainable Zero-shot Fault Diagnosis

Inferring Causal Relations between Two Sequences of Events with Language Models