D-TAIA: Domain-Aware LLM Adaptation for Multi-Task Predictive Process Monitoring
Authors: Sjoerd van Straten, Christine Jacob, Marwan Hassani
Organizations: Department of Mathematics and Computer Science, Eindhoven University of Technology, The Netherlands · Department of Mathematics and Computer Science, University of Wuppertal, Germany
Predictive Process Monitoring (PPM) enables organizations to forecast future process behavior, such as the next activity and remaining time of ongoing cases. In practice, three conditions cause existing methods to degrade, namely data scarcity, high process entropy and distributional shift. While Foundation Models (FMs), especially Large Language Models (LLMs), offer a new paradigm through broad sequential reasoning, adapting them to multi-task PPM under these conditions remains an open challenge. Existing FM-based approaches either lack mechanisms for handling distributional shift or rely on direct regression heads that can be structurally misaligned with continuous time prediction tasks. This paper introduces D-TAIA (Domain-aware Training and Attention-based Inference Architecture), a framework for a joint next activity and remaining time prediction task via parameter-efficient fine-tuning of an FM backbone. Our approach combines domain-aware triplet loss (DATL) pre-training with FAISS-based nearest neighbor retrieval for remaining time prediction, and adopts the TAIA inference strategy to preserve pre-trained sequential reasoning during fine-tuning. Evaluated across four real-world event logs, D-TAIA consistently shows SOTA or competitive performance compared to a fine-tuned LLM and a recurrent neural network baseline. Ablation studies confirm that techniques from NLP and computer vision can be transferred effectively to PPM with only a 10M-parameter backbone, though component contributions vary by dataset entropy.
Figures & tables
Authors
Mechanism
Task
LD
HE
OOD
MT
PPM – non-FM
Weinzierl et al. (2025) [ 21 ]
Cross-organization transfer learning
Outcome
∼
Amiri Elyasi et al. (2024) [ 6 ]
Graph-based control-flow encoding
RT
✓
Zare et al. (2025) [ 22 ]
Entropy-weighted attention
NA
✓
van Straten (2026); Hassani et al. (2026) [ 20 , 8 ]
Continual learning under concept drift
NA
∼
Rama-Maneiro et al. (2023) [ 17 ]
Benchmark of 10 non-FM approaches
NA, Suffix, RT
∼
Table 1: Overview of related work and the research gap. Axes: LD = Low volume of data; HE = High-Entropy robustness; OOD = Out-of-Distribution generalization within a log; MT = Multi-task NA and RT prediction. ✓ = explicitly addressed with a dedicated mechanism and supporting experiment; ∼ = a mechanism is present or claimed but not directly evaluated.
Figure 1: Overview of the D-TAIA framework.
Dataset
# Cases
# Events
# Activities
Avg. Length
Avg. Duration
Entropy
BPI2012
13,087
262,200
24
20.04
8.62
3.688
BPI2017
31,509
1,202,267
26
38.16
21.90
3.786
BPI2015_2
832
44,354
410
53.31
160.49
7.105
BPI2020_DD
10,500
56,437
17
5.37
11.67
2.827
Table 2: Characteristics of the event logs used for evaluation.
Figure 2: D-TAIA achieves higher Macro-F1 score and lower MAE than FT-LLM [ 11 ] across all backbones and datasets.
Figure 3: D-TAIA attains the best point estimate for Macro-F1 and RT MAE at every training-data budget across all four logs. The advantage over FT-LLM is clearest and most persistent at full data on the high-entropy BPI2015_2, while the three models converge most closely on BPI2017 and the near-deterministic BPI2020_DD.
Figure 4: D-TAIA’s advantage over FT-LLM and MT-RNN in macro-F1 and RT MAE is largest at short prefixes (bucket 1) and narrows toward longer prefixes (bucket 5) across all four logs. The residual gap at bucket 5 remains proportionally largest on the high-entropy BPI2015_2.
High Entropy (BPI2015_2)
Low Entropy (BPI2020_DD)
Variant
Macro-F1
MAE
Macro-F1
MAE
D-TAIA (full)
.445±.032
53.85±2.18
.811±.011
6.52±0.48
– DATL
.352±.048∗
68.45±3.95∗
.783±.018
7.85±0.68∗
– Domain ID
.385±.042
58.22±2.65
.790±.016
6.92±0.55
– FAISS
.418±.036
61.85±2.88∗
.807±.012
6.68±0.52
– TAIA
.425±.034
55.28±2.35
.806±.013
6.72±0.50
Table 3: Ablation results of the framework’s components. Bold marks the best value per column. ∗ marks a variant whose 95% CI does not overlap the full model’s (D-TAIA) on that metric.
Predictive process monitoring (PPM) leverages event logs to forecast the future of running process instances, for instance, predicting the next activity, the remaining time until case completion, or the time to the next event. While PPM research in recent years has been dominated by deep sequence models trained from scratch, such as Long Short-Term Memory (LSTM) models, foundation-model approaches---particularly large language models (LLMs)---are increasingly explored for PPM. At the same time, tabular foundation models with in-context learning capabilities offer a promising alternative but have not yet been systematically benchmarked for PPM. Thus, it remains unclear whether classical sequence-based models remain competitive in this evolving landscape. This paper compares the three modeling paradigms both conceptually and empirically through a controlled benchmark across multiple datasets and prediction tasks. The results show that sequence models consistently perform best for next activity prediction, whereas tabular foundation models are competitive on temporal tasks, with LLMs usually lagging behind despite higher cost.
Lennart Fertig, Lukas Kirchdorfer, Tobias Sesterhenn
University of Mannheim, Mannheim, Germany · SAP Signavio, Walldorf, Germany · Technical University of Clausthal, Clausthal, Germany
Predictive Process Monitoring (PPM) forecasts how ongoing organizational processes unfold, enabling information systems to move beyond execution support toward proactive analysis and monitoring. Although deep learning has improved prediction accuracy in PPM, most approaches follow a single-task learning (STL) setup, training a separate model per task. This increases maintenance effort and overlooks potential synergies. Multi-task learning (MTL), which jointly learns multiple prediction targets in one model, offers a promising alternative, yet its effectiveness in PPM remains underexplored. It remains unclear whether and under which settings MTL improves upon STL, which prediction tasks benefit most from joint learning, which task combinations are particularly synergistic, and if and how tasks should be balanced. To fill this gap, we present the first comprehensive empirical study of MTL for PPM, evaluating a variety of task combinations, neural architectures, and optimization methods. Overall, our results position MTL as a strong paradigm for PPM: we see substantial improvements in next-activity prediction and inherent mitigation of class imbalance using MTL, while task balancing is especially critical under low-capacity models.
Lukas Kirchdorfer, Keyvan Amiri Elyasi, Heiner Stuckenschmidt
Predictive Process Monitoring (PPM) models are increasingly deployed in dynamic environments where concept drift causes the underlying process distribution to shift over time. While recent work has moved toward online continual learning, existing methods train compact, task-specific networks entirely from scratch, leaving a persistent cold-start problem. Foundation Models (FMs) offer a compelling solution to this problem, but their continual fine-tuning in the process mining domain remains unexplored. We propose COMPASS (Continual Online foundation Model-based PPM with Adaptive SubSpaces), the first framework for online continual fine-tuning of FMs for PPM. COMPASS adapts loss-plateau drift detection to autonomously identify task boundaries in event streams and maintains a unified knowledge subspace including both pre-trained and task-specific directions. We evaluate our approach on nine event streams covering synthetic and real-world concept drift scenarios, across task-free and task-aware settings with multiple backbones and with consistent hyperparameter tuning across all methods. Our approach outperforms three SOTA non-FM competitors and two update strategy baselines, with particularly strong gains on streams exhibiting recurrent drift and complex, long-running cases, while incurring acceptable computational overhead compared to the non-FM competitors.
Sjoerd van Straten, Marwan Hassani
Department of Mathematics and Computer Science, Eindhoven University of Technology, The Netherlands · Department of Mathematics and Computer Science, University of Wuppertal, Germany