Predicting irregularly spaced event sequences with discrete marks poses significant challenges due to the complex, asynchronous dependencies embedded within continuous-time data streams.Existing sequential approaches capture dependencies among event tokens but ignore the continuous evolution between events, while Neural Ordinary Differential Equation (Neural ODE) methods model smooth dynamics yet fail to account for how event types influence future timing.To overcome these limitations, we propose NEXTPP, a dual-channel framework that unifies discrete and continuous representations via Event-granular Neural Evolution with Cross-Interaction for Marked Temporal Point Processes. Specifically, NEXTPP encodes discrete event marks via a self-attention mechanism, simultaneously evolving a latent continuous-time state using a Neural ODE. These parallel streams are then fused through a crossattention module to enable explicit bidirectional interaction between continuous and discrete representations. The fused representations drive the conditional intensity function of the neural Hawkes process, while an iterative thinning sampler is employed to generate future events. Extensive evaluations on five real-world datasets demonstrate that NEXTPP consistently outperforms state-of-the-art models. The source code can be found at https://github.com/AONE-NLP/NEXTPP.
Figures & tables
Figure 1: Seismic timeline for the August 30, 2009 sequence, showing 18 events within 12.3 s. A magnitude 5.2 mainshock at 11.56s marks the transition from foreshocks to aftershocks, illustrating the classic three-phase seismic cycle.
Figure 2: The overall framework NEXTPP. Left(SA): Discrete event sequence processing through embedding and self-attention layers. Right(NE): Continuous representation via latent space. X-Interaction(CA): Continuous-Discrete Interaction.
Dataset
M
Event Tokens
Sequence Length
Train
Dev
Test
Min
Mean
Max
RETWEET
3
369731
62823
61154
10
41
97
AMAZON
16
288377
40995
84048
14
44
94
TAXI
10
51854
7404
14820
36
37
38
STACKOVERFLOW
22
90497
25762
26518
41
65
101
EARTHQUAKE
7
49363
6612
14748
15
16
18
Table 1: Dataset statistics with number of marks (M), event tokens distribution across splits, and sequence length characteristics.
Method
Taxi
Amazon
S.O.
EQ
Retweet
R
E
R
E
R
E
R
E
R
E
MHP
0.382
9.53
0.635
75.9
1.388
65.0
4.128
54.9
22.920
55.7
RMTPP
0.374
10.4
0.474
68.1
1.392
56.3
2.097
54.9
26.508
45.8
NHP
0.373
10.0
0.513
68.5
1.431
53.3
2.347
54.7
22.410
39.9
THP
0.370
10.1
0.471
66.6
1.343
53.3
2.160
55.1
24.491
39.9
SAHP
0.334
10.0
0.549
68.9
1.331
56.5
1.864
54.9
21.673
40.6
Table 2: Results of all model reproductions across five real datasets. The performance of our NEXTPP model represents the average of five training runs with different random seeds. Lower scores indicate better performance. The best results are indicated in bold and suboptimal results are underlined .(R = RMSE ( ↓ ), E = Error Rate % ( ↓ ), S.O. = StackOverflow, EQ = Earthquake)
MODEL
Log-Likelihood
Taxi
Amazon
Stack Overflow
Earthquake
Retweet
RMTPP
0.219±0.033
-2.258±0.009
-2.820±0.025
-2.313±0.272
-5.754±1.785
NHP
0.440±0.007
-2.036±0.066
-2.440±0.103
-2.155±0.056
-4.020±0.133
THP
0.217±0.034
-2.215±0.016
-2.478±0.031
-2.353±0.040
-4.143±0.005
SAHP
-0.295±0.266
-2.417±0.207
-6.930±0.007
-3.136±0.610
-4.798±0.637
AttNHP
0.164±0.031
-1.364±0.195
-2.560±0.004
-2.359±0.026
-4.128±0.002
Table 3: Log-Likelihood Performance Comparison. Reported values are the mean log-likelihood over five independent runs, with the corresponding standard deviation indicating experimental variability.
Settings
Amazon
StackOverflow
Retweet
LL( ↑ )
RMSE( ↓ )
ErrRt( ↓ )
LL( ↑ )
RMSE( ↓ )
ErrRt( ↓ )
LL( ↑ )
RMSE( ↓ )
ErrRt( ↓ )
w/o NE
-2.178
0.468
66.8%
-2.474
1.338
53.3%
-4.180
20.541
40.2%
w/o CA
-1.530
0.438
71.1%
-2.657
2.199
56.4%
-4.122
20.108
47.3%
NE → GRU
-2.235
0.481
67.7%
-2.369
1.366
53.2%
-4.166
21.145
40.7%
NE → LSTM
-2.263
0.495
68.2%
-2.403
1.389
53.3%
-4.177
20.795
40.6%
NEXTPP
-1.246
0.377
66.5%
-1.971
1.152
52.2%
-3.993
19.340
35.5%
Table 4: Ablation results on Amazon, StackOverflow and Retweet.
Figure 3: Block-wise test results: The sequence is divided into several blocks according to partition ratio α , with each block evolving in its corresponding latent space, where X indicates one block per event.(Lower scores are better)
Figure 4: Heatmap comparison of the seismic sequences (shown in Figure 1 ) during model training. (a) The first row shows our Continuous-Discrete Interaction( X-Interaction ); (b) The second row displays the conventional Attention( Self-Attention ).
Figure 5: Training curves with varying event counts (Percentage of the total training dataset) for Earthquake and Retweet datasets.
Multi-class event streams arise in numerous real-world applications, where uncovering structured, interpretable inter-event relationships, together with accurate prediction, remains a central challenge. Existing neural point process models are highly expressive but encode event interactions in a black-box manner, preventing explicit discovery of structured dependencies. In this paper, we propose a structured neural marked point process (SNMPP) that achieves high modeling flexibility while enabling explicit event-wise and class-wise relationship discovery from data. Our model constructs a product-form neural influence kernel composed of a signed interaction network over event types and a delay-aware monotonic temporal network. This design enables explicit characterization of inter-class influence topology -- including excitation, inhibition, and neutrality -- while flexibly capturing diverse temporal decay patterns and potential influence delays. For efficient learning, we develop a stratified Monte Carlo estimator for stochastic training. Extensive experiments on synthetic and real-world benchmark datasets validate the ability of our approach to uncover structured relationships and deliver strong predictive performance.
Zhitong Xu, Qiwei Yuan, Yinghao Chen +2
Kahlert School of Computing, University of Utah · Celonis AI
Events in spatiotemporal domains arise in numerous real-world applications, where uncovering event relationships and enabling accurate prediction are central challenges. Classical Poisson and Hawkes processes rely on restrictive parametric assumptions that limit their ability to capture complex interaction patterns, while recent neural point process models increase representational capacity but integrate event information in a black-box manner, hindering interpretable relationship discovery. To address these limitations, we propose a Kronecker-Structured Nonparametric Spatiotemporal Point Process (KSTPP) that enables transparent event-wise relationship discovery while retaining high modeling flexibility. We model the background intensity with a spatial Gaussian process (GP) and the influence kernel as a spatiotemporal GP, allowing rich interaction patterns including excitation, inhibition, neutrality, and time-varying effects. To enable scalable training and prediction, we adopt separable product kernels and represent the GPs on structured grids, inducing Kronecker-structured covariance matrices. Exploiting Kronecker algebra substantially reduces computational cost and allows the model to scale to large event collections. In addition, we develop a tensor-product Gauss-Legendre quadrature scheme to efficiently evaluate intractable likelihood integrals. Extensive experiments demonstrate the effectiveness of our framework.
Zhitong Xu, Qiwei Yuan, Yinghao Chen +3
Kahlert School of Computing, The University of Utah · College of Arts & Sciences, Utah State University · Celonis AI
Temporal point processes (TPPs) provide a general and flexible framework for modeling sequences of events in continuous time. Neural networks have been successfully employed to model TPPs in a highly expressive and data-driven way. Neural TPPs are typically trained via Maximum Likelihood Estimation (MLE) by minimizing the negative log-likelihood (NLL), which depends on both the conditional intensity function (CIF) and its integral over time, the compensator. Recent neural TPP approaches enable exact evaluation of the NLL without numerical integration. However, these methods typically model the compensator rather than the CIF directly, impose constraints on the neural network architecture, and are computationally expensive during training, as event contributions to the NLL are evaluated sequentially rather than in parallel. In this work, we propose a novel neural TPP model that directly parametrizes the CIF as a non-negative combination of B-spline basis functions, whose coefficients are predicted by a neural network. This formulation enables exact evaluation of the NLL, preserves full flexibility in the neural architecture, allows efficient parallelization during training, and naturally supports CIF smoothness regularization through the integrated squared second derivative. Experiments on both synthetic and real-world datasets show improved computational efficiency and predictive accuracy compared to the reference neural TPP baseline.
Michele Bellomo, Riccardo Ramaschi, Alberto Dolara +1
Politecnico di Milano Milan, Italy · University College London London, UK