We consider nonparametric maximum likelihood estimation of the intensity function of a Poisson process, allowing the intensity to depend on observed covariates. Such models describe how event rates vary with external conditions, including conditions under which the intensity becomes arbitrarily small or vanishes. However, risk analysis based on the Kullback-Leibler (KL) divergence typically requires the true intensity to be bounded away from zero, a restrictive assumption in applications. We develop a unified approach that avoids this assumption and derive an oracle inequality for the covariate-averaged Hellinger distance between the true intensity and the nonparametric maximum likelihood estimator (NPMLE). As an application of the oracle inequality, we establish convergence rates for NPMLEs based on deep neural networks (DNNs). Our results show that, as in nonparametric regression, the ability of DNNs to capture underlying structure leads to improved convergence rates. Specifically, DNN-based NPMLEs mitigate the curse of dimensionality when the true intensity has a compositional structure and automatically adapt to a low-dimensional Riemannian manifold structure of the event space. For intensity functions with a compositional structure, DNN-based NPMLEs achieve nearly minimax-optimal convergence rates without explicitly incorporating this structure into the network architecture. Simulations demonstrate that DNN-based estimators outperform kernel estimators in most settings, while manifold kernels remain competitive in some designs. Finally, we analyze stimulus-evoked neural spiking using data from the Allen Brain Observatory Visual Coding Neuropixels project. DNN-based NPMLEs provide better out-of-sample performance than a conditional kernel estimator, as evaluated by held-out log-likelihood.
Figures & tables
Design/support
d
dz
Softplus
ReLU
Kernel
Pooled
Composition
1
1
0.00926 (0.00073)
0.0104 (0.00096)
0.0329 (0.0048)
0.474 (0.007)
Composition
1
5
0.0194 (0.0011)
0.0203 (0.0011)
0.144 (0.01)
0.544 (0.011)
Composition
2
1
0.0205 (0.00071)
0.0213 (0.00099)
0.0441 (0.0055)
0.398 (0.0052)
Composition
2
5
0.0346 (0.0013)
0.0329 (0.0012)
0.182 (0.0062)
0.466 (0.0067)
Near zero
1
1
0.0092 (0.0012)
0.0229 (0.0023)
0.0442 (0.00036)
2.61 (0.044)
Near zero
1
5
0.0179 (0.001)
0.0359 (0.0032)
0.597 (0.0051)
2.67 (0.031)
Table 1: Mean squared Hellinger loss at n=10000 , with Monte Carlo standard errors in parentheses, based on 20 repetitions. Here d is the event-support dimension. “Kernel” denotes the conditional kernel for Euclidean supports and the geodesic kernel for manifold supports; the manifold DNNs use reconstructed integration meshes. A dash indicates that the pooled comparator was not used.
Figure 1: Squared Hellinger loss for the compositional intensity. Rows correspond to event dimension and columns to covariate dimension. Points represent individual repetitions, connected markers are means, and vertical segments span the minimum and maximum over 20 repetitions.
Units
DNN
NLL difference (nats/presentation)
Improved
Information gain (bits/spike)
12
Softplus
−0.02901
10/12
0.01073
[−0.04042,−0.01729]
[0.00606,0.01591]
12
ReLU
−0.02080
9/12
0.00600
[−0.03092,−0.01026]
[0.00175,0.01001]
11
Softplus
−0.02610
9/11
0.00944
[−0.03791,−0.01378]
[0.00473,0.01461]
Table 2: Predictive performance relative to the conditional kernel. NLL differences and information gains are averaged equally over units; brackets give pointwise 95% paired condition-bootstrap intervals. “Improved” is the number of units with a negative NLL difference. The 11-unit analysis excludes unit 950931315, examined in the pilot.
Figure 2: Unit-specific mean test NLL differences from the conditional kernel, in nats per presentation. Negative values favor the DNN. Horizontal segments give pointwise 95% paired condition-bootstrap intervals. The asterisk denotes unit 950931315, examined in the pilot.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Squared Hellinger loss for the intensity with a moving valley. The intensity floor is 10−4 before rescaling. Points, connected markers, and vertical segments show individual repetitions, means, and ranges, as in Figure 1.
Figure 4: Squared Hellinger loss on the circle and sphere. The DNNs use ambient coordinates and reconstructed integration meshes; the kernel uses known geodesic distances and geometry. Graphical summaries follow Figure 1.
Figure 5: Squared Hellinger loss without covariates. The Euclidean kernel estimates the unconditional intensity. Points, connected markers, and vertical segments show individual repetitions, means, and ranges.
Figure 6: Neural estimators with reconstructed-mesh (M) and known-measure (O) integration. Both use ambient inputs, with different dimensions in the architecture rule ( 28 ).
Figure 7: Median total computation time by sample size, pooled over the configurations and repetitions available for each method. Totals include data generation, fitting, evaluation, and output.
Method
Runs
Total
Fit
Simulation
Evaluation
DNN–Softplus
240
31.3
26.4
2.03
0.714
DNN–ReLU
240
30.5
26.3
2
0.0455
Conditional kernel
160
459
198
0.751
252
Pooled kernel
240
207
68.9
0.762
135
Manifold kernel O
120
198
173
0.885
21.9
DNN–Softplus M
120
33
28.9
2.18
0.0408
Appendix
Table 3: Median computation times in seconds at n=10000 , pooled over the configurations available for each method. M and O denote reconstructed-mesh and known-measure integration. Total time includes data generation, fitting, evaluation, and file output; component medians need not sum to the median total.
Units
First vs second method
Macro Δ NLL 95% interval
Micro bits/spike 95% interval
12
DNN–Softplus vs Conditional kernel
-0.029006 [-0.040423, -0.017292]
0.0078515 [0.0047153, 0.010883]
11
DNN–Softplus vs Conditional kernel
-0.026096 [-0.037913, -0.013776]
0.0068535 [0.0036527, 0.0099081]
12
DNN–ReLU vs Conditional kernel
-0.020796 [-0.030918, -0.010262]
0.0056291 [0.0028146, 0.0082415]
11
DNN–ReLU vs Conditional kernel
-0.019545 [-0.029905, -0.0093311]
0.005133 [0.0024795, 0.0077339]
12
DNN–Softplus vs DNN–ReLU
-0.0082103 [-0.017285, 0.00092019]
0.0022224 [-0.00025334, 0.0047148]
11
DNN–Softplus vs DNN–ReLU
-0.0065513 [-0.016178, 0.0035349]
0.0017206 [-0.00092349, 0.0043033]
Appendix
Table 4: Pairwise predictive comparisons for the 12-unit and 11-unit analyses. Negative mean NLL differences and positive information gains favor the first method. Micro information gain pools score differences and spike counts across units. Brackets give pointwise 95% paired condition-bootstrap intervals.
Unit
Method
Mean NLL
Count deviance
Count MAE
Count RMSE
950930407
DNN–ReLU
-32.091
733.63
2.8705
3.5959
950930407
DNN–Softplus
-32.089
734.56
2.8668
3.596
950930407
Conditional kernel
-32.072
742.38
2.8862
3.6215
950930407
Pooled kernel
-32.009
791.48
2.9592
3.752
950930985
DNN–ReLU
-27.301
931.75
3.0347
3.7733
950930985
DNN–Softplus
-27.302
929.87
3.0328
3.769
Appendix
Table 5: Unit-specific prediction scores for the four estimators. Mean NLL is in nats per test presentation; count deviance is summed over the 607 test presentations. Count MAE and RMSE are in spikes per presentation.
Analysis
Method
Presentations
Mean NLL
Count deviance
Seed 20260903
DNN–Softplus
607
-6.7334
1254.6
Seed 20260903
DNN–ReLU
607
-6.707
1272.1
Seed 20260913
DNN–Softplus
607
-6.6892
1318.5
Seed 20260913
DNN–ReLU
607
-6.7151
1269.9
Seed 20260923
DNN–Softplus
607
-6.7197
1262.4
Seed 20260923
DNN–ReLU
607
-6.6971
1275.9
Appendix
Table 6: Sensitivity analyses for unit 950931315. Mean NLL is in nats per test presentation, and count deviance is summed over the test presentations. The seed and width analyses include the corresponding primary fit for reference. Condition-fold scores use different held-out stimuli.
Figure 8: Estimated temporal firing rates for unit 950931315 at preferred, intermediate, and low-response stimulus conditions selected using development counts. Time is measured from stimulus onset and intensity is in spikes per second. Curves are fitted intensities.
Figure 9: Gabor presentation timing and orientation in the development and test samples (left), and presentation counts before and after exclusions (right). The split was stratified by position–orientation combination.
Figure 10: Panel-average DNN performance relative to the conditional kernel for all 12 selected units and the 11 units excluding the pilot. Negative NLL differences and positive information gains favor the DNN. Segments are pointwise 95% paired condition-bootstrap intervals.
Figure 11: Sensitivity to integration resolution and network initialization. Left: changes in mean NLL from the 1,024-point score for each of the 48 unit–method fits. Right: mean test NLL at three optimization seeds for unit 950931315.
Figure 12: Count and timing diagnostics. Left: observed and predicted counts in method-specific deciles of predicted count, pooled over test unit–presentation pairs. Right: Kaplan–Meier survival curves of integrated-hazard increments for unit 950931315, with right censoring at presentation end, and the unit-exponential reference.
Figure 13: Prediction across stimulus conditions and recording time. Left: mean NLL for unit 950931315 in five condition-held-out folds. Right: equal-unit mean NLL by chronological quartile of the primary test presentations, using the original fits.
Cox processes model overdispersed point process data via a latent stochastic intensity, but both nonparametric estimation of the intensity model and posterior inference over intensity paths are typically intractable, relying on expensive MCMC methods. We introduce Neural Diffusion Intensity Models, a variational framework for Cox processes driven by neural SDEs. Our key theoretical result, based on enlargement of filtrations, shows that conditioning on point process observations preserves the diffusion structure of the latent intensity with an explicit drift correction. This guarantees the variational family contains the true posterior, so that ELBO maximization coincides with maximum likelihood estimation under sufficient model capacity. We design an amortized encoder architecture that maps variable-length event sequences to posterior intensity paths by simulating the drift-corrected SDE, replacing repeated MCMC runs with a single forward pass. Experiments on synthetic and real-world data demonstrate accurate recovery of latent intensity dynamics and posterior paths, with orders-of-magnitude speedups over MCMC-based methods.
Xinlong Du, Harsha Honnappa, Vinayak Rao
Edwardson School of Industrial Engineering, Purdue University · Department of Statistics, Purdue University
There remain theoretical gaps in deep neural network estimators for the nonparametric Cox proportional hazards model. In particular, it is unclear how gradient-based optimization error propagates to population risk under partial likelihood, how pointwise bias can be controlled to permit valid inference, and how ensemble-based uncertainty quantification behaves under realistic variance decay regimes. We develop an asymptotic distribution theory for deep Cox estimators that addresses these issues. First, we establish nonasymptotic oracle inequalities for general trained networks that link in-sample optimization error to population risk without requiring the exact empirical risk optimizer. We then construct a structured neural parameterization that achieves infinity-norm approximation rates compatible with the oracle bound, yielding control of the pointwise bias. Under these conditions and using the Hajek--Hoeffding projection, we prove pointwise and multivariate asymptotic normality for subsampled ensemble estimators. We derive a range of subsample sizes that balances bias correction with the requirement that the Hajek--Hoeffding projection remain dominant. This range accommodates decay conditions on the single-overlap covariance, which measures how strongly a single shared observation influences the estimator, and is weaker than those imposed in the subsampling literature. An infinitesimal jackknife representation provides analytic covariance estimation and valid Wald-type inference for relative risk contrasts such as log-hazard ratios. Finally, we illustrate the finite-sample implications of the theory through simulations and a real data application.
Sattwik Ghosal, Xuran Meng, Yi Li
Department of Biostatistics, University of Michigan
Temporal point processes (TPPs) are the canonical model class for event sequences in continuous time, with applications ranging from seismology to electronic health records. Recent benchmarks report that neural TPPs substantially outperform their classical counterparts, such as Hawkes processes. However, these comparisons set flexible neural models against Hawkes processes in their most restricted forms and covered only a narrow range of domains. We revisit this comparison under autoregressive next-event prediction, evaluating Poisson and Hawkes processes of increasing flexibility against neural TPPs on five established benchmarks and two large-scale medical tasks derived from UK Biobank disease histories and MIMIC-IV ICU records. First, once Hawkes processes are equipped with flexible, learnable baselines and kernels, the previously reported gap to neural TPPs largely closes. Second, neural TPPs are likewise limited by their intensity parametrization: a Transformer with a spline-based intensity head outperforms previous intensity-based neural TPPs and matches or exceeds the intensity-free model. Decomposing the log-likelihood into time and type terms shows that this gain lies almost entirely in modeling when events occur (log p(t)) rather than which type occurs (log p(d|t)). Third, only on MIMIC-IV do neural TPPs retain an advantage in predicting event types, underscoring the value of complex datasets for evaluating TPPs. We release TorchPP, an open-source PyTorch framework with unified implementations of classical and neural TPPs.
Hendrik Mehrtens, Florian Buettner, Oliver Stegle
German Cancer Research Center (DKFZ), Heidelberg, Germany · Goethe-Universität Frankfurt, Frankfurt, Germany · European Molecular Biology Laboratory (EMBL), Heidelberg, Germany +1