We consider nonparametric maximum likelihood estimation of the intensity function of a Poisson process, allowing the intensity to depend on observed covariates. Such models describe how event rates vary with external conditions, including conditions under which the intensity becomes arbitrarily small or vanishes. However, risk analysis based on the Kullback-Leibler (KL) divergence typically requires the true intensity to be bounded away from zero, a restrictive assumption in applications. We develop a unified approach that avoids this assumption and derive an oracle inequality for the covariate-averaged Hellinger distance between the true intensity and the nonparametric maximum likelihood estimator (NPMLE). As an application of the oracle inequality, we establish convergence rates for NPMLEs based on deep neural networks (DNNs). Our results show that, as in nonparametric regression, the ability of DNNs to capture underlying structure leads to improved convergence rates. Specifically, DNN-based NPMLEs mitigate the curse of dimensionality when the true intensity has a compositional structure and automatically adapt to a low-dimensional Riemannian manifold structure of the event space. For intensity functions with a compositional structure, DNN-based NPMLEs achieve nearly minimax-optimal convergence rates without explicitly incorporating this structure into the network architecture. Simulations demonstrate that DNN-based estimators outperform kernel estimators in most settings, while manifold kernels remain competitive in some designs. Finally, we analyze stimulus-evoked neural spiking using data from the Allen Brain Observatory Visual Coding Neuropixels project. DNN-based NPMLEs provide better out-of-sample performance than a conditional kernel estimator, as evaluated by held-out log-likelihood.
Figures & tables
Design/support
d
dz
Softplus
ReLU
Kernel
Pooled
Composition
1
1
0.00926 (0.00073)
0.0104 (0.00096)
0.0329 (0.0048)
0.474 (0.007)
Composition
1
5
0.0194 (0.0011)
0.0203 (0.0011)
0.144 (0.01)
0.544 (0.011)
Composition
2
1
0.0205 (0.00071)
0.0213 (0.00099)
0.0441 (0.0055)
0.398 (0.0052)
Composition
2
5
0.0346 (0.0013)
0.0329 (0.0012)
0.182 (0.0062)
0.466 (0.0067)
Near zero
1
1
0.0092 (0.0012)
0.0229 (0.0023)
0.0442 (0.00036)
2.61 (0.044)
Near zero
1
5
0.0179 (0.001)
0.0359 (0.0032)
0.597 (0.0051)
2.67 (0.031)
Table 1: Mean squared Hellinger loss at n=10000 , with Monte Carlo standard errors in parentheses, based on 20 repetitions. Here d is the event-support dimension. “Kernel” denotes the conditional kernel for Euclidean supports and the geodesic kernel for manifold supports; the manifold DNNs use reconstructed integration meshes. A dash indicates that the pooled comparator was not used.
Figure 1: Squared Hellinger loss for the compositional intensity. Rows correspond to event dimension and columns to covariate dimension. Points represent individual repetitions, connected markers are means, and vertical segments span the minimum and maximum over 20 repetitions.
Units
DNN
NLL difference (nats/presentation)
Improved
Information gain (bits/spike)
12
Softplus
−0.02901
10/12
0.01073
[−0.04042,−0.01729]
[0.00606,0.01591]
12
ReLU
−0.02080
9/12
0.00600
[−0.03092,−0.01026]
[0.00175,0.01001]
11
Softplus
−0.02610
9/11
0.00944
[−0.03791,−0.01378]
[0.00473,0.01461]
Table 2: Predictive performance relative to the conditional kernel. NLL differences and information gains are averaged equally over units; brackets give pointwise 95% paired condition-bootstrap intervals. “Improved” is the number of units with a negative NLL difference. The 11-unit analysis excludes unit 950931315, examined in the pilot.
Figure 2: Unit-specific mean test NLL differences from the conditional kernel, in nats per presentation. Negative values favor the DNN. Horizontal segments give pointwise 95% paired condition-bootstrap intervals. The asterisk denotes unit 950931315, examined in the pilot.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Squared Hellinger loss for the intensity with a moving valley. The intensity floor is 10−4 before rescaling. Points, connected markers, and vertical segments show individual repetitions, means, and ranges, as in Figure 1.
Figure 4: Squared Hellinger loss on the circle and sphere. The DNNs use ambient coordinates and reconstructed integration meshes; the kernel uses known geodesic distances and geometry. Graphical summaries follow Figure 1.
Figure 5: Squared Hellinger loss without covariates. The Euclidean kernel estimates the unconditional intensity. Points, connected markers, and vertical segments show individual repetitions, means, and ranges.
Figure 6: Neural estimators with reconstructed-mesh (M) and known-measure (O) integration. Both use ambient inputs, with different dimensions in the architecture rule ( 28 ).
Figure 7: Median total computation time by sample size, pooled over the configurations and repetitions available for each method. Totals include data generation, fitting, evaluation, and output.
Method
Runs
Total
Fit
Simulation
Evaluation
DNN–Softplus
240
31.3
26.4
2.03
0.714
DNN–ReLU
240
30.5
26.3
2
0.0455
Conditional kernel
160
459
198
0.751
252
Pooled kernel
240
207
68.9
0.762
135
Manifold kernel O
120
198
173
0.885
21.9
DNN–Softplus M
120
33
28.9
2.18
0.0408
Appendix
Table 3: Median computation times in seconds at n=10000 , pooled over the configurations available for each method. M and O denote reconstructed-mesh and known-measure integration. Total time includes data generation, fitting, evaluation, and file output; component medians need not sum to the median total.
Units
First vs second method
Macro Δ NLL 95% interval
Micro bits/spike 95% interval
12
DNN–Softplus vs Conditional kernel
-0.029006 [-0.040423, -0.017292]
0.0078515 [0.0047153, 0.010883]
11
DNN–Softplus vs Conditional kernel
-0.026096 [-0.037913, -0.013776]
0.0068535 [0.0036527, 0.0099081]
12
DNN–ReLU vs Conditional kernel
-0.020796 [-0.030918, -0.010262]
0.0056291 [0.0028146, 0.0082415]
11
DNN–ReLU vs Conditional kernel
-0.019545 [-0.029905, -0.0093311]
0.005133 [0.0024795, 0.0077339]
12
DNN–Softplus vs DNN–ReLU
-0.0082103 [-0.017285, 0.00092019]
0.0022224 [-0.00025334, 0.0047148]
11
DNN–Softplus vs DNN–ReLU
-0.0065513 [-0.016178, 0.0035349]
0.0017206 [-0.00092349, 0.0043033]
Appendix
Table 4: Pairwise predictive comparisons for the 12-unit and 11-unit analyses. Negative mean NLL differences and positive information gains favor the first method. Micro information gain pools score differences and spike counts across units. Brackets give pointwise 95% paired condition-bootstrap intervals.
Unit
Method
Mean NLL
Count deviance
Count MAE
Count RMSE
950930407
DNN–ReLU
-32.091
733.63
2.8705
3.5959
950930407
DNN–Softplus
-32.089
734.56
2.8668
3.596
950930407
Conditional kernel
-32.072
742.38
2.8862
3.6215
950930407
Pooled kernel
-32.009
791.48
2.9592
3.752
950930985
DNN–ReLU
-27.301
931.75
3.0347
3.7733
950930985
DNN–Softplus
-27.302
929.87
3.0328
3.769
Appendix
Table 5: Unit-specific prediction scores for the four estimators. Mean NLL is in nats per test presentation; count deviance is summed over the 607 test presentations. Count MAE and RMSE are in spikes per presentation.
Analysis
Method
Presentations
Mean NLL
Count deviance
Seed 20260903
DNN–Softplus
607
-6.7334
1254.6
Seed 20260903
DNN–ReLU
607
-6.707
1272.1
Seed 20260913
DNN–Softplus
607
-6.6892
1318.5
Seed 20260913
DNN–ReLU
607
-6.7151
1269.9
Seed 20260923
DNN–Softplus
607
-6.7197
1262.4
Seed 20260923
DNN–ReLU
607
-6.6971
1275.9
Appendix
Table 6: Sensitivity analyses for unit 950931315. Mean NLL is in nats per test presentation, and count deviance is summed over the test presentations. The seed and width analyses include the corresponding primary fit for reference. Condition-fold scores use different held-out stimuli.
Figure 8: Estimated temporal firing rates for unit 950931315 at preferred, intermediate, and low-response stimulus conditions selected using development counts. Time is measured from stimulus onset and intensity is in spikes per second. Curves are fitted intensities.
Figure 9: Gabor presentation timing and orientation in the development and test samples (left), and presentation counts before and after exclusions (right). The split was stratified by position–orientation combination.
Figure 10: Panel-average DNN performance relative to the conditional kernel for all 12 selected units and the 11 units excluding the pilot. Negative NLL differences and positive information gains favor the DNN. Segments are pointwise 95% paired condition-bootstrap intervals.
Figure 11: Sensitivity to integration resolution and network initialization. Left: changes in mean NLL from the 1,024-point score for each of the 48 unit–method fits. Right: mean test NLL at three optimization seeds for unit 950931315.
Figure 12: Count and timing diagnostics. Left: observed and predicted counts in method-specific deciles of predicted count, pooled over test unit–presentation pairs. Right: Kaplan–Meier survival curves of integrated-hazard increments for unit 950931315, with right censoring at presentation end, and the unit-exponential reference.
Figure 13: Prediction across stimulus conditions and recording time. Left: mean NLL for unit 950931315 in five condition-held-out folds. Right: equal-unit mean NLL by chronological quartile of the primary test presentations, using the original fits.
German Cancer Research Center (DKFZ), Heidelberg, Germany · Goethe-Universität Frankfurt, Frankfurt, Germany · European Molecular Biology Laboratory (EMBL), Heidelberg, Germany +1