Neural ordinary differential equations use numerical solvers inside the training loop. The solver determines the forward trajectory and also affects the gradient passed to the optimizer. That coupling creates a reliability problem for scientific machine learning and continuous-time generative modeling, including diffusion probability-flow ordinary differential equations and flow-matching models. Under loose step sizes, stiff dynamics, chaotic sensitivity, or event discontinuities, a differentiable ODE pipeline can return a finite gradient whose direction is numerically suspect. We introduce GradRepair-ODE, a reliability framework for checking, repairing, and rejecting ODE gradients at the optimizer step. The method computes several gradient candidates, compares them with directional finite-difference checks and solver diagnostics, diagnoses likely numerical failure modes, repairs selected gradients through path switching or stricter recomputation, and rejects steps whose descent direction cannot be certified. In six synthetic ODE systems, GradRepair-ODE leaves low-risk systems unchanged, repairs Robertson and Lorenz gradients to cosine similarity 1.000 against a strict reference, reduces unsafe accepted steps from 37 to 0, and rejects an event-discontinuous case instead of applying an uncertified update. The paper argues for a simple change in the training contract: an ODE gradient should reach the optimizer with numerical evidence attached.
Figures & tables
System
Naive cos
Discrete cos
GradRepair cos
Finite-difference risk
Repair rate
Harmonic
1.000
1.000
1.000
0.003
0.00
Van der Pol
1.000
1.000
1.000
0.064
0.00
Robertson
-1.000
-1.000
1.000
8.140
1.00
Lorenz
0.698
1.000
1.000
0.228
1.00
Bouncing ball
-1.000
-1.000
–
0.934
1.00
Neural RHS
1.000
1.000
1.000
0.001
0.00
Table 1: Gradient reliability across ODE systems. Gray cells mark quantities that triggered certificate attention: large finite-difference risk or a nonzero repair rate. A dash indicates that no repaired gradient was accepted. Cosine similarity is measured against a strict reference gradient. Finite-difference risk is log10(1+finite\mbox−differenceresidual) , with the raw residual preserved in the experiment log.
Figure 1: Gradient direction agreement against a strict reference. GradRepair-ODE leaves smooth low-risk systems unchanged, while exposing failure or repair needs in stiff, chaotic, and event-driven systems.
System
Primary evidence
Routing action
Final step
Cost multiplier
Harmonic
consistent paths
none
accepted
1.0
Van der Pol
consistent paths
none
accepted
1.0
Robertson
finite-difference residual
strict recomputation
accepted
301.0
Lorenz
path disagreement
checkpointed routing
accepted
9.3
Bouncing ball
finite-difference residual
reject
rejected
41.0
Neural RHS
consistent paths
none
accepted
1.0
Table 2: Failure routing induced by the certificate. Gray cells mark final accepted or rejected interventions selected by the certificate. The table reports the primary numerical evidence, selected repair action, final optimizer decision, and function-evaluation multiplier for each system.
System
Naive loss
GradRepair loss
Naive unsafe
GradRepair unsafe
GradRepair rejected
Repair rate
Decision
Harmonic
1.273e-05
1.273e-05
0
0
0
0.00
trust
Van der Pol
3.081
3.081
0
0
0
0.00
trust
Robertson
0.03433
0.03483
18
0
0
1.00
repair
Lorenz
48.42
48.33
18
0
0
1.00
repair
Bouncing ball
6.408
1.068
1
0
18
1.00
reject
Neural RHS
0.09854
0.09854
0
0
0
0.00
trust
Table 3: Training stability summary. Red cells mark unsafe updates accepted by the naive loop, green cells mark repaired or cleared unsafe counts, yellow cells mark certified rejection, and gray cells mark trusted no-intervention decisions. Unsafe accepted counts steps that the naive loop applied despite a non-trusted certificate.
Figure 2: Loss trajectories under naive training and GradRepair-ODE. The goal is reliability control rather than aggressive optimization; rejected steps preserve evidence when no safe gradient is available.
Policy
Unsafe accepted
Repaired
Rejected
Main effect
naive
37
0
0
no control
detect only
37
0
0
warning only
no finite-difference evidence
18
36
0
event risk passes
no repair routing
0
0
54
over-rejects
no step certification
18
54
0
applies failed repair
full GradRepair-ODE
0
54
18
repair or reject
Table 4: Ablation study from the same recorded step evidence. Red cells mark unsafe accepted updates, green cells mark repaired or cleared unsafe counts, and yellow cells mark rejected updates.
Figure 3: Threshold sensitivity for the certificate decision. The reliability layer has a stable operating region: stricter thresholds increase repair and rejection, while unsafe accepted steps disappear once suspicious gradients are no longer trusted by default.
Figure 4: Robertson case study. The naive and discrete paths point in the opposite direction from the strict reference, while GradRepair-ODE routes to strict recomputation and restores cosine similarity to 1.000 .
System
Naive ms
GradRepair ms
Time multiplier
NFE multiplier
Harmonic
4.48
4.48
1.0
1.0
Van der Pol
0.54
0.54
1.0
1.0
Robertson
0.34
29.04
85.9
301.0
Lorenz
2.10
17.74
8.4
9.3
Bouncing ball
0.23
2.74
11.9
41.0
Neural RHS
2.69
2.69
1.0
1.0
Table 5: Measured diagnostic cost. Yellow cells mark time or function-evaluation multipliers above 5× . Wall-clock measurements are generated by the same deterministic experiment script and reported in milliseconds.
Method
Mechanism
Unsafe accepted
Rejected
Loss spikes
naive
coarse update
37
0
1
naive + clipping
norm control only
47
0
3
naive + line search
loss decrease check
37
17
0
GradRepair-ODE
repair or reject
0
18
0
Table 6: Comparison with classical safeguards. Red cells mark unsafe accepted updates, green cells mark cleared unsafe counts, yellow cells mark rejected updates, and light red cells mark loss spikes.
Figure 5: Per-step GradRepair decisions. Trusted steps are left unchanged, repaired steps are routed to a safer gradient path, and rejected steps are withheld when no smooth-gradient claim is certified.
Figure 6: Repair cost measured as the multiplier in function evaluations relative to the naive gradient path, shown on a logarithmic scale to preserve the full Robertson cost.
Figure 7: Reliability gain measured as cosine improvement relative to the naive gradient path. The largest gains occur where the naive direction is inconsistent with the strict reference.
Figure 8: Directional finite-difference residuals. Large residuals identify candidates whose directional derivative evidence is inconsistent with the proposed gradient.
Department of Informatics and Computer Science, University of Pisa, Italy · Department of Physics and Astronomy and INFN, University of Florence, Sesto Fiorentino, Italy