stat.MEOct 12, 2026
SaveImplicit and explicit reparameterizations of the Folded Normal and Rice distributions
Organizations: The Harvard Biophysics Graduate Program, Harvard University, Cambridge, Massachusetts, United States · Department of Molecular and Cellular Biology, and John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, Massachusetts, United States · LCLS Data Systems, SLAC National Accelerator Laboratory, Menlo Park, CA 94025. · Center for Computational Mathematics, Flatiron Institute, New York, NY, USA
Abstract
Variational inference is a flexible tool for modeling experimental data and has proven useful in crystallography. Pathwise gradient estimators provide low-variance gradients for reparameterized variational inference, but are not readily available for some distributions central to crystallography. This work derives and implements implicit and explicit reparameterized gradient estimators for the folded Normal and Rice distributions in \prog{PyTorch}, and compares their variance analytically and empirically. In each case the implicit estimator achieves lower variance for the location derivative, particularly at small location-to-scale ratios, while the scale-derivative variances are nearly equal. For both distributions, implicit estimator is proved to be the conditional expectation of the explicit estimator given the sampled magnitude, so its variance is never larger. As a practical application, French-Wilson scaling is formulated as a variational inference problem, using the folded Normal and Rice distributions as variational families for centric and acentric reflections, respectively. Both estimators recover the exact posterior to better than on the amplitude, and converge at the same rate. The implementations are available in the open-source package \prog{rs-distributions}.
Figures & tables
Figure 1: Theoretical and empirical variance of the pathwise estimators as a function of Monte Carlo sample size , at (small ). Left, the derivative with respect to ; right, the derivative with respect to . The two empirical curves match their respective theoretical predictions: for the explicit estimator and equation 20 for the implicit estimator (left), where the expected conditional variance is the analytical gap from Theorem 1 . For the derivative (right) both empirical curves coincide with the theoretical prediction of equation 13 . At this parameter setting, the implicit derivative estimator achieves variance roughly 50-fold smaller than the explicit estimator.
Figure 2: Theoretical (top) and empirical (bottom) variance of the explicit ( ) and implicit ( ) derivative estimators of as a function of and . The colorbar denotes the variance value for each pair. The empirical variance was computed across simulations using Monte Carlo samples each. The theoretical variance was calculated using equations 13 and 20 , for the explicit and implicit estimators, respectively.
Figure 3: Variance ratio of the implicit estimator relative to the explicit estimator, as a function of the location-to-scale ratio , for the test function . Because the explicit variance is the upper bound, the ratio lies in and is independent of the number of Monte Carlo samples. The location-derivative ratio falls to at small , while the scale-derivative ratio dips only to near .
Figure 4: Theoretical and empirical variance of the pathwise estimators as a function of Monte Carlo sample size , at (small ). Left, the derivative with respect to ; right, the derivative with respect to . The two empirical curves match their respective theoretical predictions: for the explicit estimator and equation 34 for the implicit estimator (left), where the expected conditional variance is the analytical gap from Theorem 2 ; for the derivative (right) the theoretical curves coincide with the empirical curves. For the derivative at this parameter setting, the implicit estimator achieves variance roughly 100-fold smaller than the explicit estimator.
Figure 5: Theoretical (top) and empirical (bottom) variance of the explicit ( ) and implicit ( ) derivative estimators of as a function of and . The colorbar denotes the variance value for each pair. The empirical variance was computed across simulations using Monte Carlo samples each. The theoretical variance was calculated using equations 26 and 34 , for the explicit and implicit estimators, respectively.
Figure 6: Variance ratio of the implicit estimator relative to the explicit estimator, as a function of the location-to-scale ratio , for the test function . As in the case, the ratio lies in and is independent of the number of Monte Carlo samples. The location-derivative ratio falls to at small , while the scale-derivative ratio dips only to near .
Figure 7: Recovery of the reference posterior, optimized with Adam, as a function of the number of Monte Carlo samples used per gradient. Panels show the amplitude and the posterior width , plotted separately for centric and acentric reflections. Each point is a median over ten replicates, and the bands span the interquartile range (too narrow to visualize). The same comparison for all five optimization algorithms is given in Fig. S3.
Figure 8: Difference in amplitude mean absolute percentage error (MAPE) against the reference posterior (implicit minus explicit), as a function of number of Monte Carlo samples used at each training step. Left, the centric reflections at or below zero intensity, where the two gradient variances differ; right, intensities with , where the implicit and explicit gradient variances are approximately the same. Positive values favor the explicit estimator at this number of optimization steps. Points are means over the ten per-replicate paired differences, and the bands are one standard error of the mean.
| error (%) | error (%) | |||
|---|---|---|---|---|
| Method | centric | acentric | centric | acentric |
| Explicit | 0.22 | 0.11 | 1.76 | 1.54 |
| Implicit | 0.28 | 0.15 | 2.06 | 1.82 |
| cctbx | 0.06 | 0.11 | 0.07 | 0.39 |
Table 1: The mean absolute percentage error of each method against the exact posterior, evaluated on an experimental hen egg-white lysozyme dataset (PDB: 9B7C).
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
| Optimizer | Explicit | Implicit |
|---|---|---|
| SGD | ||
| SGD + momentum | ||
| RMSprop | ||
| Adam | ||
| AdaBelief |
Table S1: Learning rates used for the different optimizers, one per gradient estimator.
Figure S1: Theoretical (top) and empirical (bottom) variance of the explicit ( ) and implicit ( ) derivative estimators of as a function of and . The colorbar denotes the variance value for each pair. The empirical variance was computed across simulations using Monte Carlo samples each. The theoretical variance was calculated using equations 13 and 20 of the main text, for the explicit and implicit estimators, respectively.
Figure S2: Theoretical (top) and empirical (bottom) variance of the explicit ( ) and implicit ( ) derivative estimators of as a function of and . The colorbar denotes the variance value for each pair. The empirical variance was computed across simulations using Monte Carlo samples each. The theoretical variance was calculated using equations 26 and 34 of the main text, for the explicit and implicit estimators, respectively.
Figure S3: Recovery of the reference posterior as a function of the number of Monte Carlo samples used per gradient, for each of the five optimization algorithms tested. The first and second row show the amplitude for centric and acentric reflections, respectively. The third and fourth rows show the posterior width for centric and for acentric reflections, respectively. The two amplitude rows share a vertical scale, and so do the two posterior-width rows. Lines are medians over ten replicates and bands span the interquartile range.
| error (%) | error (%) | for | |||||
|---|---|---|---|---|---|---|---|
| Algorithm | Estimator | centric | acentric | centric | acentric | centric | acentric |
| SGD | Explicit | 3.91 | 2.33 | 2.64 | 2.83 | 0.99417 | 0.99539 |
| Implicit | 3.91 | 2.33 | 2.64 | 2.83 | 0.99417 | 0.99539 | |
| SGD + mom. | Explicit | 2.10 | 0.91 | 2.85 | 2.60 | 0.99896 | 0.99924 |
| Implicit | 2.10 | 0.91 | 2.85 | 2.60 | 0.99896 | 0.99924 | |
| RMSprop | Explicit | 0.69 | 0.33 | 2.42 | 1.67 | 0.99975 | 0.99992 |
Table S2: Recovery of the reference posterior at , for every optimization algorithm and both estimators. Errors are mean absolute percentage errors and correlations ( ) are Pearson coefficients against the reference amplitude. Each value is the median over ten replicates.
Figure S4: Ratio of implicit to explicit gradient variance, plotted against the observed signal-to-noise (SNR) of the reflection. The ratio is measured at the initialization and at the converged parameters of an Adam fit after steps at . The top row shows the variance ratio for the location-derivative, and the bottom row shows the variance ratio for the scale-derivative. The ratio is shown separately for the centric reflections (left column) and the acentric reflections (right column). Each point is a median over reflections and ten replicates, and the bands span the interquartile range within each bin. The axis is divided into equal-width bins between and .
Figure S5: Ratio of implicit to explicit gradient variance at the initialization parameters, and at each algorithm’s own converged parameters after steps at . The top row is the centric location derivative and the bottom row is the centric scale derivative. The initialization parameters are the same in all cases, so the curve is identical across each row. Points are medians over reflections and ten replicates and bands span the interquartile range within each bin. The SNR axis is divided into equal-width bins between and . The three lowest bins hold nine reflections each, and the fullest hold about three hundred.
Figure S6: Centric amplitude error and objective against training step, one column per optimization algorithm, at Monte Carlo samples per gradient. Top, mean absolute percentage error against the reference posterior; bottom, the negative ELBO. Lines are medians over ten replicates and bands span the interquartile range. Every algorithm drives the objective to within of the same value, so the fits agree closely on the optimum they approach. Stochastic gradient descent’s objective is still descending at the end of training, so it has not converged.