Continuous Noisy-Loop Models
Organizations: Center for AI Research, VinUniversity Vinhomes Ocean Park, Gia Lam, Hanoi, Vietnam · Intelligent Autonomous Systems, TU Darmstadt Hochschulstrasse 10, 64289 Darmstadt, Germany
Abstract
Efficient learning and inference need not imply a compact probabilistic circuit. We quantify this distinction for continuous noisy readings of agreement between adjacent latent spins on a known square grid with edges. With known margins and bounded Lipschitz channels, minimax squared Hellinger risk is from independent vectors, without a reliability lower bound. A spline estimator reaches this rate in polynomial arithmetic time given a fixed positive reliability bound; an exponential-size mixture removes that requirement. At fixed positive reliability, nonnegative decomposable circuits need nodes for fixed total-variation error below one, even when each leaf uses at most coordinates, . Optimal Kullback--Leibler (KL) approximation error is in either direction with at most nodes and leaf scopes , for fixed and . Node counts exclude leaf computation. For three fixed masks, optimal joint conditional KL has order when . Local cycle identities relate joint divergence to identifiable channel products and separate weak-channel recovery from density estimation. Factorized evidence preserves a planar zero-field Ising model: given local integrals, exact queries cost arithmetic operations and one root extraction.
Figures & tables
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
| Returned / rejected | Maximum error | ||||||
|---|---|---|---|---|---|---|---|
| Graph | Jobs | Determinant | Bonds | Log | Bond | ||
| Forest | 0.25 | 112–32,512 | 6/6 | 12/0 | 12/0 | ||
| Forest | 0.8 | 112–32,512 | 6/6 | 12/0 | 12/0 | ||
| Forest | 0.95 | 112–32,512 | 6/6 | 12/0 | 12/0 | ||
| Forest | 0.9999 | 112–32,512 | 6/6 | 12/0 | 12/0 | ||
| Disjoint squares | 0.25 | 112–32,512 | 6/6 | 12/0 | 12/0 | ||
| Moments | Observable scalar and corner blocks, , input anchor , cap ; all training vectors serve short-cycle calibration and regression. |
|---|---|
| Likelihood, EM | Centered scalar splines, , cap , no anchor constraint; validation selects the checkpoint and a cold or moment start. Likelihood uses projected Adam ( Kingma and Ba, 2015 ) : rate , batch 256, validation every 32 updates, patience 12, at most 10,000 updates. Generalized EM ( Dempster et al., 1977 ) uses full E steps and up to 20 backtracking projected-gradient M-step updates; validation after each iteration, patience six. |
| HCLT | HCLTs ( Liu and Van den Broeck, 2021 ) in PyJuice ( Liu et al., 2024 ) , with Gaussian leaves in probit coordinates; minimum standard deviation ; structure learned from the first 4096 training vectors; widths , full-batch EM or Anemone ( Liu et al., 2026 ) . Batches of 256; Anemone accumulates 4096 vectors per update. Validation each epoch, patience 15, at most 10,000 epochs. |
| GMM | Full covariances in probit coordinates; components for loop targets at , otherwise . Covariance regularization , tolerance , one initialization, at most 200 iterations. |
| MPC | Linear-tree region graph over the midpoint order, Gaussian probit leaves; widths for loop targets and for Gaussian targets. Adam at rate . |
| OSOS | Same region graph, unit-period complex Fourier leaves, landing-Adam optimizer (landing methods: Ablin and Peyré, 2022 ; Ablin et al., 2024 ) at rates ; regularization , safe-step parameter ; widths for loop targets and for Gaussian targets. Validation projects weights by SVD and scores the actual Gram normalizer. |
| Noisy-loop fits | Baselines | Reference | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Target | Mom. | Lik. | EM | HCLT | GMM | MPC | OSOS | Indep. | |
| 24 | Ramp | ||||||||
| 24 | Smooth | ||||||||
| 24 | Gaussian | ||||||||
| 112 | Ramp | ||||||||
| 112 | Smooth | ||||||||
| Table 4 | ||||||
|---|---|---|---|---|---|---|
| Method | Ramp | Smooth | Gaussian | Ramp | Smooth | Gaussian |
| Fixed random 25% of coordinates hidden | ||||||
| Moments | ||||||
| Likelihood | ||||||
| EM | ||||||
| Task | Likelihood | EM | HCLT | GMM |
|---|---|---|---|---|
| Ramp, | ||||
| Smooth, | ||||
| Gaussian, | ||||
| Ramp, | ||||
| Smooth, | ||||
| Gaussian, |
| Task | Mom. corner | Mom. shrunk | Corner lik. | Corner EM |
|---|---|---|---|---|
| Ramp, | ||||
| Smooth, | ||||
| Gaussian, | ||||
| Ramp, | ||||
| Smooth, | ||||
| Gaussian, |
| Tiled risk | Per hidden edge | Uniform | |||||
|---|---|---|---|---|---|---|---|
| Lower | Estimate (SE) | Upper | |||||
| 8 | 24 | 2 | 0.23 | 5.17 (0.03) | 36.10 | 0.2156 | 5.90 (0.03) |
| 4 | 0.23 | 3.00 (0.02) | 12.04 | 0.1249 | |||
| 6 | 0.00 | 2.12 (0.02) | 12.04 | 0.0883 | |||
| 12 | 60 | 2 | 1.06 | 10.85 (0.04) | 90.25 | 0.1809 | 13.00 (0.05) |
| 4 | 0.95 | 8.30 (0.03) | 36.10 | 0.1383 | |||
| KL | TV | ||||||
|---|---|---|---|---|---|---|---|
| estimate | lower bound | estimate | |||||
| 6 | 60 | 5 | 0.180 | 0.150 | 0.0363 | 0.330 | 0.7303 |
| 8 | 112 | 6 | 0.150 | 0.107 | 0.0342 | 0.406 | 0.8587 |
| 12 | 264 | 8 | 0.113 | 0.068 | 0.0266 | 0.468 | 0.9549 |
| 16 | 480 | 8 | 0.113 | 0.050 | 0.0213 | 0.506 | 0.9835 |
| 24 | 1,104 | 10 | 0.090 | 0.065 | 0.0285 | 0.775 | |
| Seeds | Fitted KL | Independence KL | Excess | ||
|---|---|---|---|---|---|
| 24 | 512 | 10/10 | |||
| 2,048 | 10/10 | ||||
| 8,192 | 10/10 | ||||
| 32,768 | 10/10 | ||||
| 112 | 512 | 10/10 | |||
| 2,048 | 10/10 |
Probabilistic Circuits
Probabilistic circuits are structured generative models that enable tractable probabilistic inference while balancing expressivity and computational cost.
See all 21 papers