Structurally Separated DAG Learning with Multi-Scale Normalized Closure
Organizations: Arizona State University
Abstract
Continuous DAG learning commonly enforces acyclicity through scalar functions of a weighted adjacency that aggregate evidence from weighted closed walks. We add a complementary layer: matrix-valued closure logits---explicit multi-hop structural relation scores for each ordered pair---supervised alongside data fit, sparsity, and a DAGMA-style barrier on the weighted adjacency. The distinction is not scalarization of the overall objective but how path evidence is represented and exposed before aggregation. SMNC (scale-normalized multi-scale closure) builds these scores on a structural gate by repeated squaring in log space and divides each scale by its path-length budget, counteracting product-style attenuation along long cycles; diagonal entries support node-wise cycle supervision and off-diagonals optional pairwise reachability cues. In the linear model, a bounded structure--magnitude factorization ties strong effective edges to strong gates; a nonlinear induced gate offers another construction with the same intent. For sufficiently strong cycles, normalized logits admit a length-independent lower bound; under a unique-active-cycle assumption, the hard final-scale readout yields a polynomial log-gate gradient factor rather than exponential product-style decay. On cue-free, unstandardized Gaussian linear and nonlinear ER benchmarks, these models improve over DAGMA, run with default hyperparameters at a reduced budget, on denser graphs and are close to it on sparse ones.
Figures & tables
| Gaussian linear SEMs | Gaussian nonlinear SEMs | |||||||
|---|---|---|---|---|---|---|---|---|
| bounded WG + SMNC | DAGMA | induced + SMNC | DAGMA | |||||
| ER | dSHD | F1-dir | dSHD | F1-dir | dSHD | F1-dir | dSHD | F1-dir |
| 2 | 2.20 3.19 | 0.99 0.01 | 2.00 3.08 | 0.99 0.01 | 69.80 23.86 | 0.83 0.05 | 68.80 23.53 | 0.82 0.06 |
| 4 | 22.80 17.05 | 0.97 0.02 | 48.00 41.36 | 0.94 0.05 | 85.40 11.28 | 0.89 0.01 | 116.60 8.08 | 0.84 0.01 |
| 6 | 20.80 15.01 | 0.98 0.01 | 88.20 47.33 | 0.93 0.04 | 127.00 23.40 | 0.89 0.02 | 204.20 6.02 | 0.80 0.01 |
| 7 | 58.80 22.17 | 0.96 0.01 | 102.20 47.01 | 0.93 0.03 | 173.60 28.08 | 0.87 0.02 | 290.60 12.66 | 0.75 0.02 |
| Configuration | Hard mask | Soft targets pos./neg. | dSHD | F1-dir |
|---|---|---|---|---|
| Nonlinear backbone (cycle off) | No | – | 24 | 0.250 |
| Induced + SMNC, no cues | No | – | 14 | 0.533 |
| SMNC + hard exclusion | Yes | – | 14 | 0.500 |
| SMNC + positive-only soft cues | No | 10 | 0.722 | |
| SMNC + hard + two-sided soft cues | Yes | 6 | 0.833 |
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
| 4 | 1.000 | |||
| 8 | 1.523 | |||
| 16 | 5.429 | |||
| 24 | 34.513 | |||
| 32 | 610.845 |
| Log-det only | + raw | + SMNC | (%) | |
|---|---|---|---|---|
| 4 | 0.1814 | 0.1605 | 99.00 | |
| 8 | 0.1966 | 0.1471 | 98.77 | |
| 16 | 0.1999 | 0.1045 | 97.77 | |
| 24 | 0.2000 | 0.1762 | 97.01 | |
| 32 | 0.2000 | 0.1981 | 96.42 |
| First break of both cycles | (%) | |||||
| Log-det only | + raw | + SMNC | Raw | SMNC | ||
| 60 | 24 | – | 65 | 296 | 99.88 | 98.12 |
| 60 | 32 | – | – | 296 | 98.69 | 98.01 |
| 60 | 40 | – | – | 296 | 99.54 | 97.96 |
| 60 | 48 | – | – | 296 | 99.91 | 97.92 |
| 80 | 24 | – | 43 | 124 | 99.93 | 98.00 |
| Log-det only | + raw | + SMNC | |
|---|---|---|---|
| 4 | – | 1.0 | 0.6 |
| 8 | – | 1.0 | 0.6 |
| 16 | 0.6 | 0.7 | 1.0 |
| 24 | 0.6 | 0.6 | 1.0 |
| 32 | 0.6 | 0.6 | 1.0 |
| SEM | ER | SMNC | DAGMA | NOTEARS | GOLEM | COSMO | SDCD | |
|---|---|---|---|---|---|---|---|---|
| Linear | 2 | 200 | 2.20 3.19 | 2.00 3.08 | 42.60 10.53 | 285.00 148.04 | 112.80 19.46 | 236.20 17.84 |
| 4 | 400 | 22.80 17.05 | 48.00 41.36 | 188.60 32.79 | 209.80 73.46 | 218.80 71.45 | 444.00 37.25 | |
| 5 | 500 | 16.80 6.30 | 42.20 16.86 | 247.60 21.96 | 261.20 81.94 | 259.20 64.40 | 545.00 39.52 | |
| 6 | 600 | 20.80 15.01 | 88.20 47.33 | 411.00 83.81 | 529.40 173.66 | 324.40 77.75 | 682.60 64.06 | |
| 7 | 700 | 58.80 22.17 | 102.20 47.01 | 637.40 92.05 | 944.20 72.19 | 424.00 89.53 | 849.40 32.16 | |
| 8 | 800 | 248.40 103.58 | 376.80 206.54 | 1170.80 130.44 | 1136.80 133.16 | 653.40 170.38 | 1296.40 227.89 |
| SEM | ER | SMNC | DAGMA | NOTEARS | GOLEM | COSMO | SDCD | |
|---|---|---|---|---|---|---|---|---|
| Linear | 2 | 200 | 0.99 0.01 | 0.99 0.01 | 0.88 0.03 | 0.58 0.15 | 0.74 0.04 | 0.56 0.02 |
| 4 | 400 | 0.97 0.02 | 0.94 0.05 | 0.72 0.05 | 0.78 0.06 | 0.75 0.07 | 0.47 0.05 | |
| 5 | 500 | 0.98 0.01 | 0.96 0.02 | 0.70 0.02 | 0.78 0.06 | 0.76 0.05 | 0.43 0.04 | |
| 6 | 600 | 0.98 0.01 | 0.93 0.04 | 0.60 0.07 | 0.66 0.09 | 0.76 0.05 | 0.29 0.10 | |
| 7 | 700 | 0.96 0.01 | 0.93 0.03 | 0.51 0.05 | 0.49 0.05 | 0.73 0.05 | 0.09 0.04 | |
| 8 | 800 | 0.84 0.06 | 0.78 0.11 | 0.32 0.05 | 0.21 0.06 | 0.65 0.08 | 0.02 0.01 |
| SEM | ER | SMNC | DAGMA | NOTEARS | GOLEM | COSMO | SDCD | |
|---|---|---|---|---|---|---|---|---|
| Linear | 2 | 200 | 1.60 2.19 | 2.00 3.08 | 48.80 13.01 | 94.00 38.03 | 96.00 17.32 | 155.20 15.53 |
| 4 | 400 | 20.00 14.35 | 51.60 42.76 | 227.60 44.91 | 110.00 30.17 | 183.20 48.67 | 424.00 44.07 | |
| 5 | 500 | 19.20 10.16 | 43.40 19.36 | 316.00 25.69 | 153.60 42.27 | 221.60 60.60 | 580.00 35.97 | |
| 6 | 600 | 21.20 16.41 | 88.40 46.40 | 498.40 73.59 | 299.20 114.88 | 280.40 57.70 | 882.80 112.81 | |
| 7 | 700 | 49.20 10.26 | 101.80 46.14 | 686.80 72.30 | 656.80 102.22 | 370.80 66.28 | 1280.00 36.33 | |
| 8 | 800 | 246.00 99.12 | 348.60 174.32 | 1109.60 98.00 | 1231.20 63.70 | 551.60 116.66 | 1553.20 20.43 |
| Method | Steps used (vs. own default) | Precision | RAW col. | Notes & hyperparameters |
|---|---|---|---|---|
| DAGMA (linear) | 90,000 (4 20k warm + 10k final) vs. 180,000 (4 30k + 60k) | float64 | thresh. | ; , lr are the vendored script’s own defaults, unmodified; only the iteration budget was reduced. a |
| DAGMA (nonlinear) | 40,000 (3 10k warm + 10k final) vs. 230,000 (3 50k + 80k) | float64 | thresh. | MLP with one hidden layer of width 10, as in DAGMA’s own nonlinear example; , , lr , , , factor , , , all equal to the defaults of DAGMA’s nonlinear fit ; only the iteration budget was reduced. d |
| NOTEARS (DAGrad) | 5 1,000 Adam steps vs. up to 100 outer L-BFGS-B solves in NOTEARS’s own code | float64 | thresh. | DAGrad’s PyTorch implementation of the NOTEARS objective with the exponential penalty and Adam, not the original SciPy L-BFGS-B solver. Linear: loss, , lr , , . Nonlinear: a one-hidden-layer sigmoid MLP of width 40, , same budget and threshold. These are the runner’s defaults, the same for every dataset. e |
| GOLEM | 50,000 vs. 100,000 (half the script’s default) | float32 (script default) | proc. | RAW columns report GOLEM’s post-hoc pruned (“processed”) output. Its raw output is dense (e.g. directed-edgewise SHD for one , ER 4 run, vs. , the table’s processed-output mean over 5 datasets at that setting). , are the script’s own equal-variance defaults, unmodified; only the iteration budget was reduced. b |
| COSMO | 2,000 epochs, batch 64, at one fixed config; n/a own default (grid-search method) | float64 | thresh. | evaluated at one fixed hyperparameter point : the geometric midpoint of each parameter’s published search range (lr , -reg , -reg , priority-reg , temperature , shift ), with threshold . COSMO’s own documented protocol runs a grid search per dataset and keeps the configuration with the best validation ROC-AUC. c |
| No cues | Hard-only | Soft-only | Hard+Soft | ||||||
| ER | dSHD | F1-dir | dSHD | F1-dir | dSHD | F1-dir | dSHD | F1-dir | |
| (a) Unstandardized inputs | |||||||||
| 0.3 | 2 | 65.8 22.3 | 0.839 0.051 | 61.8 15.8 | 0.847 0.040 | 80.6 24.6 | 0.817 0.051 | 75.0 16.4 | 0.827 0.038 |
| 4 | 87.6 12.0 | 0.889 0.014 | 81.4 13.9 | 0.897 0.017 | 88.0 11.4 | 0.891 0.012 | 84.8 16.6 | 0.895 0.020 | |
| 5 | 117.2 31.5 | 0.876 0.034 | 102.0 27.5 | 0.892 0.029 | 116.2 31.2 | 0.879 0.033 | 100.2 33.1 | 0.896 0.035 | |
| 6 | 127.6 21.0 | 0.886 0.020 | 115.8 24.7 | 0.897 0.024 | 128.0 20.6 | 0.888 0.020 | 113.6 23.7 | 0.901 0.022 | |
Graph Structure Learning
Graph structure learning infers or adapts network connections from data for statistical discovery and downstream prediction.
See all 144 papers