Diffusion Tree Sampling for Inference Time Adaptation of Crystal Generative Models
Organizations: McGill University Mila - Quebec Artificial Intelligence Institute
Abstract
Diffusion-based generative models have become strong priors over plausible crystal structures, but turning that prior into promising candidates requires steering toward several conflicting properties at once. Fine-tuning is expensive and produces a new model for every new objective, while differentiable guidance is not applicable to many real-world tasks. We instead adapt these generative models at inference time using Diffusion Tree Sampling (DTS), a Monte Carlo tree search procedure over the denoising trajectory. We make two technical contributions on top of DTS that are particularly relevant for AI-for-science applications. First, we show how to use existing property-conditional generative models (e.g., models fine-tuned for classifier-free guidance) as guided proposals inside DTS, without distorting the target distribution. Second, we extend DTS to multi-objective sampling, so that a single search produces samples spanning an entire Pareto front. We instantiate both adaptations on de novo crystal generation: improving the stability of crystals generated by different models, using pretrained conditional models to generate crystals belonging to rare space groups or with high bulk modulus, and generating crystals along a Pareto front of high dielectric constant and high bandgap materials. Across these tasks, DTS-based inference-time adaptation successfully improves model outputs without modifying the models themselves.
Figures & tables
| Reward | # Structures | # SUN | % Stable | SUN Unique Systems | Avg E /atom | |
| 0.1 | Null | 5050 29 | 1311 39 | 66.5 1.1 | 912.3 33.7 | 0.091 0.004 |
| 5226 42 | 1432 73 | 75.7 0.4 | 966 12 | 0.073 0.000 | ||
| 0.3 | Null | 3500 0 | 1095 55 | 65.3 1.5 | 904.3 29.4 | 0.1 0.004 |
| 3500 0 | 1209 31 | 70.2 0.9 | 971.3 29.0 | 0.085 0.002 | ||
| 0.5 | Null | 3000 0 | 974 27 | 66.6 1.4 | 894.7 27.4 | 0.090 0.003 |
| 3000 0 | 1087 23 | 68.6 1.3 | 979.7 15.2 | 0.087 0.002 |
| Method | Structures | Stability (%) | Uniqueness (%) | Novelty (%) | SUN (%) | Mean |
| Matched 1M NFE budget | ||||||
| Best-of- | 1,000 | |||||
| SMC (2 resamples) | 554 | |||||
| SMC (4 resamples) | 332 | |||||
| SVDD-PM / TreeG-SC | 725 | |||||
| TreeG-SD | 1,000 | |||||
| Method | HV | GD+ | R2 | ||||||
| Training set | 27,136 | 20 | 4,180 | 1,920 | 526 | 109 | 0.309 | 0.000 | 0.149 |
| MatterGen (cond.) | 665 0 | 14 1 | 262 3 | 138 6 | 28 2 | 4 0 | 0.118 0.011 | 0.098 0.024 | 0.292 0.002 |
| MO-DTS | 645 40 | 16 1 | 402 54 | 234 27 | 67 12 | 17 4 | 0.173 0.057 | 0.054 0.015 | 0.338 0.040 |
| Model | DTS | IS | SUN crystals within 30 GPa | Bulk MAE to 300 GPa | Near-target density to training |
| Unconditional | ✓ | ✗ | 237 | 108.23 | 2.80 |
| Density-Conditioned | ✗ | ✗ | 61 | 131.02 | 4.72 |
| Density-Conditioned | ✓ | ✗ | 302 | 62.82 | 4.31 |
| Density-Conditioned | ✓ | ✓ | 260 | 70.33 | 2.15 |
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
| Method | Reward | Time |
| DTS | - | 5.9 h |
| DTS | 0 | 5.2 h |
| Best-of-N baseline | 0 | 1.5 h |
| SMC | - | 2.2 h |
| Method | Reward | Time |
| DTS | - | 3.2 h |
| DTS | 0 | 3.0 h |
| Best-of-N baseline | 0 | 2.7 h |
| Method | Selection | # SUN | Stable (%) | SUN unique systems | Avg. /atom | |
| — | MatterGen baseline | All 2500 | ||||
| 0.1 | DTS (Null-Reward) | First- | ||||
| 0.3 | DTS (Null-Reward) | First- | ||||
| 0.5 | DTS (Null-Reward) | First- | ||||
| 0.1 | DTS + reward | First- | ||||
| 0.3 | DTS + reward | First- |
| Selection | Method | Stability (%) | SUN (%) | Mean |
| Matched 1,000 output candidates | ||||
| First 1,000 | Best-of- | |||
| DTS (Null-Reward) | ||||
| DTS | ||||
| TreeG-SD | ||||
| Lowest 1,000 | Best-of- | |||
| Method | HV | GD+ | IGD+ | R2 | ||||||
| Training set | 27,136 | 20 | 4,180 | 1,920 | 526 | 109 | 0.309 | 0.000 | 0.000 | 0.149 |
| MatterGen (cond.) | 665 0 | 14 1 | 262 3 | 138 6 | 28 2 | 4 0 | 0.118 0.011 | 0.098 0.024 | 0.143 0.005 | 0.292 0.002 |
| DTS-C =0.5 | 645 40 | 16 1 | 402 54 | 234 27 | 67 12 | 17 4 | 0.173 0.057 | 0.054 0.015 | 0.180 0.045 | 0.338 0.040 |
| DTS-C =0.3 | 674 44 | 15 3 | 403 31 | 231 23 | 68 6 | 19 1 | 0.236 0.052 | 0.054 0.012 | 0.120 0.043 | 0.287 0.037 |
| DTS-C =0.1 | 804 46 | 11 1 | 455 35 | 242 49 | 59 21 | 13 5 | 0.155 0.032 | 0.030 0.025 | 0.214 0.028 | 0.375 0.024 |
| DTS-U =0.1 | 1,433 | 12 | 198 | 60 | 9 | 3 | 0.061 | 0.180 | 0.249 | 0.369 |
| Method | # SG 204 and SUN |
| DTS with IS | |
| DTS without IS |
| Method | SUN Structures | HV | GD+ | R2 | ||||
| Best-of- | ||||||||
| DTS, no IS | ||||||||
| DTS, IS |
| Timesteps | # Branching nodes | Mean # children | Median | Log weight 5% | Median log weight | Log weight 95% |
| 1000–901 | – | 100.00 | 0.348 | 0.03 | ||
| 900–801 | 169 | 5.29 | 0.763 | 0.05 | ||
| 800–701 | 304 | 5.30 | 0.688 | 0.07 | ||
| 700–601 | 477 | 3.36 | 0.501 | 0.36 | ||
| 600–501 | 188 | 2.88 | 0.500 | 1.50 | ||
| 500–401 | 21 | 2.76 | 0.500 | 0.71 |
| Timesteps | # Branching nodes | Mean # children | Median | Log weight 5% | Median log weight | Log weight 95% |
| 1000–901 | 3 | 73.00 | 0.508 | 0.002 | ||
| 900–801 | 210 | 6.29 | 0.709 | 0.002 | ||
| 800–701 | 481 | 3.59 | 0.795 | 0.003 | ||
| 700–601 | 284 | 3.33 | 0.791 | 0.087 | ||
| 600–501 | 130 | 3.81 | 0.519 | 0.482 | ||
| 500–401 | 76 | 3.78 | 0.547 | 0.495 |
| Timesteps | # Branching nodes | Median | Log weight 5% | Median log weight | Log weight 95% |
| 1000–900 | 48 | 0.740 | 0.0133 | ||
| 900–800 | 122 | 0.816 | 0.0138 | ||
| 800–700 | 244 | 0.826 | 0.0167 | ||
| 700–600 | 314 | 0.658 | 0.0742 | ||
| 600–500 | 161 | 0.674 | 0.3683 | ||
| 500–400 | 21 | 0.803 | 0.3521 |
Diffusion Model Sampling
Diffusion model sampling studies how to generate high-quality samples efficiently by designing reverse processes, solvers, and inference-time controls.
See all 92 papers