CrystalAR: Autoregressive Modeling for Symmetric Crystal Generation

Ziqiao Meng; Rongsheng Chen; Zhi Zheng; Nikita Kazeev; Zhenkun Wang; Andrey Ustyuzhanin; Wee Sun Lee

cs.AIOct 20, 2026

CrystalAR: Autoregressive Modeling for Symmetric Crystal Generation

Authors: Ziqiao Meng, Rongsheng Chen, Zhi Zheng, Nikita Kazeev, Zhenkun Wang, Andrey Ustyuzhanin, Wee Sun Lee

Organizations: Institute for Functional Intelligent Materials (IFIM), National University of Singapore · School of Computing, National University of Singapore · Southern University of Science and Technology

Abstract

Deep generative modeling has achieved remarkable progress in crystal generation. Given the rich symmetries inherent in crystalline structures, recent work has sought to explicitly incorporate space-group symmetry into generative models, enabling more effective exploration of novel symmetry templates. Yet introducing symmetry templates increases the combinatorial complexity of stoichiometric relationships between atomic species and crystallographic sites, while creating a fundamental tension between flexible exploration of discrete crystallographic templates and faithful realization of these templates as continuous three-dimensional structures. To address these challenges, we propose CrystalAR, a unified autoregressive framework that models Wyckoff symmetry templates and complete crystal structures within a single generative sequence. Specifically, CrystalAR reformulates continuous coordinate generation as a symmetry-conditioned hierarchy of discrete spatial localization decisions, autoregressively refining the positions of all symmetry-inequivalent sites across resolution levels while jointly modeling their spatial configuration within each level. This discrete hierarchical formulation further enables constraint-aware search over both crystallographic templates and geometric realizations. As a result, CrystalAR retains strong symmetry-template generation capability while achieving competitive de novo generation performance and template-free crystal structure prediction (CSP) accuracy. These results suggest that symmetry-template generation can be retained alongside broadly competitive de novo generation performance and competitive CSP without reliance on symmetry templates retrieved from prior data, providing a promising alternative paradigm for symmetry-preserving crystal generation.

Figures & tables

Figure 1: Overview of CrystalAR. Given a symmetry group

G

, CrystalAR autoregressively generates the element-decorated Wyckoff template

T=(W,A)

, lattice parameters

L

, and free Wyckoff coordinates

F

. Each inequivalent site is restricted to its corresponding Wyckoff manifold. The free coordinates are then generated through level-wise coarse-to-fine refinement, and symmetry operations expand them into the complete crystal structure.

Figure 2: Unified Transformer architecture of CrystalAR. A single decoder-only Transformer jointly models the Wyckoff template, lattice, and hierarchical coordinates. Template and lattice tokens follow standard next-token prediction, while coordinates are refined level by level with causal attention across levels and parallel site interaction within each level. Each site query combines its element, Wyckoff constraint, previous region, region center, and refinement level.

Model	Method	Novel Unique	P1 (%)	SG $\chi^{2}\downarrow$	$W_{N_{\mathrm{el}}}\downarrow$	$\mathrm{JSD}_{\mathrm{Wyckoff}}\downarrow$	S.U.N. (%) $\uparrow$	S.S.U.N. (%) $\uparrow$
Model	Method	Templates (#) $\uparrow$	ref = 2.23	SG $\chi^{2}\downarrow$	$W_{N_{\mathrm{el}}}\downarrow$	$\mathrm{JSD}_{\mathrm{Wyckoff}}\downarrow$	S.U.N. (%) $\uparrow$	S.S.U.N. (%) $\uparrow$
Models without Explicit Template Generation
DiffCSP	Diffusion	48	70.66	23.8303	0.3762	0.5185	0.12	0.05
DiffCSP++	Diffusion	5	2.19	0.1049	0.4787	0.0343	0.19	0.19
Symmetry-native Models
WyFormer DiffCSP++	AR + Diffusion	190	1.56	0.2238	0.1087	0.0248	0.13	0.13

Table 2: Symmetry evaluation on MP-20 using 2,500 generated structures per method. We group methods into models with and without explicit Wyckoff-template generation , with symmetry-native models highlighted in red. We evaluate template exploration by the number of novel unique templates, distribution fidelity by P1 ratio, SG

\chi^{2}

W_{N_{\mathrm{el}}}

, and

\mathrm{JSD}_{\mathrm{Wyckoff}}

, and material discovery quality by S.U.N. and S.S.U.N. Symmetry statistics are computed over novel templates. Bold and underline denote the best and second-best results, respectively. CrystalAR (SV) denotes the flexible-size variant that explicitly predicts both the number of atoms and the number of symmetry-inequivalent sites.

Figure 3: Representative S.S.U.N. crystal structures generated by CrystalAR, illustrating diverse chemical compositions, space groups, and crystallographic configurations.

	MP-20		MPTS-52
Model	MR (%) $\uparrow$	RMSE $\downarrow$	MR (%) $\uparrow$	RMSE $\downarrow$
CDVAE (1)	33.90	0.1045	5.34	0.2106
DiffCSP (1)	51.49	0.0631	12.19	0.1786
FlowMM (1)	61.39	0.0566	17.54	0.1726
CrystalFlow (1)	62.02	0.0710	21.00	0.1613
OMatG (1)	63.75	0.0720	25.15	0.1931

Table 3: Crystal structure prediction on MP-20 and MPTS-52 under single-hypothesis (1) and multi-hypothesis (20) generation. We evaluate structure recovery by match rate (MR,

\uparrow

) and geometric accuracy by RMSE (

\downarrow

). CrystalAR performs template-free CSP without supplying a retrieved Wyckoff template. Bold denotes the best result in each column, while underline denotes the best single-hypothesis result.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Generative Model	Unified	Symmetry	Template-Gen	Template-free CSP	Inter-Geom
CDVAE ( Xie et al., 2022 )	VAE+Diff.	✓			✓	✓
DiffCSP ( Jiao et al., 2023 )	Diff.	✓			✓	✓
MatterGen ( Zeni et al., 2023 )	Diff.	✓			✓	✓
KLDM ( Cornet et al., 2025 )	Diff.	✓			✓	✓
ADiT ( Joshi et al., 2025 )	Diff.	✓				✓
FlowMM ( Miller et al., 2024 )	Flow	✓			✓	✓

Appendix

Table 4: “Generative Model” denotes the underlying generative modeling backbone employed by each method. “Unified” indicates whether the model is trained and performs inference over the complete crystal geometry in an end-to-end fashion using a single unified model. “Symmetry” specifies whether Wyckoff symmetry is explicitly modeled at the level of individual atomic sites. “Template-Gen” indicates whether Wyckoff positions are generated by the model itself, rather than retrieved from predefined symmetry templates or sampled from the empirical distribution of the training data. “Template-Free CSP” indicates whether the method supports crystal structure prediction without template retrieval. “Inter-Geom” indicates whether the model explicitly and effectively captures interatomic geometry. “-” denotes that the item is not applicable.

Setting	De novo	CSP: MP-20	CSP: MPTS-52
Parameters (M)	32.804	33.968	91.807
Hidden width	384	384	512
Template/coordinate layers	5/5	5/5	8/8
Attention heads	6	6	8
Feed-forward width	1,536	1,536	2,048
Lattice-encoder layers	4	4	6

Appendix

Table 5: Configurations of the three models used for the reported results.

Count policy	Search	Valid	Unique	Novel	Stable	S.U.N.
Empirical prior (CrystalAR)	Sampling	64.40	62.24	18.76	5.44	0.20 (5)
Empirical prior (CrystalAR)	SGBS	91.44	89.08	42.76	6.00	0.28 (7)
Learned STOP	Sampling	59.96	58.72	21.72	3.48	0.16 (4)
Learned STOP	SGBS	90.12	88.64	48.92	3.52	0.16 (4)
Composition prefix	SGBS	91.44	89.52	41.32	5.40	0.20 (5)

Appendix

Table 6: Count prior and search ablation for de novo generation. All entries are percentages over 2,500 attempts; S.U.N. counts are shown in parentheses.

Configuration	Valid	Unique	Novel	Stable	S.U.N.
Learned STOP reference	90.12	88.64	48.92	3.52	0.16 (4)
Full-cell AR	49.12	48.88	35.04	0.72	0.00 (0)
Wyckoff-letter coordinates	89.80	88.08	51.36	4.16	0.08 (2)
Flat 512-bin coordinates	68.16	66.48	29.12	4.24	0.00 (0)
Continuous coordinates	41.64	40.84	17.12	1.72	0.08 (2)
No periodic encoding	89.96	88.04	50.00	4.12	0.00 (0)

Appendix

Table 7: Representative representation and module ablations. Rates are percentages over 2,500 attempts; S.U.N. counts are shown in parentheses.

Chemistry intervention	Valid	Unique	Novel	Stable	S.U.N.
Original elements	88.36	86.00	39.68	5.92	0.28 (7)
Element redraw, $T_{E}=1.0$	87.44	85.24	41.84	5.56	0.04 (1)
Element redraw, $T_{E}=2.5$	78.72	78.60	57.92	1.60	0.12 (3)
$T_{E}=2.5$ with uncapped clearance	78.72	78.60	58.44	0.96	0.12 (3)

Appendix

Table 8: Fixed-skeleton chemistry interventions. All arms use the same generated symmetry skeleton and 2,500 attempts; rates are percentages and S.U.N. counts are shown in parentheses.

Figure 4: Count prior and constrained search. The panels show validity, stability, and S.U.N. for sampling and SGBS. CrystalAR uses the empirical count prior; the learned-STOP reference uses a learned stopping policy. All values are computed over 2,500 attempts, and the exact percentages and counts appear in Table 6 .

Figure 5: Novelty–stability trade-off under fixed symmetry support. The panels show novelty, stability, and S.U.N. for the four chemistry interventions in Table 8 . The element sampling temperature changes chemical exploration while the generated symmetry skeleton is held fixed.

Figure 6: Feasible-set supervision improves CSP across adapter capacities. Both panels use 2,000 adaptation updates and 9,046 structures. (A) MR for standard CE and feasible-set supervision, with Wilson 95% confidence intervals. (B) Paired MR gains, with bootstrap 95% confidence intervals; ∗∗∗ indicates

p<0.001

under exact McNemar tests with Holm correction.

Figure 7: Effect of coordinate search and hypothesis count on CSP. (A) MR changes relative to coordinate beam 16 with the distance mask, with paired bootstrap 95% confidence intervals. (B) MR as the number of generated hypotheses increases, with a Wilson 95% confidence band over 9,046 structures.

Figure 8: Visualization. More S.U.N structures generated by CrystalAR.

Crystal Structure Generation

Crystal structure generation uses computational models to create periodic atomic arrangements for predicting and designing materials.

See all 61 papers

Feb 19, 2026

NextCrystal: a Symmetry-Driven Generative Framework for Crystal Structure Prediction

Crystal structure prediction (CSP), which aims to predict the 3D atomic arrangement of a crystal from its composition, is central to materials discovery and mechanistic understanding. Crystal symmetry plays a crucial role in CSP, but given the composition in a unit cell, existing methods either struggle with the NP-hard combinatorial challenge of enforcing symmetry rigorously or rely on retrieving known templates, inherently limiting both physical fidelity and the discovery of genuinely new materials. To address this challenge, we introduce NextCrystal, a symmetry-driven generative framework that employs large language models to encode chemical semantics and directly generate fine-grained Wyckoff site patterns from atomic stoichiometry, eliminating reliance on database lookups. To overcome the combinatorial complexity of site assignments, we incorporate domain knowledge via an efficient, linear-complexity heuristic beam search, rigorously enforcing algebraic consistency between site multiplicities and atomic stoichiometry. By integrating this symmetry-consistent template into a diffusion backbone, the framework constrains the stochastic generative trajectory to a physically plausible geometric manifold. NextCrystal achieves state-of-the-art performance on stability, uniqueness, and novelty (SUN) benchmarks, as well as superior structural matching, establishing a rigorous paradigm for exploring previously unexplored crystallographic space without relying on prior structural templates. As a representative application, first-principles screening of HfO2 candidates generated by NextCrystal identifies a previously unreported dynamically stable Pnma phase, 0.056~eV/atom lower in energy than the conventional high-pressure Pnma phase.

Jinming Mu, Lixin He, Xudong Zhu +1

Laboratory of Quantum Information, University of Science and Technology of China, Hefei 230026, Anhui, China · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei 230088, Anhui, China

Crystal Structure Prediction Crystal Structure Generation

Ask PDF

Aug 13, 2026

Symmetry-Breaking De Novo Crystal Generation via Markovian Jump Diffusion

Generating crystals has recently attracted significant interest due to their broad applications in materials science. However, existing generative models struggle to produce complete crystallographic specifications, limiting their ability to capture global symmetry and structural dependencies. In particular, current state-of-the-art approaches generate crystals only up to site symmetries and rely on sampling space groups from empirical distributions during generation. Inspired by spontaneous symmetry breaking in physics, where crystals break symmetries under external conditions, we propose a novel diffusion-based framework that generates full structure specifications by reversing from the lowest-symmetry priors. Our method leverages a Markovian jump-diffusion process to model these symmetry-breaking dynamics, enabling it to traverse different space groups in a physically motivated manner. Our model, dubbed Symmetry-breaking Crystal Diffusion (SbCD), introduces a principled approach to explicitly incorporate inter-space-group transitions into the generative process. In de novo generation experiments on MP20 and MPTS-52, SbCD outperforms its symmetry-preserving counterpart by a substantial margin, offering a promising perspective for generative modeling of crystalline materials.

Van Khoa Nguyen, Alexandros Kalousis

HES-SO Geneva · University of Geneva

Symmetry Breaking Materials Science Crystal Structure Generation Diffusion Models

Ask PDF

Aug 7, 2026

DynaCrys: Crystal Generation with Dynamic Space-Group Diffusion

The search for new crystalline materials spans an enormous compositional and structural space. Generating candidates in this space requires jointly modeling discrete crystallographic symmetry, elemental composition, and continuous geometry. We introduce DynaCrys, a generative model for crystals in which the space group co-evolves with Wyckoff occupations and elements through a coupled symbolic diffusion process. The structured space-group transitions follow crystallographic group-subgroup relations. As the space group changes, a shared, pretrained symmetry codebook provides both the legality-constrained stochastic decoder and the symmetry-constrained crystal-geometry model with a common representation of the corresponding Wyckoff vocabulary. Across large-scale evaluations using two independent relaxation-and-evaluation engines, DynaCrys achieves best-in-class performance in symmetry-aware discovery of stable, unique, and novel crystals, both overall and under the additional requirement of nontrivial post-relaxation symmetry. It also enables fast sampling while generating structures with consistently low relaxation-induced structural displacements.

Zhuotao Jin, Xiaoyun Wang, Nicholas Brawand +5

NVIDIA · Center for Computational Science and Engineering, Massachusetts Institute of Technology · Department of Materials Science and Engineering, Massachusetts Institute of Technology +4

Crystal Structure Generation Diffusion Models Computational Materials Science

Ask PDF

Abstract

Crystal Structure Generation

Explore similar work

NextCrystal: a Symmetry-Driven Generative Framework for Crystal Structure Prediction

Symmetry-Breaking De Novo Crystal Generation via Markovian Jump Diffusion

DynaCrys: Crystal Generation with Dynamic Space-Group Diffusion