Offline Materials Optimization with CliqueFlowmer

Jakub Grudzien Kuba; Benjamin Kurt Miller; Sergey Levine; Pieter Abbeel

cs.AIMar 6, 2026

Offline Materials Optimization with CliqueFlowmer

Authors: Jakub Grudzien Kuba, Benjamin Kurt Miller, Sergey Levine, Pieter Abbeel

Organizations: BAIR, UC Berkeley, Berkeley, California, USA · FAIR, Meta, San Francisco, California, USA

Abstract

Recent advances in deep learning inspired neural network-based approaches to computational materials discovery (CMD). A plethora of problems in this field involve finding materials that optimize a target property. Nevertheless, the increasingly popular generative modeling methods are ineffective at boldly exploring attractive regions of the materials space due to their maximum likelihood training. In this work, we offer an alternative CMD technique based on offline model-based optimization (MBO) that fuses direct optimization of a target material property into generation. To that end, we introduce a domain-specific model, dubbed CliqueFlowmer, that incorporates recent advances of clique-based MBO into transformer and flow generation. We validate this model's optimization abilities and show that materials it produces strongly outperform those from generative baselines. To support specialized materials discovery applications and broader interdisciplinary research, we release our code, model weights, and additional project resources at https://github.com/znowu/CliqueFlowmer, https://colab.research.google.com/drive/1usUg7zezFkcYHlm2MdYwZUNJXf_YkWnY?usp=sharing, and https://x.com/kuba_AI/status/2033382617442345321.

Figures & tables

Figure 1: The unit cell of a hypothetical material. The cell has a shape of a parallelepiped determined by three axes, $\vec{{\textnormal{a}}}$ , $\vec{{\textnormal{b}}}$ , and $\vec{{\textnormal{c}}}$ . The angles between the axes are $\text{ang}(\vec{{\textnormal{b}}},\vec{{\textnormal{c}}})=\alpha$ , $\text{ang}(\vec{{\textnormal{c}}},\vec{{\textnormal{a}}})=\beta$ , $\text{ang}(\vec{{\textnormal{a}}},\vec{{\textnormal{b}}})=\gamma$ . In this cell, there are five atoms, whose type sequence is ${\mathbf{a}}=[\text{N, Cl, C, O, S}]$ .

Figure 2: Computational materials discovery through MBO with CliqueFlowmer. Known materials are encoded, with a transformer encoder and an attention-based pooling layer, into a fixed-dimensional latent space. The latent variable admits a clique decomposition, with respect to the target property, where each clique contributes additively to the target property. The representations are optimized, with evolution strategies-based gradient optimization. Atom types are then decoded with an autoregressive transformer and beam search. The flow model reconstructs the material’s geometry from the atom sequence and the latent representation by which it is conditioned via cross-attention.

Panel of Figure 3 — Figure 3: Examples of materials optimized by CliqueFlowmer for band gap minimization. Each figure shows a material’s unit cell, and the caption describes its composition. We can observe that the optimized materials often include Dysprosium (Dy) and Silicon (Si) as components. The corresponding band gap values are $0.06$ , $0.01$ , $0.14$ , $0.02$ , and $0.03$ .

Metric	CrystalFormer	DiffCSP	DiffCSP++	MatterGen	CliqueFlowmer	-Top
E ${}_{\text{form}}$ ( $\downarrow$ )	0.71	0.59	0.65	0.60	-0.81	-0.99
Stable ( $\uparrow$ )	14.1	19.2	19.8	18.5	14.5	13.6
S.U.N. ( $\uparrow$ )	12.8	18.6	18.5	17.6	14.4	13.4
E ${}_{\text{form}}$ / S.U.N. ( $\downarrow$ )	1.06	1.02	1.10	0.98	-0.65	-1.06
Band Gap ( $\downarrow$ )	0.52	0.63	0.48	0.57	0.03	0.07
Stable ( $\uparrow$ )	14.1	19.2	19.8	18.5	61.3	69.4

Table 1: Target property values (Formation Energy and Band Gap), stability, and S.U.N. rate (percentage) across generative model baselines and MBO with our model. Best results in each row are in bold ; second-best are underlined . CliqueFlowmer and CliqueFlowmer -Top reduce formation energy from the initial sample average of 0.46 to -0.81 and -0.99, respectively. The materials also achieve competitive stability and S.U.N. rates. Furthermore, among discovered S.U.N. materials, they achieve even lower property values (Eform / S.U.N) than the baselines. CliqueFlowmer and CliqueFlowmer-Top also reduce the band gap from the sample average of 0.57 to 0.03 and 0.07, respectively, while achieving superior stability and S.U.N. rates and property values among S.U.N. materials (Band Gap / S.U.N.). Thus, our models solve the optimization problem in Equation ( 1 ) while attaining high stability, novelty, and uniqueness of proposed materials. More metrics in Appendix A.1 .

Figure 4: The distribution of the property value (formation energy) among discovered materials. We compare CliqueFlowmer (blue) to materials from MP-20 dataset (green, left ) and to those generated by MatterGen (purple, right ). The values of materials optimized by CliqueFlowmer concentrate around negative values (close to

E_{\text{form}}=-1

) and have a significant tail that goes even lower than that. Meanwhile, due to the data-fitting nature of MatterGen’s training, its property value distribution is similar to that of the MP-20 data, and is concentrated around positive values (close to

E_{\text{form}}=1

). For more histograms, see Appendix 9 .

Panel of Figure 5 — Figure 5: Latent interpolation between two materials. We linearly interpolate $z^{(t)}=(1-t)z^{(0)}+tz^{(1)}$ between As 3 Rh and MgInBr 3 and decode each $z^{(t)}$ . The unit cells evolve smoothly in the cell shape, atom positions, and atom count. More interpolation visualizations in Appendix A.5 .

Figure 6: Comparison of gradient estimators—back-propagation (BP) vs. evolution strategies (ES), and weight decay. We plot the average change of the target property due to optimization, over 100 materials. Each algorithm performed 1000 steps, which was sufficient to reject back-propagation as divergent.

Figure 7: Average

f(\mathsf{M})

(we used formation energy) with standard error of the mean (SEM) over the course of linear interpolation. The curve was smoothed out with the Gaussian kernel. The value tends to be higher along the interpolation trajectory.

Panel of Figure 8 — Figure 8: Latent interpolation of cliques between As 3 Rh and MgInBr 3 —2D visualization of primitive cells. Top: The studied clique does not significantly affect composition or the unit cell shape, but it does affect atom position and atom count slightly. In particular, it affects positions of two of the four Rh atoms that, eventually, from the bottom of the cell move to its top. Bottom: The clique does not alter the material geometry much, but it does significantly affect its composition by injecting Vanadium (V) atoms.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Metric	CrystalFormer	DiffCSP	DiffCSP++	MatterGen	CliqueFlowmer	-Top
E ${}_{\text{form}}$ ( $\downarrow$ )	0.71	0.59	0.65	0.60	-0.81	-0.99
Metastable ( $\uparrow$ )	47.1	55.5	50.2	57.0	26.0	24.1
Stable ( $\uparrow$ )	14.1	19.2	19.8	18.5	14.5	13.6
Unique ( $\uparrow$ )	99.6	99.8	99.8	100	99.5	99.6
Novel ( $\uparrow$ )	70.7	96.9	83.8	86.0	99.5	99.6
S.U.N. ( $\uparrow$ )	12.8	18.6	18.5	17.6	14.4	13.4

Appendix

Table 2: Target property values (Formation Energy and Band Gap), metastability, stability, uniqueness, novelty, and S.U.N. rate (percentage) across generative model baselines and MBO with our model. Best results in each row are in bold ; second-best are underlined . We observe that, apart from superior property values, in each task, CliqueFlowmer and CliqueFlowmer -Top attain a very high (near or equal to 100%) novelty rate. This result satisfies our motivation for development of CMD methods that incorporate MBO and explore the materials space more aggressively than generative baselines.

Metric	CrystalFormer	DiffCSP	DiffCSP++	MatterGen	CliqueFlowmer	-Top
Eform ( $\downarrow$ )	-0.59	-0.56	-0.47	-0.68	-2.43	-2.87
S.U.N. ⋆ ( $\uparrow$ )	47.00	27.00	33.00	38.00	12.00	4.00
S.U.N. ( $\uparrow$ )	15.00	2.00	7.00	6.00	2.00	0.00
Eform / S.U.N. ⋆ ( $\downarrow$ )	-0.62	-0.64	-0.76	-0.84	-0.99	-2.79
Band Gap ( $\downarrow$ )	0.81	0.86	0.77	1.00	0.20	0.17
S.U.N. ⋆ ( $\uparrow$ )	47.00	27.00	33.00	38.00	2.00	3.00

Appendix

Table 3: Target property values (formation energy and band gap) and the S.U.N. rate, computed with DFT, for 100 structures with the lowest energy above hull according to M3GNet. We write S.U.N. ⋆ when we count metastable materials as stable, and S.U.N. when we only us strictly stable materials.

Metric	Base	Base-Top	Early	Early-Top
Eform ( $\downarrow$ )	-0.81	-0.99	-0.34	-1.22
S.U.N. ( $\uparrow$ )	14.4	13.4	8.3	7.6
Eform / S.U.N. ( $\downarrow$ )	-0.65	-1.06	-0.46	-1.42
Band Gap ( $\downarrow$ )	0.03	0.07	0.38	0.25
S.U.N. ( $\uparrow$ )	61.3	69.4	28.4	30.7
Band Gap / S.U.N. ( $\downarrow$ )	0.02	0.05	0.09	0.11

Appendix

Table 4: Comparison, in terms of target properties (formation energy and band gap), S.U.N. rates, and properties per S.U.N. for two settings of the latent-space optimization: the Base one (

2000

ES steps) and the Early -stopped one (

1000

ES steps). The results are calculated with target property oracles and local hulls. Best results in each row are in bold ; second-best are underlined . The Base version consistently works better in terms of average property value, as well as it achieves much higher S.U.N. rates.

Metric	Base	Base-Top	Early	Early-Top
Eform ( $\downarrow$ )	-2.43	-2.87	-2.15	-3.25
S.U.N. ∗ ( $\uparrow$ )	12.00	4.00	21.00	24.00
S.U.N. ( $\uparrow$ )	2.00	0.00	5.00	8.00
Eform / S.U.N. ∗ ( $\downarrow$ )	-0.99	-2.79	-1.93	-1.42
Band Gap ( $\downarrow$ )	0.20	0.17	0.42	0.30
S.U.N. ∗ ( $\uparrow$ )	2.00	3.00	7.00	7.00

Appendix

Table 5: Comparison, in terms of target properties (formation energy and band gap), S.U.N. rates, and properties per S.U.N. for two settings of the latent-space optimization: the Base one (

2000

ES steps) and the Early -stopped one (

1000

ES steps). The results are calculated with DFT and HSE. Best results in each row are in bold ; second-best are underlined . The Base version still works better in terms of average property value but the Early-stopped version achieves much higher S.U.N. rates, highlighting the bias induced by ML oracles, as well as robustness of our methods in property optimization.

Figure 9: The distribution of the property value (band gap) among discovered materials. We compare CliqueFlowmer (blue) to materials from MP-20 dataset (green, left ) and to those generated by MatterGen (purple, right ). For all evaluations, we remove materials with

\Delta_{\text{band}}=0

(our model excels at discovering them without loss of S.U.N. metrics—see Table 1 for the exact results and better interpretability. We visualize the property in the log scale. Materials optimized by CliqueFlowmer have values accumulated near zero while others are more evenly spread out.

Figure 10: Full version of Figure ( 4 ). Our method puts significant weight on

\Delta_{\text{band}}=0

materials.

Figure 11: Wall clock time of material optimization with CliqueFlowmer. The optimization (MBO) step remains negligible, especially as the number of materials increases. The wall clock time is dominated by beam search but it remains linear in the number of structures.

Panel of Figure 12 — Figure 12: Latent interpolation of materials. The materials change smoothly by modifying their unit cell shapes, positions of atoms, as well as atom composition. The majority of the composition changes (removal and arrival of new atom types) happens mainly around the midpoint of the interpolation, between timestep $t=0.25$ and $t=0.75$ .

Figure 13: Intuition behind performance discrepancy between BP and ES. The function approximator ( red ) of the ground truth function ( blue ) has an artifact that is inconsistent with the ground truth. Near that artifact’s location ( gray ), the derivative of the approximator (taken with respect to the red square ) points updating to the right. Meanwhile, exploring the region with ES allows to identify the correct (left) update direction. The predicted values of the perturbed points are, from left to right: 0.99, 1.44, 1.67, 1.71, 1.70, and 1.70

Figure 14: Ablation of the weight decay strength $\lambda$ in the latent-space optimization algorithm. Weight decay $\lambda=0.4$ delivers the strongest compromise between target property values and stability.

Figure 15: Impact of clique decomposition on the target property (formation energy) optimization across different values of ES weight decay. Clique decomposition enables substantially stronger optimization performance over a broad range of weight decay values, whereas the no-clique variant degrades rapidly as regularization increases.

Conditioning	$\omega=0$	$\omega=2$	$\omega=4$
Flat	63	45	32
Cross-attention	71	80	77

Appendix

Table 6: Effect (match ratio in %) of cross-attention on encode–decode consistency.

Panel of Figure 16 — Figure 16: Two ways to model the distribution of lattice lengths, demonstrated on variable $a$ . In the first scheme (left), the length is naively modeled as log-normally distributed. In the second, final scheme (right), the length is scaled down by the cubic root of the atom count, and the canonical length is modeled as a logit-normal variable. The log-normal density ( red ) fits the histogram ( blue ) better in the canonical case (right).

Guidance strength $\omega$	Match ratio (%)
0	71
2	80
4	77

Appendix

Table 7: Effect of classifier-free guidance strength on encode–decode consistency.

Figure 17: The density plot of the lifted logit-normal distribution (

\epsilon=0.2

). Most of the mass is placed on values of

t

in the middle of the range

(0,1)

and a substantial amount near the endpoints.

Category	Parameter	Value	Description
Model	n_cliques	8	Number of latent cliques
	clique_dim	16	Dimensionality of each clique
	knot_dim	1	Overlap between adjacent cliques
	transformer_dim	256	Transformer hidden dimension
	n_blocks	4	Transformer layers
	n_heads	4	Attention heads

Appendix

Table 8: CliqueFlowmer hyperparameters.

Materials Discovery

See all 29 papers

OMatG-flash: An All-Atom Flow Map with Reinforce Adjoint Matching for Scalable Materials Discovery
Sep 22, 2026Thomas Egg, Harry Winston Sullivan, Ellad B. Tadmor +1Flow Matching Crystal Structure Prediction
New York University · University of Minnesota
Topology-Stratified Materials Discovery with A Flow-Based Generative Model
Sep 22, 2026Jingyi Zhou, Oyshee Chowdhury, Noah Oyeniran +1Materials Science Generative Modeling
Department of Aerospace Engineering and Mechanics, University of Alabama, Tuscaloosa, 35487, Alabama, United States. · Alabama Materials Institute, The University of Alabama, Tuscaloosa, 35487, Alabama, United States.
Property-Guided Molecular Generation and Optimization via Latent Flows
Mar 27, 2026Alexander Arjun Lobo, Urvi Awasthi, Leonid ZhukovMolecular Generation Molecular Optimization
BCG X AI Science Institute

Sep 22, 2026J/K move · Enter open · S save

OMatG-flash: An All-Atom Flow Map with Reinforce Adjoint Matching for Scalable Materials Discovery

Thomas Egg, Harry Winston Sullivan, Ellad B. Tadmor, Stefano Martiniani

New York University · University of Minnesota

Flow Matching Crystal Structure Prediction Inference Acceleration Crystal Structure Generation Materials Discovery

Open paper AskPDF arXiv

The discovery of novel inorganic materials drives technological breakthroughs in critical fields such as computing and energy storage. Generative AI has promised to accelerate the materials discovery pipeline, but state-of-the-art flow and diffusion models remain bottlenecked by the cost of proposing candidate materials. To address this, we introduce OMatG-flash, an all-atom flow map for inorganic crystal structure prediction (CSP) and de novo generation (DNG). OMatG-flash is a Pareto-optimal inference engine for materials, sampling candidate materials with an order of magnitude fewer inference steps and less wall-clock time than existing flow and diffusion models while demonstrating benchmark performance on par with the state-of-the-art. To enable post-training fine-tuning we apply Reinforce Adjoint Matching to flow maps, further improving match rates and RMSE on the unconditional CSP task. OMatG-flash showcases the potential of flow maps to accelerate generation of high-quality candidate inorganic materials and demonstrates a step forward in sample throughput necessary for data-hungry materials discovery workflows.

Abstract

Materials Discovery

Explore similar work

OMatG-flash: An All-Atom Flow Map with Reinforce Adjoint Matching for Scalable Materials Discovery

Topology-Stratified Materials Discovery with A Flow-Based Generative Model

Property-Guided Molecular Generation and Optimization via Latent Flows