cs.LGOct 30, 2026

On the Generation Order in GFlowNets

Authors: Timofei Gritsaev, Evgeny Katasonov, Ildus Sadrtdinov, Esmeralda S. Whitammer, Dmitry Vetrov

Organizations: University of Edinburgh

Abstract

Sampling from discrete distributions specified by unnormalised probability functions is a classical statistical problem. GFlowNets address this problem by constructing objects sequentially, adding one discrete component at a time in either a predefined or a learned order. The choice of order determines which conditional distributions the neural network must learn, some of which may be easier to learn than others. Yet whether existing GFlowNet algorithms discover interpretable generation orders and make use of them during training remains unexplored. We introduce controlled problems where generation order is experimentally shown to affect convergence and standard GFlowNet training struggles to discover useful orders. To address this difficulty, we develop Learning-Order GFlowNet, a training recipe that learns interpretable generation orders without order annotations and improves convergence on these problems. Finally, we investigate gradient signal-to-noise ratio (SNR) as a possible explanation for order emergence: orders associated with faster convergence exhibit higher SNR at comparable validation loss, suggesting both a mechanism favouring their emergence and an SNR-based characterisation of a good generation order.

Figures & tables

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Stable GFlowNets with TV Monitoring and Probabilistic Guarantees

    May 3, 2026Zengxiang Lei, Ananth Shreekumar, Jonathan Rosenthal +6Generative Flow NetworksFlow-Based Generative Modeling

  2. Learning a Mixture of GFlowNets

    Oct 6, 2026Tiago da Silva, Amauri H. Souza, Salem LahlouGenerative Flow Networks

  3. Your GFlowNet Secretly Learns an Optimal Transport Plan

    Jun 4, 2026Ian Maksimov, Nikita Morozov, Denis Belomestny +1Generative Flow NetworksFlow-Based Generative Modeling