Deep Reinforcement Learning for Fano Hypersurfaces

Marc Truter

math.AGMar 16, 2026

Deep Reinforcement Learning for Fano Hypersurfaces

Authors: Marc Truter

Abstract

We design a deep reinforcement learning algorithm to explore a high-dimensional integer lattice with sparse rewards, training a feedforward neural network as a dynamic search heuristic to steer exploration toward reward dense regions. We apply this to the discovery of Fano 4-fold hypersurfaces with terminal singularities, objects of central importance in algebraic geometry. Fano varieties with terminal singularities are fundamental building blocks of algebraic varieties, and explicit examples serve as a vital testing ground for the development and generalisation of theory. Despite decades of effort, the combinatorial intractability of the underlying search space has left this classification severely incomplete. Our reinforcement learning approach yields thousands of previously unknown examples, hundreds of which we show are inaccessible to known search methods.

Figures & tables

Figure 1. The $6$ -dimensional dynamic heuristic (deep reinforcement learning) search for terminal Fano $4$ -fold hypersurfaces projected onto $2$ -dimensions. While the quasismooth terminal points are fully classified, the search discovers previously unknown nonquasismooth terminal ones. See Figure 9 for full details.

Figure 2. Flowchart of the fixed heuristic search algorithm.

Figure 3. Flowchart of the dynamic heuristic search algorithm.

Figure 4. Exhaustive search of Fano 4-fold hypersurfaces with terminal singularities. In total $84,733$ terminal examples were found, $7,346$ quasismooth, and $77,387$ nonquasismooth. Each frame shows points in $\mathbb{Z}^{2}$ , obtained by projecting the original $\mathbb{Z}^{6}$ search space onto consecutive coordinate pairs via $(a_{1},\ldots,a_{6})\mapsto(a_{i},a_{i+1})$ .

Figure 5. Classification of $11,617$ quasismooth Fano $4$ -fold hypersurfaces with terminal singularities. Each frame shows points in $\mathbb{Z}^{2}$ , obtained by projecting the original $\mathbb{Z}^{6}$ search space onto consecutive coordinate pairs via $(a_{1},\ldots,a_{6})\mapsto(a_{i},a_{i+1})$ .

Figure 6. The cumulative number of terminal Fano $4$ -fold hypersurfaces found in the exhaustive search against degree.

Panel of Figure 7 — Figure 7. ( 7A ) The real locus of the affine cone of the member of $X_{6}\subset\mathbb{P}(1,2,3)$ whose coefficients are all equal to $1$ , which is quasismooth and hence admits only quotient singularities. ( 7B ) The real locus of the affine cone of the member of $X_{10}\subset\mathbb{P}(1,3,4)$ whose coefficients are all equal to $1$ , which is nonquasismooth; the hyperquotient singularity is visible as the line passing through the origin.

Hyperparameter	Values
MLP Neural Network Layers	(40,)
Activation function	LeakyReLU
LeakyReLU slope	0.01
Optimiser	Adam
Optimiser learning rate	0.001
TD discount factor, $\gamma$	0.2

Table 1. Hyperparameters used in the dynamic heuristic (deep reinforcement learning) search.

Figure 8. Fixed heuristic search. The search found $113,996$ nonquasismooth Fano $4$ -fold hypersurfaces with terminal singularities. Each frame shows points in $\mathbb{Z}^{2}$ , obtained by projecting the original $\mathbb{Z}^{6}$ search space onto consecutive coordinate pairs via $(a_{1},\ldots,a_{6})\mapsto(a_{i},a_{i+1})$ .

Figure 9. Dynamic heuristic (deep reinforcement learning) search. The search found $85,262$ nonquasismooth Fano $4$ -fold hypersurfaces with terminal singularities. Each frame shows points in $\mathbb{Z}^{2}$ , obtained by projecting the original $\mathbb{Z}^{6}$ search space onto consecutive coordinate pairs via $(a_{1},\ldots,a_{6})\mapsto(a_{i},a_{i+1})$ .

Panel of Figure 10 — Figure 10. ( 10A ) A histogram showing the distribution of distances from points found in the fixed but not dynamic search to their closest neighbours. ( 10B ) A histogram showing the distribution of distances from points found in the dynamic but not fixed search to their closest neighbours. The dynamic search finds points at greater distances than the fixed search, corresponding to points that are increasingly inaccessible to the latter. ( 10C ) A graph showing the number of nonquasismooth terminal examples found against the number of steps taken by the search. As expected, the dynamic search finds fewer terminal points in the same number of steps as the fixed one, since more steps are spent in unprofitable territory in order to reach terminal points at greater distances.

RL for Combinatorial Optimization

Reinforcement learning for combinatorial optimization learns sequential decisions and search strategies for difficult discrete problems.

See all 106 papers

Hierarchical Reinforcement Learning for Sparse-Reward Search in Commutative Algebra
Jun 22, 2026Giorgi Butbaia, Paul Orland, Coco Huang +7Hierarchical RL Sparse-Reward RL
Department of Mathematics, California Institute of Technology, Pasadena, CA · Department of Mathematics, Temple University, Philadelphia, PA · Department of Mathematics, University of Kansas, Lawrence, KS +1
Special Lagrangian cones in Deep Learning
Sep 17, 2026Tejas Kotwal, Govind Menon
Institute for Advanced Study, Princeton
Discovering Lattice Reduction Strategies via Self-Play
Jun 13, 2026Mohamed Malhou, Kristin Lauter, Ludovic PerretReinforcement Learning RL for Combinatorial Optimization
FAIR, Meta Superintelligence Labs & Sorbonne Université CNRS, LIP6 Paris, France · EPITA, EPITA Research Lab (LRE) Le Kremlin-Bicêtre, France

Jun 22, 2026J/K move · Enter open · S save

Hierarchical Reinforcement Learning for Sparse-Reward Search in Commutative Algebra

Giorgi Butbaia, Paul Orland, Coco Huang, Davide Passaro+6

Department of Mathematics, California Institute of Technology, Pasadena, CA · Department of Mathematics, Temple University, Philadelphia, PA · Department of Mathematics, University of Kansas, Lawrence, KS+1

Hierarchical RL Sparse-Reward RL

Open paper AskPDF arXiv

Applying machine learning techniques to solving long-standing mathematical conjectures can be particularly challenging due to their extreme reward sparsity. As an illustrative example, we consider Kalai's algebraic Hirsch conjecture and recast the construction of its counterexamples as a sparse-reward reinforcement learning problem on graphs. We propose a constrained options-based HRL framework with an equivariant graph neural network policy, which allows us to learn useful temporal abstractions for this task. We evaluate our approach over a wide range of degrees and demonstrate that it consistently outperforms classical RL algorithms as well as greedy search. By exploiting the hierarchical structure of the problem, we effectively provide a first-of-its-kind application of HRL to a problem in commutative algebra.

Abstract

RL for Combinatorial Optimization

Explore similar work

Hierarchical Reinforcement Learning for Sparse-Reward Search in Commutative Algebra

Special Lagrangian cones in Deep Learning

Discovering Lattice Reduction Strategies via Self-Play