Conformal Graph Prediction with Z-Gromov-Wasserstein Distances
Authors: Gabriel Melo, Thibaut de Saivre, Anna Calissano, Florence d'Alché-Buc
Organizations: LTCI, Télécom Paris, Intistut Polytechnique de Paris, Palaiseau, France · Department of Statistical Science, University College London, London, UK
Supervised graph prediction addresses regression problems where the outputs are structured graphs. Although several approaches exist for graph-valued prediction, principled uncertainty quantification remains limited. We propose a conformal prediction framework for graph-valued outputs, providing distribution-free coverage guarantees in structured output spaces. Our method defines nonconformity via the Z-Gromov-Wasserstein distance, instantiated in practice through Fused Gromov-Wasserstein (FGW), enabling permutation invariant comparison between predicted and candidate graphs. To obtain adaptive prediction sets, we introduce Score Conformalized Quantile Regression (SCQR), an extension of Conformalized Quantile Regression (CQR) to handle complex output spaces such as graph-valued outputs. We evaluate the proposed approach on a synthetic task and a real problem of molecule identification.
Figures & tables
Figure 1: Conformal metabolite prediction set at 90% marginal coverage for molecule identification. The green check-mark denotes the ground-truth.
Figure 2: Z-GW distance
Figure 3: Example of the retrieval on the Coloring dataset. The green check-mark denotes the ground-truth.
Conformal Set Size ↓
Candidate Set Size
Reduction % ↑
Dataset
Model fθ
Method
Emp. Cov. (%)
Mean
Median
Mean
Median
Mean
Median
Empty rate (%) ↓
Coverage 1−α=90%
Colors
Any2Graph Acc: 82%
CP
90.2
4
1
205
223
95.8%
98.9%
8.9%
SCQR ∣L(x)∣
90.3
4
1
201
223
95.6%
98.9%
8.8%
SCQR ResNet(x)
93.0
4
1
205
218
96.2%
98.9%
5.7%
Spectraverse Adduct: [M+CH3COOH-H]−
MSAlign Acc: 71.8%
CP
90.0
59
56
152
159
60.4%
61.0%
0.0%
Table 1: Comparison of conformal prediction (CP) and score conformalized quantile regression (SCQR) on graph-valued outputs with finite candidate sets. Acc denotes the base predictor retrieval accuracy. We report the metrics on non-empty predicted conformal sets, hence the changes in candidate set sizes.
Method
Marginal Coverage
Worst-Slab Coverage
CP
90.2%
64.7%
SCQR ∣L(x)∣
90.3%
67.6%
SCQR ResNet(x)
90.2%
73.5%
Table 2: Marginal and worst-slab coverage for different SCQR conditioning attributes on the Coloring dataset. Slabs group inputs by number of candidates.
Figure 4: Empirical coverage as a function of the probability of ground-truth absence from the candidate library P(Yn+1∈/L(Xn+1)) , evaluated on the Coloring dataset.
Figure 5: Empirical coverage versus candidate set size, for CP (left) and SCQR (right). For SCQR, the attribute function ω(x) is task-dependent: ω(x)=∣L(x)∣ (candidate set size) for Coloring, and ω(x)=\textscDreaMS(x) (mass-spectrum embedding) for Molecule Retrieval.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Coloring dataset examples using Any2Graph as graph predictor.
Parameter
Value
Optimizer
Adam
Learning rate
0.001
Batch size
32
Early stopping patience
3
Early stopping minimum loss improvement
0.0001
Appendix
Table 3: embedding-based SCQR neural network training parameters
Figure 7: Time taken to compute a FGW distance between a pair of Coloring dataset graphs with various amounts of nodes. Notice how the performance of uniform initialization is offset from the others, given that it is the default initialization of the FGW algorithm, not requiring prior computations. The adjacency cost matrix scales very badly with the amount of nodes. At 10 nodes per graph, it is already 20 times slower than the other options.
Figure 8: Computation time of the FGW distance between a pair of coloring dataset graphs with the FD prior. Although the adjacency cost matrix yields good conformal performance, it is not recommended when working with large graphs.
Conformal Set Size ↓
Candidate Set Size
Reduction % ↑
Dataset
Model fθ
Method
Emp. Cov. (%)
Mean
Median
Mean
Median
Mean
Median
Empty rate (%) ↓
Coverage 1−α=90%
Colors
Any2Graph Acc: 82%
CP
90.2
4
1
205
223
95.8%
98.9%
8.9%
SCQR ∣L(x)∣
90.3
4
1
201
223
95.6%
98.9%
8.8%
SCQR ResNet(x)
93.0
4
1
205
218
96.2%
98.9%
5.7%
Spectraverse Adduct: [M+CH3COOH-H]−
MSAlign Acc: 71.8%
CP
90.0
59
56
152
159
60.4%
61.0%
0.0%
Appendix
Table 4: Comparison of conformal prediction (CP) and score conformalized quantile regression (SCQR) on graph-valued outputs with finite candidate sets. Acc denotes the base predictor retrieval accuracy.
Category
Acronym
Description
Cost Matrix C
A
Adjacency
L
Laplacian
Lsym
Laplacian (symmetrically normalized)
SP
Shortest Paths
Initial Transport Plan G0
FD
Feature Diffusion: Solution to the Earth Mover’s Distance problem for given nodes of feature augmented vector (F,AF)
LFD
Laplacian Feature Diffusion: Same as Feature Diffusion but with features (F,LF)
Appendix
Table 5: Summary of acronyms used in the experiments.
Task
Cost Matrix
Empirical Coverage
Mean set size ↓
Median set size ↓
Mean reduction (%) ↑
Median reduction (%) ↑
Empty rate (%) ↓
Colors
A
90.2%
4
1
95.8%
98.9%
9.0%
Lsym
90.4%
4
1
95.8%
98.9%
8.5%
L
90.8%
4
1
95.5%
98.7%
8.3%
SP
90.1%
205
223
2.2%
0.0%
2.6%
Appendix
Table 6: Cost matrix impact on different tasks. The colors task uses the feature diffusion G0 prior.
Task
G0
Empirical Coverage
Mean set size ↓
Median set size ↓
Mean reduction (%) ↑
Median reduction (%) ↑
Empty rate (%) ↓
Colors
LFD
90.4%
4
1
95.8%
98.9%
8.7%
FD
90.2%
4
1
95.8%
98.9%
9.0%
Uniform
90.1%
4
1
95.8%
98.9%
8.7%
LFD-sym
90.0%
4
1
95.8%
98.9%
8.8%
Id
90.4%
6
2
94.4%
97.9%
5.1%
Appendix
Table 7: G0 prior impact on different tasks. The colors task uses the adjacency cost matrix. Note that uniform G0 prior often converges very slowly.