Geometric Mean Pooling for Equal-Weight Multiplicative Coarse-Graining
Authors: Ang-Kun Wu, Fangdi Wen, Jingtao Zhang
Organizations: Department of Physics and Astronomy, University of Tennessee, Knoxville Knoxville, Tennessee 37996, USA · Department of Physics and Astronomy, Rutgers University New Brunswick, New Jersey 08901, USA · Google Mountain View, CA 94043, USA
As an alternative to the additive and extremal biases of average and max pooling, we introduce Geometric Mean Pooling (GMP), a signed pooling operator that combines the product of feature signs with the geometric mean of feature magnitudes. Motivated by local-to-global composition in quantum many-body physics, GMP retains both joint sign information and a characteristic multiplicative scale without introducing learnable pooling parameters. We show that non-overlapping hierarchical GMP preserves the corresponding global multiplicative statistic and evaluate it on synthetic sequence tasks, iterative coarse-graining, image classification, and molecular lipophilicity regression. On the synthetic tasks, GMP recovers product-based signals more accurately than average and max pooling and maintains predictive performance under the tested levels of multiplicative input noise. On image and molecular data, however, its effectiveness depends on the representation, target parameterization, and placement of local and global pooling. These results position GMP as a complementary, regime-dependent inductive bias for tasks in which equal-weight multiplicative composition is plausible, rather than as a universal replacement for standard pooling operators.
Figures & tables
Figure 1: Sequence-length and cell-size sweeps for Case B classification (top) and R2 regression (bottom). Left: varying sequence length with cell size 3 . Right: varying cell size with sequence length 120 . Clean classification inputs are standard Gaussian; clean regression inputs are lognormal with standard Gaussian logarithms. Targets are computed before adding Gaussian input noise with standard deviation 0.05 . Error bars denote the standard deviation across ten random seeds.
Setting
Accuracy
Case A, signed
0.9330 ± 0.0659
Case A, unsigned
0.5107 ± 0.0165
Case B, signed
0.9856 ± 0.0047
Case B, unsigned
0.5072 ± 0.0136
Table 1: Ablation of the sign-parity term in geometric mean pooling. Results are reported as mean ± standard deviation across 10 random seeds. The sequence length is 120 , and the cell size is 3 for Case B.
Figure 2: R3: global regression under multiplicative and additive input noise, x~=xexp(ϵmul)+ϵadd , with logx∼N(0,1) and independent Gaussian perturbations of standard deviations σmul and σadd . Over the tested noise ranges, GMP maintains higher predictive performance than max pooling, including at low noise levels.
Figure 3: R4: preservation of a multiplicative observable across pooling levels. MAE is evaluated against the clean geometric-mean target ( σ=1 , p=25% , L=256 ), using a global geometric readout after each local pooling step for all methods. GMP maintains an approximately constant readout error across the hierarchy.
Figure 4: Validation accuracy on MNIST, Fashion-MNIST, and CIFAR-10 across ten random seeds. Each panel reports the three pooling primitives (GMP, max, and average) under three architectures: variable global pooling with fixed intermediate max pooling, variable local pooling with fixed global max pooling, and synchronized local-plus-global pooling. Local GMP is generally more competitive than global GMP on MNIST and Fashion-MNIST, whereas the gap between GMP and the gap between GMP and the average- and max-pooling baselines increases on CIFAR-10. Error bars denote standard deviation across 10 seeds. All models are trained with cross-entropy loss and Adam [ 23 ] using batch size 128 and learning rate 10−3 .
Baseline
RMSE
R2
Morgan FP + XGBoost
0.8870
0.4631
Morgan FP + Random Forest
0.8718
0.4813
Morgan FP + FC
0.8752
0.4773
SMILES Embedding + FC
1.1596
0.0824
Table 2: Lipophilicity regression performance on the original logarithmic label scale, using a scaffold split.
Figure 5: Test R2 on the auxiliary target z=exp(y) for the Morgan-fingerprint 2D CNN under all combinations of local and global pooling operators. Each molecule is represented by a radius-2, 2,048-bit Morgan fingerprint, projected to a learned 32×32 latent feature map, and processed by a three-block convolutional hierarchy. Values and error bars denote the mean and standard deviation over ten random seeds.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Sweep
Value
Max acc
Average acc
GMP acc
ρ -sweep (seed=42, seq_len=32)
0.0
0.4680
0.5390
1.0000
0.3
0.5080
0.4860
1.0000
0.5
0.5000
0.4900
1.0000
0.7
0.5130
0.4830
1.0000
0.9
0.6040
0.6040
1.0000
Seq_len sweep (seed=42, ρ=0 )
16
0.5190
0.5190
1.0000
Appendix
Table 3: Case A classification accuracy with seed 42 . The correlation sweep fixes N=32 ; the sequence-length sweep uses independent inputs ( ρ=0 ).
N
R2
MSE ( ×10−3 )
Max
Avg
GMP
Max
Avg
GMP
16
0.137
0.609
1.000
60.7
27.5
0.0
32
0.081
0.620
1.000
29.3
12.1
0.0
64
0.071
0.631
0.993
14.6
5.8
0.0
128
0.019
0.638
1.000
7.9
2.9
0.0
Appendix
Table 4: R1 global geometric-mean regression on noiseless lognormal inputs with seed 42 . Results are reported as R2 and MSE; MSE values are scaled by 103 .
Figure 6: Test R2 for lipophilicity regression on the original label scale using CNNs with different combinations of local and global pooling. Error bars denote the standard deviation across ten random seeds. Max pooling achieves the highest R2 in this setting.
Density aggregation is a central problem in machine learning, for instance when combining predictions from a Deep Ensemble. The choice of aggregation remains an open question with two commonly proposed approaches being linear pooling (probability averaging) and geometric pooling (logit averaging). In this work, we address this question by studying the normalized generalized mean of order r∈R∪{−∞,+∞} through the lens of log-likelihood, the standard evaluation criterion in machine learning. This provides a unifying aggregation formalism and shows different optimal configurations for different situations. We show that the regime r∈[0,1] is the only range ensuring systematic improvements relative to individual distributions, thereby providing a principled justification for the reliability and widespread practical use of linear (r=1) and geometric (r=0) pooling. In contrast, we show that aggregation rules with r∈/[0,1] may fail to provide consistent gains with explicit counterexamples. Finally, we corroborate our theoretical findings with empirical evaluations using Deep Ensembles on image and text classification benchmarks.
Modern image classifiers widely adopt global average pooling (GAP) followed by a linear classification head. This linearity ensures that the image-level logits equal the average of logits obtained by applying the classification head pointwise to the feature grid prior to GAP. Consequently, standard classifiers may inherently retain spatial class evidence that remains recoverable even when the image-level prediction is incorrect. This structure naturally suggests a multiple-instance learning (MIL) interpretation, where an image is viewed as a bag of spatial instances. Within this formulation, we demonstrate that standard classifiers trained with a single label per image can still learn the intended classification task in multi-object scenes. We further exploit this property to decompose image-level logits into a prediction grid, providing a post-hoc diagnostic to extract spatial class evidence that GAP otherwise obscures. Our systematic evaluation reveals that off-the-shelf models consistently recover the ground-truth class within foreground regions. The MIL interpretation further suggests that common classifier failures reflect known limitations of mean aggregation.
Graph pooling is commonly applied in graph classification, yet its empirical gains over standard WL-1 expressive GNNs are often marginal or inconsistent. We study this gap by analysing the interaction between node features and graph topology and their effect on pooling objectives. Our analysis reveals that pooling operators require node features that are well-aligned with the graph's topology -- a condition often overlooked and not guaranteed in empirical networks. We formalise fundamental requirements for node features to enable effective pooling, and introduce a quantitative measure of feature quality. Our empirical evaluation shows that, when these requirements are satisfied, pooling can be beneficial and improve performance on appropriate datasets.
Jan von Pichowski, Alžbeta Hrabošová, Ingo Scholtes +1
Center for AI and Data Science University of Würzburg, Germany · Department of Computing Science Umeå University, Sweden