For compact convex sets L,K⊂Rn, denote by λK(L) the smallest size of a homothet of K that contains L. We define a measure of symmetry based on the n-simplex Δ=Δn⊂Rn as the ratio
ρΔ(L):=λΔ(L)λ−Δ(L).
We study this measure and deduce the following results: (1) The classical Minkowski measure of symmetry m∗(L) can be defined as an affine-invariant version of ρΔ(L). (2) We improve the stability analysis for the Minkowski measure of symmetry; if m∗(L)≥n−ε then L is 1−ε1-close to Δ in the Banach--Mazur distance. (3) We obtain a novel characterization of simplices as the only convex bodies K for which the function L↦λK(L) is additive (a property we term ``outer additivity''). (4) Motivated by the expressivity of ReLU neural networks, we study the depth complexity of polytopes in Rn under the two operations: Minkowski sum and convex hull of a union. We prove the sharp bound ρΔ(P)≤2d−1 for every polytope P of depth complexity d. In other words, simplices cannot be approximated by low-depth polytopes.
Deep networks often exhibit a preference for "simple" solutions, and such a simplicity bias is widely believed to play a key role in generalization. Yet a broadly applicable, quantitative measure of simplicity remains elusive. We introduce polynomial representations as a distribution-aware, low-dimensional surrogate for neural functions: we approximate a network's predictive behavior along data-dependent interpolation paths using orthogonal polynomial bases, yielding a compact functional representation. We show that the effective degree of this representation serves as a practical simplicity metric that is predictive of generalization across tasks and architectures, and consistently outperforms existing generalization proxies such as sharpness. Finally, polynomial representations naturally yield a differentiable simplicity regularizer, which consistently improves generalization in image and text classification, fine-tuning contrastive vision-language models, and reinforcement learning.
We develop a framework for analyzing parameter symmetries in deep ReLU networks and obtain a complete characterization of the generic parameter fibers for three-layer bottleneck architectures. Our approach provides explicit semi-algebraic descriptions of these fibers and yields a polynomial time algorithm for deciding functional equivalence of two parameters. The symmetries include discrete and continuous transformations arising from layer composition, and depend on whether deeper layers hide or preserve geometric structure from preceding layers. Finally, we show that some of these symmetries induce local conservation laws along gradient flow, while others do not.
Johanna Marie Gegenfurtner, Moritz Grillo, Guido Montúfar
Deep neural networks have been widely used as universal approximators for functions with inherent physical structures, including permutation symmetry. In this paper, we construct symmetric deep neural networks to approximate symmetric Korobov functions and prove that both the convergence rate and the constant prefactor scale at most polynomially with respect to the ambient dimension. This represents a substantial improvement over prior approximation guarantees that suffer from the curse of dimensionality. Building on these approximation bounds, we further derive a generalization-error rate for learning symmetric Korobov functions whose leading factors likewise avoid the curse of dimensionality.