cs.CCSep 26, 2025

Parameterized Hardness of Zonotope Containment and Neural Network Verification

Authors: Vincent FroeseMoritz GrilloChristoph HertrichMoritz Stargalla

Organizations: Technische Universität Berlin · Max Planck Institute for Mathematics in the Sciences · University of Technology Nuremberg

Abstract

Neural networks with ReLU activations are a widely used model in machine learning. It is thus important to have a profound understanding of the properties of the functions computed by such networks. Recently, there has been increasing interest in the (parameterized) computational complexity of determining these properties. In this work, we close several gaps and resolve an open problem posed by Froese et al. [COLT '25] regarding the parameterized complexity of various problems related to network verification. In particular, we prove that, for all 2\ell\ge 2, deciding positivity (and thus surjectivity) of a function f:RdRf:\mathbb{R}^d\to\mathbb{R} computed by an \ell-layer ReLU network is W[1\ell-1]-hard when parameterized by the input dimension dd. The case =2\ell=2 implies that zonotope non-containment (a problem that is of independent interest in computational geometry, control theory, and robotics) is W[1]-hard with respect to the ambient dimension dd. Moreover, we show that approximating the maximum within any multiplicative factor and computing the LpL_p-Lipschitz constant for p(0,]p\in(0,\infty] in \ell-layer networks is NP-hard and W[1\ell-1]-hard with respect to dd. For 3\ell\ge 3, approximating the LpL_p-Lipschitz constant is NP- and W[2\ell-2]-hard. We further show that the above problems are NP- and W[tt]-hard (for all t1t\ge 1) with respect to \ell for constant dd. Notably, our hardness results imply that the naive enumeration-based methods for these fundamental problems running in n(1)dpoly(N)n^{(\ell-1) d}\cdot\operatorname{poly}(N) time are all essentially optimal under the Exponential Time Hypothesis.

Explore similar work

Jul 15, 2026cs.CC

Random Parameter Noise Does Not Make Exact ReLU Verification Easy

We study exact verification of ReLU networks in an adversarial smoothed model. Every network weight and bias is independently perturbed by Gaussian noise, clipped to [2,2][-2,2], and rounded to the exact dyadic grid determined by the input bit complexity. We show that, under the standard assumption NP⊈BPP\mathrm{NP}\not\subseteq\mathrm{BPP}, there is no sound and complete verifier whose expected running time is polynomial in network size, bit complexity, and inverse noise level for every base instance. The conclusion already holds at the fixed noise level σ=211σ_\star=2^{-11} for one-hidden-layer networks over a unit box, with hidden fan-in at most three and base coefficients in [1,1][-1,1]. The proof combines an exact gap embedding with a quantitative robustness argument. For every E3SAT formula ΦΦ with mm clauses, a four-ReLU-per-clause construction satisfies maxx[0,1]ngΦ(x)=(munsat(Φ))/3\max_{x\in[0,1]^n} g_Φ(x)=(m-\operatorname{unsat}(Φ))/3, and coordinatewise threshold rounding never decreases the objective. A weighted parameter-sensitivity inequality and Gaussian concentration then show that a verification gap linear in mm survives the aggregate perturbation of all coefficients with probability at least 1em/81-e^{-m/8}. The proof includes clipping, exact dyadic rounding, output-layer perturbations, polynomial-bit sampling of the rounded Gaussian law, and the conversion from expected smoothed running time to a BPP algorithm. Computational checks test the exact identity and illustrate the different scaling of extensive and constant gaps; they are diagnostics rather than evidence for the complexity theorem. The result concerns worst-case base networks in the stated absolute-noise model, but it shows that parameter nondegeneracy alone does not yield a universal smoothed-polynomial guarantee for exact verification.
Mojtaba Soltanalian
Aug 24, 2026cs.LG

Every Layer Counts: An Exponential L2L_2 Depth Hierarchy for ReLU Networks

We prove a depth hierarchy for ReLU neural networks in which every additional ReLU layer can save exponentially many neurons. For all k2k\geq2, we construct a globally [0,1][0,1]-valued, 11-Lipschitz function realized by a depth-(k+1)(k+1) network of width O(d4)\mathcal{O}(d^4), whereas any depth-kk network with unrestricted weights and width at most 2d2d(k1)\frac{2^d}{2d(k-1)} has squared L2L_2 error at least 1/241/24 under an absolutely continuous distribution supported at exponential distance from the origin. To the best of our knowledge, this is the first exponential hierarchy across all adjacent fixed depths, and the first exponential separation for ReLU networks between two fixed depths whose shallower network has depth at least 33. The lower bound also immediately yields the corresponding hierarchy for exact computation. Moreover, the case k=2k=2 gives a compactly supported separation between depths 33 and 22 with unrestricted shallow-network weights, answering a question raised by Safran, Eldan, and Shamir (2019). The distribution used in our construction nevertheless has all its mass at exponential radius, placing the hierarchy outside the regularity regime in which such a separation would imply major threshold-circuit lower bounds. We also prove an exact separation for a more regular target, which is globally [0,1][0,1]-valued and O(d)\mathcal{O}(\sqrt d)-Lipschitz and maps the unit hypercube onto [0,1][0,1]. It is computed by a polynomial-width depth-44 network, whereas any depth-33 network agreeing with it on the unit hypercube requires exponentially many first-layer neurons, even with unrestricted weights.
Itay Safran
Jul 22, 2026cs.LG

Shallower ReLU Network Representations via Exact Linear Algebra

We study the depth required by ReLU networks to exactly represent piecewise linear functions, focusing specifically on the maximum function. This problem has recently received significant attention in both the ML and TCS literature. We prove that maxn(x)=max{x1,,xn}\max_n(x)=\max\{x_1,\ldots,x_n\} is exactly representable with two hidden layers for every n12n\leq 12. Previously, this was only known up to n5n\leq5 [Bakaev, Brunck, Hertrich, Stade, Yehudayoff, STOC'26]. We obtain our constructions through an exact computer-assisted search within a space of candidate solutions: After a symmetry reduction, we obtain a finite system of linear equations over Q\mathbb{Q} such that any solution yields a valid representation of the maximum function. The resulting constructions have a structured first hidden layer, which enables recursive substitution into deeper networks. This yields an exact ReLU representation of maxn\max_n with at most log6(n/2)+1\lceil \log_6(n/2) \rceil+1 hidden layers. Consequently, every continuous piecewise-linear function on Rd\mathbb{R}^d admits an exact representation with at most log6((d+1)/2)+1\lceil\log_6((d+1)/2)\rceil+1 hidden layers; in particular, two hidden layers suffice for d11d\leq 11. Again, these results improve upon [Bakaev, Brunck, Hertrich, Stade, Yehudayoff, STOC'26], who proved analogous logarithmic bounds with base three.
Kilian Rueß, Gennadiy Averkov, Florestan Brunck +7