Lipschitz Thinking: Ten Years of Certifiable-by-Design Robust Neural Networks

Fabio Brau; Giorgio Piras; Maura Pintor; Battista Biggio

cs.LGOct 5, 2026

Lipschitz Thinking: Ten Years of Certifiable-by-Design Robust Neural Networks

Authors: Fabio Brau, Giorgio Piras, Maura Pintor, Battista Biggio

Organizations: Department of Electrical and Electronic Engineering, University of Cagliari, Cagliari, Italy

Abstract

The Lipschitz property of a deep neural network provides a direct measure of its sensitivity to input perturbations and, when explicitly controlled, offers a principled way to limit the propagation of errors and improve robustness. Over the past decade, Lipschitz-bounded layers have been incorporated into increasingly expressive and high-performing deep models, narrowing the gap between empirical robustness and formal, by-design guarantees of stability. This article introduces the fundamental concepts underlying Lipschitz-bounded neural networks, explaining the principles behind Lipschitz-constrained layers, the mechanisms used to enforce their bounds, and how they yield robustness certificates at the cost of a single forward pass. The tutorial concludes by discussing emerging and open directions, highlighting Lipschitz control as a general framework for offering guaranteed, by-design stability.

Figures & tables

Strategy	Robustify	Certify	Employment Time
Adversarial Training [ 13 ]	✓	✗	Train
Formal Verification [ 14 ]	✗	✓	Test
Randomized Smoothing [ 15 ]	✓	(✓)	Train/Test
Lipschitz-Bounded Networks [ 16 ]	✓	✓	Train/Test

TABLE I: Comparison of robustness-related strategies (i.e., defenses). Adversarial training increases robustness without providing guarantees, while Randomized smoothing certifies only in probabilistic terms. The Employment column indicates whether the method is applied at test time, requires training or both.

Fig. 1: Taxonomy of $1$ -Lipschitz layer constructions, organized by enforcement of the Lipschitz constraints.

Panel of Figure 3 — Fig. 2: The weight $W_{i}$ lives in the ambient space, off $\mathcal{O}(d)$ , the set of admissible orthogonal weight matricesA plain gradient step on the classification loss $\ell$ takes it to $\tilde{W}_{i+1}$ , in a direction that has no reason to respect the constraint; the penalty term $R_{\beta}$ then pulls the iterate towards $\mathcal{O}(d)$ , producing the updated weight $W_{i+1}$ . Because the two components compete through the weight $\beta$ , the iterate approaches the manifold but does not guarantee to reach it.

Panel of Figure 4 — Fig. 3: Weights $W_{i}$ lives in the manifold $\mathcal{L}$ , while the learnable parameters $\theta_{i}$ lives in the flat unconstrained parameter space $\Theta$ . The update of the parameter happens through a gradient step that moves $\theta_{i}$ to $\theta_{i+1}$ . The map $\Phi$ then sends each parameter to a weight matrix that lies in $\mathcal{O}(d)$ by construction, so both the updated iterate $W_{i+1}=\Phi(\theta_{i+1})$ is admissible.

Kernel Parameterization
Layer	Dense: $\Phi(\theta)$	Convolution: $\Phi(\theta)$
AOL [ 30 ]	$\mathrm{AOL}(\theta)=\theta\,\Big(\underbrace{\textstyle\sum_{j}\big\|\theta^{\top}\theta\big\|_{ij}}_{D(\theta)}\Big)^{-1/2}$	$\mathrm{AOL}(\theta)=\theta\,\Big(\underbrace{\textstyle\sum_{j,\,p}\big\|\theta^{\top}\!\star\theta\big\|_{ij,p}}_{D(\theta)}\Big)^{-1/2}$
BCOP [ 20 ]	$BB(\theta)=A_{k}$ , $A_{i+1}=A_{i}\big(I+\tfrac{1}{2}(I-A_{i}^{\top}A_{i})\big)$	$\mathrm{BCOP}(\theta)=[\mathrm{BB}(\theta_{0}),\dots,\mathrm{BB}(\theta_{2k-2})]$ $\dagger$
SOC [ 31 ]	$\exp(A)=\sum_{j\geq 0}\frac{A^{j}}{j!}$ , $A=\theta-\theta^{\top}$	$\exp_{\star}(A)=\sum_{j\geq 0}\frac{A^{\star j}}{j!}$ , $A=\theta-\theta^{\top}$
Cholesky [ 37 ]	$\mathrm{Chol}(\theta)=\theta\,L^{-\top}$ , $\theta^{\top}\theta=LL^{\top}$	—
Spectral Parameterization

TABLE II: The parameterization map

\Phi

for dense (

\theta\in\mathbb{R}^{c\times c}

) and convolutional (

\theta\in\mathbb{R}^{c\times c\times k\times k}

) parameters. For kernels,

\theta^{\top}

denotes the adjoint kernel and

\mathcal{F}

the spatial DFT, so that

\mathcal{F}\theta

is a collection of

c\times c

complex blocks to which the dense map is applied blockwise (with ∗ in place of ⊤ ).

\sigma

denotes the ReLU,

b

a bias.

Panel of Figure 6 — Fig. 4: The weights $W_{i}$ starts on the manifold $\mathcal{O}(d)$ . The update is performed not in the ambient space (i.e., using the Euclidean gradient $\nabla\ell(W_{i})$ ), since it generally points off the manifold. However, the update is projected onto the tangent space $T_{w_{i}}\mathcal{O}(d)$ (shaded), giving the Riemannian gradient $\mathfrak{g}$ , the steepest direction along the manifold. The step is then taken by the geodesic exponential map , which travels along the shortest path on $\mathcal{O}(d)$ leaving $W_{i}$ with initial direction $-\alpha\mathfrak{g}$ , tracing the curve to $W_{i+1}=\exp_{W_{i}}(-\alpha\mathfrak{g})$ ; the iterate has moved and is still admissible.

Fig. 5: In scale representation of the margin landscape of a naive ResNet-56 and a Lipschitz-bounded classifier [ 24 ] around a CIFAR-10 image on the 2D plane spanned by

-\nabla M(x,y)

(the boundary direction) and a random orthogonal direction. Distances are measured in

\ell_{2}

in pixel space, where the bold curve is the decision boundary

M=0

. For the ResNet-56, the margin drops from

10.2

0

within a surprisingly small (and unpredictable) distance from the boundary of

\approx 0.07

, while for the 1-Lipschitz model it decreases slowly with a bounded rate, offering a certified radius

C_{f}(x)\approx 0.24

Fabio Brau is an Assistant Professor at the University of Cagliari, conducting research on AI security and trustworthiness. He holds a Master’s in Mathematics from the University of Pisa and a Ph.D. in Embedded Systems from the Scuola Superiore Sant’Anna, where he worked on robustness certification for deep neural networks. His research spans Adversarial Machine Learning and Trustworthy AI for Computer Vision and NLP systems, with contributions to Lipschitz-constrained architectures for certifiable robustness. He has published in top-tier venues including NeurIPS, CVPR, AAAI, and IEEE TPAMI, and is involved in European research projects on trustworthy AI.

Table 8

Giorgio Piras is a Postdoctoral Researcher at the sAIfer Lab, Department of Electrical and Electronic Engineering, University of Cagliari, Italy, working on adversarial machine learning and large language model security. He received his BSc (110/110, 2019) and MSc with honors (2021) from the University of Cagliari, and his PhD with honors (2025) from Sapienza University of Rome (National PhD-AI program), with the thesis Adversarial Pruning: Improving Evaluations and Methods , supervised by Prof. Battista Biggio. He was a visiting student at Karlsruhe Institute of Technology. He reviews for NeurIPS, ICLR, AAAI, USENIX, ACM CCS, Pattern Recognition, Neurocomputing, Machine Learning, and IEEE TIFS.

Table 9

Maura Pintor (Member, IEEE) is an Assistant Professor at the PRA Lab, Department of Electrical and Electronic Engineering, University of Cagliari, Italy, where she received her PhD in Electronic and Computer Engineering in 2022. Her research focuses on adversarial machine learning, particularly evaluating, debugging, and improving the robustness of machine learning systems to make AI more reliable, secure, and trustworthy. She is an Area Chair for NeurIPS, Associate Editor for Pattern Recognition, and Consulting Associate Editor for IEEE TIFS. She is a member of the IEEE Information Forensics and Security Technical Committee, IAPR, and ELLIS, and maintains the open-source library SecML-Torch.

Table 10

Battista Biggio (Fellow, IEEE) (MSc 2006, PhD 2010) is Full Professor at the University of Cagliari, Italy. He has provided pioneering contributions in machine learning security, playing a leading role in this field. His seminal paper on “Poisoning Attacks against Support Vector Machines” won the prestigious 2022 ICML Test of Time Award. His work on “Wild Patterns” won the 2021 Best Paper Award and Pattern Recognition Medal from Elsevier Pattern Recognition. He has managed more than 10 research projects, and serves as a PC member of ICML and USENIX Security, and as Area Chair of NeurIPS. He chaired IAPR TC1 (2016-2020), and served as Associate Editor for IEEE TNNLS, IEEE CIM, and Elsevier PRJ. He is now Associate Editor-in-Chief for PRJ. He is also Fellow of IEEE, Senior Member of ACM, and member of IAPR and ELLIS.

Table 11

Explore similar work

Lipschitz-Based Robustness Certification Under Floating-Point Execution
Mar 6, 2026Toby MurrayFloating-Point Lipschitz Continuity
University of Melbourne, Australia
Understanding Certified Training with Interval Bound Propagation
Jun 17, 2023Yuhao Mao, Mark Niklas Müller, Marc Fischer +1Neural Network Verification Learnability
Department of Computer Science, ETH Zürich, Swizterland
Beyond Lipschitz: Data-Driven Robustness via Discrete Modulus of Continuity
May 27, 2026Jürgen Dölz, Michael Multerer, Michele PalmaLipschitz Continuity Continuum
Institute for Numerical Simulation University of Bonn Germany · Dalle Molle Institute for Artificial Intelligence USI Lugano Switzerland

Mar 6, 2026, cs.LGJ/K move · Enter open · S save

Lipschitz-Based Robustness Certification Under Floating-Point Execution

Toby Murray

University of Melbourne, Australia

Floating-Point Lipschitz Continuity Neural Network Verification Robustness Verification Rectified Linear Unit

Open paper AskPDF arXiv

Lipschitz-based robustness certification bounds a network's sensitivity through concrete numerical computation rather than symbolic reasoning, and so scales efficiently. It is increasingly used even where verifiable guarantees matter. Yet, as with most prior work on robustness certification and verification, soundness is typically proved against a semantic model assuming exact real arithmetic. Deployed networks instead execute in floating-point, creating a gap between certified properties and executed behaviour. As motivating evidence, we give counterexamples showing that real arithmetic robustness guarantees can fail under floating-point execution, even for previously verified certifiers. We then develop a formal, compositional theory relating real arithmetic Lipschitz-based sensitivity bounds to floating-point execution under standard rounding-error models for feed-forward ReLU networks. We derive sound conditions for floating-point robustness, including bounds on certificate degradation and sufficient conditions for the absence of overflow. We also give an efficient floating-point Gram iteration algorithm for Lipschitz bounds and prove that it never under-estimates the true norm. Separately, when a model is certified pre-deployment, we show how measuring its actual deviation against a high-precision execution can substantially reduce certificate degradation. We formalise the theory and its soundness, and implement an executable certifier, evaluated across dense networks spanning image, tabular, and many-class classification. To our knowledge, ours is the first method for soundly accounting for floating-point effects in Lipschitz-based robustness certification, and, done efficiently, the first floating-point-sound robustness checking procedure of any kind to certify models' entire test sets -- even those with 500,000 examples -- while retaining enough precision to be practical.

Abstract

Explore similar work

Lipschitz-Based Robustness Certification Under Floating-Point Execution

Understanding Certified Training with Interval Bound Propagation

Beyond Lipschitz: Data-Driven Robustness via Discrete Modulus of Continuity