cs.LGNov 6, 2026

Robust Deep Prototype Classifiers via Lipschitz Training

Authors: Adia Khalid Khan, Lukas Gehring, Benjamin Paaßen

Organizations: Faculty of Technology, Bielefeld University

Abstract

Robustness and interpretability are interrelated and desirable properties of machine learning models. A model that changes its prediction under imperceptible perturbations behaves inconsistently with human-recognizable features and, hence, is not interpretable. Prototype-based classifiers such as Generalized Learning Vector Quantization address both at once. They are interpretable by construction because they make the decision by comparing the input to the prototypical example; and their hypothesis margin yields a closed form robustness certificate. However, on high-dimensional data, these shallow models need deep embedding networks to achieve high accuracy, and this voids the robustness certificate. The margin is a statement about distances in the embedding space, and an unconstrained network can map an arbitrarily small input perturbation to arbitrarily large attacks in the embedding space. We close this gap with a theorem: if the embedding network is locally L-Lipschitz around a point, and the classifier is ε-robust around its embedding, then the full model is ε/L-robust in the input space. Based on this, we propose a Lipschitz training scheme to achieve L-Lipschitz properties for embedding networks. We evaluate three classification heads (linear, linear Support Vector Machine, and GLVQ) crossed with two embedding networks (standard and Lipschitz-constrained) on MNIST, CIFAR-10, and Imagenette-160, under FGS, PGD and C&W attacks. The evaluation results show that combining a prototype-based head with a Lipschitz-constrained embedding provides the strongest robustness. This suggests that the Lipschitz constraint provides an additional robustness benefit in addition to the advantage provided by the prototype-based head. Since the Lipschitz constraint is enforced by sampling rather than by construction, the reported robustness is empirical rather than certified.

Figures & tables

Explore similar work

Oct 11, 2026

Adversarial Detection for Pre-trained Classifiers via Lipschitz-Margin Analysis with Certification

Deep classifiers deployed in safety-critical services like autonomous-driving perception remain vulnerable to small adversarial perturbations. Certified ℓ2\ell_2 defenses today face a costly trade-off: randomized smoothing (RS) requires hundreds of noisy queries per input, while Lipschitz networks (LN) constrain the architecture and sacrifice clean accuracy. We take a detection-oriented approach instead: rather than certifying that predictions are invariant inside an ℓ2\ell_2 ball, we certify that adversarial inputs within an ℓ2\ell_2 ball are detected --- a guarantee aligned with security-critical deployments where missing an attack is costlier than flagging a clean input. To realize this, we prove a Lipschitz-margin reduction from certified ℓ2\ell_2 robustness to certified detection, which reduces the problem to handling a single Lipschitz constant LL. We introduce two detector families: DAdv\mathbb{D}_{Adv}, which estimates LL from a single defender-side perturbed pair, and Dlc\mathbb{D}_{lc}, which augments that pair with a local margin centroid to keep the estimate above what the certificate requires. Both deploy on any pretrained classifier without retraining. With a valid global bound, the detector is fully certified at that radius; with a local estimate L′L', certification is sample-wise and applies only when our certified condition holds. Across MNIST, CIFAR-10, and ImageNet-1k on four backbones (LeNet-5, VGG16, ResNet-50, ViT-B/16), our detectors match or outperform five baselines (CTT, SortNet, CRM, ContraNet, A2^2D). For CIFAR-10 with 1-Lipschitz SortNet, our reduction delivers 100%100\% certified detection at the lowest FPR among compared certified detectors; on the standard CIFAR-10 VGG16, our detectors reject all evaluated in-budget adversarial examples, with 62-82% both rejected and formally certified.
Oct 5, 2026

Lipschitz Thinking: Ten Years of Certifiable-by-Design Robust Neural Networks

The Lipschitz property of a deep neural network provides a direct measure of its sensitivity to input perturbations and, when explicitly controlled, offers a principled way to limit the propagation of errors and improve robustness. Over the past decade, Lipschitz-bounded layers have been incorporated into increasingly expressive and high-performing deep models, narrowing the gap between empirical robustness and formal, by-design guarantees of stability. This article introduces the fundamental concepts underlying Lipschitz-bounded neural networks, explaining the principles behind Lipschitz-constrained layers, the mechanisms used to enforce their bounds, and how they yield robustness certificates at the cost of a single forward pass. The tutorial concludes by discussing emerging and open directions, highlighting Lipschitz control as a general framework for offering guaranteed, by-design stability.
Jun 17, 2026

Convex training of Lipschitz-regularized shallow neural networks

In this work, we introduce a training procedure for shallow neural networks that promotes robustness against adversarial attacks. We solve a non-convex Lipschitz-regularized training program by introducing a convex restriction that can be efficiently solved to global optimality. Our approach can be employed as a post-processing step by taking a pre-trained network as an initial solution to then solving the convex program whose optimal network is guaranteed to be no worse than the initial one. We illustrate the improvements of our training procedure with experiments using real world datasets for regression tasks under an adversarial setting. We show numerically that solving our proposed convex program yields networks with lower objective values on the Lipschitz-regularized program compared to existing methods. Additionally, we show that on certain datasets, networks obtained using our convex training program are both more accurate and robust with respect to adversarial attacks.