Lipschitz Network

Lipschitz networks are neural networks designed with a bounded Lipschitz constant, ensuring that small input changes result in proportionally small output changes, thereby enhancing robustness against adversarial attacks and noise. Current research focuses on developing architectures and training methods that achieve this constraint while maintaining sufficient expressive power, exploring techniques like optimal transport, spectral normalization, and novel activation functions such as the "sandwich layer" and N-activation. This work is significant because it addresses the critical need for reliable and trustworthy deep learning models in safety-critical applications and improves the theoretical understanding of function approximation capabilities within constrained network architectures.

Papers

September 18, 2024

Bridging the Gap Between Approximation and Learning via Optimal Approximation by ReLU MLPs of Maximal Regularity
Ruiyang Hong, Anastasis Kratsios
LeArning Abstract Average Approximation Universal Approximators Lipschitz Network Geometric Regularity Function Approximator

August 1, 2024

OTAD: An Optimal Transport-Induced Robust Model for Agnostic Adversarial Attack
Kuo Gai, Sicong Wang, Shihua Zhang
Adversarial Training Adversarial Perturbation Optimal Transport Adversarial DEfense Lipschitz Network Optimal Transport Map Agnostic Adversarial

May 19, 2024

On Robust Reinforcement Learning with Lipschitz-Bounded Policy Networks
Nicholas H. Barbara, Ruigang Wang, Ian R. Manchester
Deep Reinforcement Learning Robust Policy Robust Reinforcement Learning Policy Parameterization Policy Network Lipschitz Network

December 14, 2023

Improve Robustness of Reinforcement Learning against Observation Perturbations via $l_\infty$ Lipschitz Policy Networks
Buqing Nie, Jingtian Ji, Yangqing Fu, Yue Gao
Reinforcement Learning Deep Reinforcement Learning Native Robustness Robust Reinforcement Learning Lipschitz Network

November 10, 2023

1-Lipschitz Neural Networks are more expressive with N-Activations
Bernd Prach, Christoph H. Lampert
Neural Network Activation Function Lipschitz Network Capturing Extreme Activation

September 29, 2023

Certified Robustness via Dynamic Margin Maximization and Improved Lipschitz Regularization
Mahyar Fazlyab, Taha Entesari, Aniket Roy, Rama Chellappa
Native Robustness Adversarial Perturbation Robust Training Margin Maximization Better Robustness Lipschitz Bound Lipschitz Network Lipschitz Regularization

May 25, 2023

DP-SGD Without Clipping: The Lipschitz Neural Network Way
Louis Bethune, Thomas Massena, Thibaut Boissin, Yannick Prudent, Corentin Friedrich, Franck Mamalet, Aurelien Bellet, Mathieu Serrurier, David Vigouroux
Differential Privacy SGD Lipschitz Constant Clipped Stochastic Gradient Descent Lipschitz Network

March 23, 2023

Transforming Radiance Field with Lipschitz Network for Photorealistic 3D Scene Stylization
Zicheng Zhang, Yinglu Liu, Congying Han, Yingwei Pan, Tiande Guo, Ting Yao
Neural Radiance Field Novel View Synthesis 3D Scene Stylization Lipschitz Network Photorealistic Style Transfer 3D Stylization

March 6, 2023

A Unified Algebraic Perspective on Lipschitz Neural Networks
Alexandre Araujo, Aaron Havens, Blaise Delattre, Alexandre Allauzen, Bin Hu
Convex Set Semidefinite Programming Algebraic Framework Orthogonal Transforms Lipschitz Network

January 27, 2023

Direct Parameterization of Lipschitz-Bounded Deep Networks
Ruigang Wang, Ian R. Manchester
Deep Neural Network Lipschitz Bound Tighter Bound Lipschitz Network

October 28, 2022

Improving Lipschitz-Constrained Neural Networks by Learning Activation Functions
Stanislas Ducotterd, Alexis Goujon, Pakshal Bohra, Dimitris Perdios, Sebastian Neumayer, Michael Unser
Neural Architecture Activation Function ReLU Activation Total Variation Lipschitz Network Spline Approximation Functional Constrained Optimization

October 4, 2022

Rethinking Lipschitz Neural Networks and Certified Robustness: A Boolean Function Perspective
Bohang Zhang, Du Jiang, Di He, Liwei Wang
Certified Robustness Boolean Function Lipschitz Network Lipschitz Learning

August 5, 2022

Almost-Orthogonal Layers for Efficient General-Purpose Lipschitz Networks
Bernd Prach, Christoph H. Lampert
High Efficiency Lipschitz Operator Lipschitz Constant Orthogonal Transforms Lipschitz Network

April 13, 2022

Approximation of Lipschitz Functions using Deep Spline Neural Networks
Sebastian Neumayer, Alexis Goujon, Pakshal Bohra, Michael Unser
Average Approximation ReLU Network Lipschitz Network Spectral Norm Spline Activation

December 27, 2021

Sparsest Univariate Learning Models Under Lipschitz Constraint
Shayan Aziznejad, Thomas Debarre, Michael Unser
Continuous Time Regression Problem Lipschitz Network Sparse Local Model Regression Framework Lipschitz Constraint

December 7, 2021

Bridging the Model-Reality Gap with Lipschitz Network Adaptation
Siqi Zhou, Karime Pereida, Wenda Zhao, Angela P. Schoellig
Model Based Unknown Dynamic Uncertain Dynamic Lipschitz Network Model Reference Adaptive