Monotone Neural Network

Monotone neural networks are neural network architectures designed to learn functions with specific monotonicity properties, ensuring that the output increases (or decreases) monotonically with respect to the input. Current research focuses on developing novel architectures like monotone gradient networks (mGradNets) and algorithms that enforce monotonicity, often employing techniques like specialized loss functions or iterative methods such as the Forward-Backward-Forward algorithm. This area is significant because monotone functions are crucial in various applications, including solving inverse problems, optimization, and generative modeling, offering improved robustness, convergence guarantees, and memory efficiency compared to traditional methods. The resulting models find use in diverse fields like image processing, MRI acceleration, and stochastic optimization.

Papers

November 5, 2024

Neural Networks and (Virtual) Extended Formulations
Christoph Hertrich, Georg Loho
Neural Network Lower Bound Virtual Co Presence Complexity Class Monotone Neural Network Implicit Formulation Domain Complexity

April 10, 2024

Gradient Networks
Shreyas Chaudhari, Srinivasa Pranav, José M. F. Moura
Gradient Method Monotone Neural Network Gradient Based Parameter Network Gradient

March 30, 2024

Learning truly monotone operators with applications to nonlinear inverse problems
Younes Belkouchi, Jean-Christophe Pesquet, Audrey Repetti, Hugues Talbot
Financial Application Inverse Problem Forward Forward Algorithm Monotone Neural Network Monotone Inclusion Monotone Operator

December 24, 2023

Semi-Bandit Learning for Monotone Stochastic Optimization
Arpit Agarwal, Rohan Ghuge, Viswanath Nagarajan
Submodular Maximization Stochastic Optimization Monotone Neural Network Knapsack Constraint Semi Bandit

November 2, 2023

Monotone Generative Modeling via a Gromov-Monge Embedding
Wonjun Lee, Yifei Yang, Dongmian Zou, Gilad Lerman
Generative Adversarial Network Latent Space Deep Generative Model Monotone Neural Network

April 3, 2023

Accelerated parallel MRI using memory efficient and robust monotone operator learning (MOL)
Aniket Pramanik, Mathews Jacob
Memory Trace Low Rate ACCELERATION Accelerated MRI Monotone Neural Network Parallel Magnetic Resonance Imaging Monotone Operator Learning

January 25, 2023

Learning Gradients of Convex Functions with Monotone Gradient Networks
Shreyas Chaudhari, Srinivasa Pranav, José M. F. Moura
Gradient Based Convex Function Gradient Based Optimization Convex Objective Monotone Neural Network Neural Network Gradient

July 12, 2022

Size and depth of monotone neural networks: interpolation and approximation
Dan Mikulincer, Daniel Reichman
Average Approximation Large Depth Interpolation Regime Size Matter Monotone Function Monotone Neural Network

June 17, 2022

Learning a Single Neuron with Adversarial Label Noise via Gradient Descent
Ilias Diakonikolas, Vasilis Kontonis, Christos Tzamos, Nikos Zarifis
Gradient Descent Adversarial Noise Convex Loss Single Neuron Monotone Neural Network

June 6, 2022

Memory-efficient model-based deep learning with convergence and robustness guarantees
Aniket Pramanik, M. Bridget Zimmerman, Mathews Jacob
Early Stage Convergence Robustness Guarantee Computational Imaging Model Based Deep Learning Monotone Neural Network INverse Problem Memory effIcieNt

March 31, 2022

Neural Q-learning for solving PDEs
Samuel N. Cohen, Deqing Jiang, Justin Sirignano
Q Learning Ordinary Differential Equation High Dimensional Partial Differential Equation Elliptic Partial Differential Equation Monotone Neural Network Fitted Q Iteration

February 10, 2022

Monotone Learning
Olivier Bousquet, Amit Daniely, Haim Kaplan, Yishay Mansour, Shay Moran, Uri Stemmer
Generalization Capacity PAC Learning Monotone Function Monotone Neural Network

November 15, 2021

Reachability analysis of neural networks using mixed monotonicity
Pierre-Jean Meyer
Neural Network Reachability Analysis Partial Monotonicity Tight Approximation Monotone Neural Network Interval Bound Propagation Lipschitz Activation