Circuit realization and hardware linearization of monotone operator equilibrium networks

Thomas Chaffey

eess.SYSep 17, 2025

Circuit realization and hardware linearization of monotone operator equilibrium networks

Authors: Thomas Chaffey

Organizations: School of Electrical and Computer Engineering, University of Sydney, New South Wales, Australia

Abstract

It is shown that the port behavior of a resistor-diode network corresponds to the solution of a ReLU monotone operator equilibrium network (a neural network in the limit of infinite depth), giving a parsimonious construction of a neural network in analog hardware. We furthermore show that the gradient of such a circuit can be computed directly in hardware, using a procedure we call hardware linearization. This allows the network to be trained in hardware, which we demonstrate with a device-level circuit simulation. We extend the results to cascades of resistor-diode networks, which can be used to implement feedforward and other asymmetric networks. We finally show that different nonlinear elements give rise to different activation functions, and introduce the novel diode ReLU which is induced by a non-ideal diode model.

Explore similar work

Complete Identification of Deep ReLU Networks through Łukasiewicz Logic
Jan 30, 2026Yani Zhang, Helmut BölcskeiRectified Linear Unit Networks Logic
Chair for Mathematical Information Science, ETH Zürich
Lock-in EP: An In-Situ Training Algorithm for Oscillatory Hardware
Oct 5, 2026Sowjanya Tammali, Wilkie Olin-AmmentorpAnalog Oscillatory Dynamics
Computer Science, Missouri University of Science and Technology, 500 West 15th Street, Rolla, 65409, MO, USA. · Mathematics & Computer Science Division, Argonne National Laboratory, 9700 S. Cass Avenue, Lemont, 60439, IL, USA.
Physical Analogue Kolmogorov-Arnold Networks based on Reconfigurable Nonlinear-Processing Units
Feb 7, 2026Manuel Escudero, Mohamadreza Zolfagharinejad, Sjoerd van den Belt +2Kolmogorov-Arnold Networks Neuromorphic Computing
1NanoElectronics Group, MESA+ Institute and BRAINS Center for Brain-Inspired Computing, University of Twente, P.O. Box 217, Enschede,2026 7500 AE, The Netherlands. · 2CAESUniversity GroupofandTwente,BRAINSEnschede,Center 7500for Brain-InspiredAE, The Netherlands.Computing · Institute of Physics, University of Münster, Münster, 48149 Germany.

Jan 30, 2026, cs.AIJ/K move · Enter open · S save

Complete Identification of Deep ReLU Networks through Łukasiewicz Logic

Yani Zhang, Helmut Bölcskei

Chair for Mathematical Information Science, ETH Zürich

Rectified Linear Unit Networks Logic Axiom Task Arithmetic Completion

Open paper AskPDF arXiv

Two deep ReLU networks can have entirely different architectures and parameters, yet realize the same function. We provide a complete characterization of this nonuniqueness. This is effected by building a symbolic calculus for deep ReLU networks, equivalence and simplification of networks becoming derivation of formulae, in close parallel to Shannon's analysis of switching circuits through Boolean logic. Inspired by Shannon, who turned circuit synthesis into the manipulation of Boolean formulae by the axioms of Boolean algebra, we turn ReLU network identification into the derivation of Łukasiewicz formulae by the axioms of many-valued (MV) logic. Two non-degenerate ReLU networks realize the same function on the unit cube if and only if one is obtained from the other by finitely many applications of the MV axioms for integer weights and biases, the divisible MV axioms for rational ones, and the Riesz MV axioms for real ones. The MV logic axioms characterize all symmetries of ReLU networks, the single-layer ones, which for tanh networks are the only kind, and the deep ones, spanning three or more layers. Our framework consists of three steps, an extraction algorithm turning a network into a substitution graph, whose represented formula has the network's input-output map as its truth function, a completeness theorem, by which functionally equivalent formulae are interderivable, and a construction algorithm returning from graphs to networks. The substitution graph is layered, carrying at each node a formula in the variables of the layer feeding it, encodes the network uniquely, and induces a new normal form for MV logic, compositional rather than flat as in the literature, hence retaining the algebraic structure of the network, with three local operations--node rewrite, layer collapse, layer expansion--realizing every derivation.

Abstract

Explore similar work

Complete Identification of Deep ReLU Networks through Łukasiewicz Logic

Lock-in EP: An In-Situ Training Algorithm for Oscillatory Hardware

Physical Analogue Kolmogorov-Arnold Networks based on Reconfigurable Nonlinear-Processing Units