cs.LGMar 25, 2026
SaveCircuit-Inspired High-Order Neural Networks with Unified Neural Dynamics Modeling for PDE Solving and Visual Perception
Organizations: School of Artificial Intelligence (Institute of Artificial Intelligence), Beihang University, Beijing, China. · Communication University of China, Beijing, China. · School of Automation Science and Electrical Engineering, Beihang University, Beijing, China. · Department of Computer Science and Engineering, University at Buffalo, Buffalo, New York, USA. ·
Abstract
Deep networks often rely on architectural heuristics to shape representation evolution, limiting their ability to model data governed by intrinsic dynamics. We present the Circuit-inspired High-Order Neural Network (CHONN), a modular framework that treats representation evolution as a latent potential process and increases its effective order through Kirchhoff-inspired cascade composition. A single Kirchhoff Neural Cell implements a stable first-order update, while serially composed cells form higher-order dynamical operators within one block. This construction is interpretable, numerically stable and compatible with common neural backbones. Theoretical analysis shows that cascaded cells induce end-to-end high-order operators, and controlled experiments demonstrate that intra-block high-order construction differs from generic depth stacking, especially on derivative-sensitive measures. Across steady-state operator learning, long-horizon physical forecasting and ImageNet-1K recognition, CHONN improves structural fidelity, rollout stability and visual representation learning. These results identify high-order circuit composition as a general principle for neural dynamics modeling.
Figures & tables
Figure 1: CHONN builds high-order representation evolution through Kirchhoff-inspired cascade composition. a, Neural-population analogy motivating the separation of connection, intensity and evolution in signal propagation. b, Positional encodings introduce variation across positions through externally supplied index-dependent cues. c, Recurrent state models introduce an internal state, but a single transition corresponds to a first-order update. d, CHONN composes circuit-inspired latent updates in series, allowing representation dynamics to accumulate across stages. e, A Kirchhoff Neural Cell (KNC) represents the hidden feature as a latent potential driven by input injection, leakage and coupling, followed by output readout. f, Cascading KNCs yields second- and higher-order evolution factors, making the order of latent representation dynamics explicit within one block.
Figure 2: Architectural instantiation of CHONN and its integration into neural-operator and encoder–decoder backbones. a, Structure of a single Kirchhoff Neural Cell (KNC). The hidden representation is modeled as a latent potential , where input injection, leakage dissipation and output readout jointly define a first-order latent evolution unit. b, Cascaded Kirchhoff Block (CKB). Multiple KNCs are connected in series, so that the output of each stage drives the next stage and progressively enriches the effective order of the latent dynamics. c, Trainable realization of an -stage CKB. The serial KNC cascade is embedded within a residual gated block together with normalization, projection, local mixing and nonlinear activation, making the high-order evolution module compatible with deep neural architectures. d, FNO with CHONN . CKB modules are inserted alongside spectral and convolutional transformations in the Fourier neural operator backbone, introducing explicit latent evolution into operator learning. e, U-Net with CHONN . CKB modules are placed within the encoder–decoder hierarchy, including deep downsampling, bottleneck and upsampling stages, to enrich multi-scale feature evolution. Together, these instantiations show that CHONN functions as a modular plug-in mechanism for introducing intrinsic high-order representation evolution into different backbone architectures.
Figure 3: Explicit high-order construction improves structural fidelity and changes the scaling path. a, Comparison between CHONN and a depth-stacked first-order baseline on the order-controlled benchmark. Target operators T1–T4 correspond to first- to fourth-order instances of the operator family. For each target order, physical-domain, spectral-domain and derivative relative errors are reported across model orders or baseline depths 1–4. CHONN shows more consistent improvement with increasing order, with the clearest separation from the depth-stacked baseline appearing in the derivative-sensitive metric. b, Relative reduction in derivative error achieved by CHONN over pure first-order depth stacking. Larger reductions are observed for higher target orders and stronger model configurations, indicating that explicit intra-block high-order construction is not equivalent to simply increasing the number of first-order blocks. c, Parameter–performance scaling comparison between pure first-order depth scaling (D1–D5) and stacked high-order CHONN blocks ( and ). High-order block stacking follows a more favourable parameter–error trajectory, particularly for derivative-sensitive errors, suggesting that first increasing the order within each block and then stacking such blocks is more effective than continuing to deepen a purely first-order network.
Figure 4: CHONN improves steady-state neural operator learning on Darcy flow and Poisson problems. a, Physical illustration of the Darcy-flow problem, where a spatially heterogeneous permeability field determines the pressure field and the resulting flow through the porous medium. b, Representative source term and solution field for the Poisson equation, illustrating the steady-state mapping from a prescribed spatial forcing field to the corresponding solution. c, Multi-metric quantitative comparison on the Darcy benchmark. Across RMSE, nRMSE, CSV, maximum error, boundary discrepancy, and frequency-band errors, CHONN consistently achieves lower error than FNO and U-Net. d, Relative gain of CHONN over the stronger Darcy baseline for each metric. The largest improvements appear in boundary-sensitive and frequency-related measures, indicating that the proposed model improves not only global accuracy but also structural fidelity. e, Qualitative Darcy predictions and residual maps on representative samples. CHONN better preserves the global pressure structure and reduces localized artifacts, yielding cleaner absolute- and relative-error patterns than baseline models. f, Training loss curves on the Darcy task. CHONN converges more rapidly and reaches a substantially lower loss floor, suggesting that the Kirchhoff-inspired representation leads to a more stable and efficient optimization process. g, Cascade-stage ablation on the Poisson equation. Increasing the number of passes improves overall solution-field accuracy with only moderate growth in model size, while varying cascade depths yield distinct trade-offs among relative error, nRMSE, and Max MSE. Together, these results show that circuit-inspired high-order construction improves prediction accuracy, structural consistency, optimization stability, and scaling efficiency across representative steady-state PDE benchmarks.
Figure 5: Circuit-inspired high-order construction improves spatiotemporal forecasting, long-horizon rollout stability and visual recognition. a, Representative shallow-water evolution from an initial localized height disturbance at to the propagated wave field at a later time , illustrating the spatiotemporal forecasting task. b, Multi-metric comparison on the shallow-water benchmark. CHONN is compared with U-Net and FNO across RMSE, nRMSE, CSV, maximum error, boundary discrepancy and frequency-related errors, including low-, middle- and high-frequency components. c, Relative gain of CHONN over the stronger shallow-water baseline for each metric, showing consistent improvements in global reconstruction, boundary consistency and frequency-sensitive structural fidelity. d, Representative Navier–Stokes vorticity fields at the beginning and end of a rollout sequence, showing the transition from an initial field to a more complex flow structure. e, Average 40-step rollout relative error on Navier–Stokes dynamics under different input-history settings ( , and ). CHONN consistently yields lower rollout error than FNO. f, Per-step rollout error curves for FNO and CHONN under different input-history settings. CHONN shows lower error accumulation over the rollout horizon, with a clearer advantage in later prediction steps. g, Representative natural-image samples from ImageNet-1K. h, Accuracy–complexity comparison on ImageNet-1K, with Top-1 accuracy plotted against FLOPs for representative vision backbones. CHONN reaches a favourable accuracy–complexity trade-off compared with ConvNeXt (CNX), Swin Transformer (ST), VMamba (VM), MambaVision (MV) and EfficientVMamba (EVM). i, Ablation of routing topology and circuit constraints. Series cascade yields larger gains than parallel expansion, and adding Kirchhoff-inspired constraints further improves accuracy. j, Training efficiency at 30 epochs. CHONN achieves higher early-stage Top-1 accuracy than VMamba across Tiny, Small and Base settings. Together, these results show that Kirchhoff-inspired high-order dynamics improve spatiotemporal field prediction, long-horizon rollout stability and visual representation learning across different task families.
More in this topic
Dynamic Neural Networks
Dynamic neural networks adapt their computation, capacity, structure, or internal state to changing inputs, tasks, and resource budgets.
See all 82 papers