math.OCNov 5, 2026

A Bilevel Neurodynamic Approach for Distributed SCAD-Regularized Sparse Least Squares

Authors: Juntao Li, Qihang Guo, Deyuan Meng

Organizations: School of Mathematics and Statistics, Henan Normal University, Xinxiang 453007, P. R. China · Seventh Research Division, School of Automation Science and Electrical Engineering, Beihang University (BUAA), Beijing 100191, P. R. China

Abstract

This paper proposes a bilevel particle swarm optimization (PSO)-neurodynamic method for solving the distributed sparse least-squares problem regularized by the smoothly clipped absolute deviation (SCAD) penalty to global optimality. The SCAD penalty is nonsmooth and nonconvex, so existing distributed neurodynamic methods have not been extended to it directly. Through two-time-scale singular perturbation analysis, a rigorous full-trajectory convergence theory with explicit convergence rates is established for the inner collaborative multi-agent neurodynamic system without requiring the second-order sufficient condition. A theory-guided inner-outer collaborative optimization framework is further developed, in which a red-light green-light sensitivity mechanism is embedded in the outer PSO search to enhance global search while concentrating computational effort on the SCAD shape parameters, thereby improving recovery accuracy and computational efficiency. Simulation results on four sparse-recovery scenarios demonstrate that the proposed method accurately recovers the true sparsity pattern and achieves faster convergence and lower relative error than competing neurodynamic methods.

Figures & tables

Explore similar work

Nov 5, 2026

Projection-Free Gradient Tracking Neurodynamic for Distributed Nonnegative Sparse Optimization

This paper proposes a continuous-time, projection-free neurodynamic system based on gradient tracking for distributed nonnegative sparse optimization. Unlike traditional methods, our approach avoids projecting states onto the feasible set. Instead, it employs gradient tracking to estimate average gradients under heterogeneous costs, coupling primal and dual dynamics such that feasibility and sparsity emerge naturally from the equilibrium conditions. The system ensures unique solutions through a locally Lipschitz vector field and guarantees that every equilibrium corresponds to an exact KKT point. We establish rigorous convergence guarantees for convex objectives through three distinct analytical layers. These include a centralized kernel rate, a computable distributed rate derived from Jacobian analysis without restrictive stability assumptions, and a global exponential stability proof based on Lur'e system theory. Our analysis quantitatively separates these layers, revealing that the convergence rate is primarily limited by graph sparsity and remains robust to variations in momentum time scales. Numerical experiments on strongly convex quadratic problems validate the theoretical rates, demonstrate effective active set recovery, and show that the system maintains stable tail convergence rates even under strong local heterogeneity.
May 19, 2026

Convergence of Consensus-Based Particle Methods for Nonconvex Bi-Level Optimization

In this paper, we study a consensus-based optimization method for nonconvex bi-level optimization, where the objective is to minimize an upper-level function over the set of global minimizers of a lower-level problem. The proposed approach is derivative-free, and constructs its consensus point via smooth quantile selection combined with a Gibbs-type Laplace approximation. We establish convergence guarantees for both the associated mean-field dynamics and its finite-particle approximation. In particular, under suitable assumptions on smooth quantile localization, error bounds, and stability, we show that the mean-field law reaches any arbitrary prescribed Wasserstein neighborhood of the target bi-level solution with an explicit exponential rate up to the hitting time. Numerical experiments on a two-dimensional constrained problem and neural network training further support the theoretical results.
Aug 24, 2026

SGHA: A Single-Loop Fully First-Order Algorithm for Nonconvex-Strongly-Convex Bilevel Optimization

In this work, we study the oracle complexity of finding an εε-stationary point for nonconvex-strongly-convex (NC-SC) bilevel optimization using only first-order oracles. Existing methods achieving the best-known complexity guarantees typically rely on double-loop, penalty-based procedures. We propose a novel single-loop algorithm based on a constrained reformulation in which lower-level stationarity is imposed as a constraint. Specifically, we construct a regularized Lagrangian by introducing a quadratic regularizer and restricting the dual variable to a bounded domain, and then apply Smoothed Gradient Descent Ascent [Zhang et al., 2020], with Hessian-vector products approximated via finite differences of gradients. We refer to the resulting deterministic and stochastic algorithms as SGHA and Stoc-SGHA, respectively. In the deterministic setting, SGHA achieves an oracle complexity of O(κˉy5ε−2)O(\barκ_y^{5}ε^{-2}), where κˉy\barκ_y denotes the relevant condition number. In the stochastic setting, Stoc-SGHA achieves an oracle complexity of O(κˉy17ε−6ρ−3)O\left(\barκ_y^{17}ε^{-6}ρ^{-3}\right) with probability at least 1−ρ1-ρ for any ρ∈(0,1)ρ\in(0,1), and an oracle complexity of O(κˉy17ε−6)O\left(\barκ_y^{17}ε^{-6}\right) in expectation under an additional bounded-iterate assumption. Moreover, under an additional stochastic smoothness assumption imposed only on the lower-level objective, the stochastic oracle complexity of Stoc-SGHA improves to O(κˉy11ε−4ρ−2)O\left(\barκ_y^{11}ε^{-4}ρ^{-2}\right) with high probability and O(κˉy11ε−4)O\left(\barκ_y^{11}ε^{-4}\right) in expectation, matching the εε-dependence of the lower bounds.