We consider inverse problems regularized by a trained neural network with convex activations and unrestricted weights. The resulting optimization problem is generally nonconvex, and gradient-based methods may converge to poor local minima. We introduce a penalized epigraph lifting of the network and show that the lifted problem admits a difference-of-convex (DC) decomposition. For Lipschitz continuous activations, we establish finite exactness: the original objective is recovered exactly at a finite and explicitly computable penalty threshold ρˉ. Building on this structure, we propose H-DC (Homotopic DC), a homotopic scheme that solves a sequence of DC subproblems with increasing ρ, warm-starting each level from the previous solution. Numerical experiments on synthetic problems, a manifold denoiser regularizer, and image inpainting with a learned Mumford--Shah regularizer show that H-DC consistently escapes poor local minima and reaches solutions with lower objective values than gradient-based methods from the same initialization.
Figures & tables
Figure 1: Visual comparison between the homotopy defined in 5 and the Gaussian homotopy, for a function of the form the objective value of ( P ), where R is a neural network randomly generated as described in Section 4.1 . We observe the properties established in Proposition 1 : Gρ always lies below the target function, and Gρ is monotonically increasing with respect to ρ .
Figure 2: Four randomly generated 1D instances. The penalized lifted objectives Gρ (dashed) and oribinal objective (black) are shown, along with solutions of H-DC (purple diamond), subgradient (orange circle) and Gaussian homotopy (green triangle).
Figure 3: Right: Comparison of the optimality gap obtained with Adam and H-DC and Gaussian homotopy in dimension one. Left: Same comparison in dimension two. The median is shown by a solid black line, while the mean is shown by a dashed line.
Figure 4: Optimization trajectories from a 5×5 grid of initializations for H-DC (left) and Adam (right). Ground truth x∗ marked by a red star.
Figure 5: Inpainting results (30% pixels kept). From left to right: ground truth, observation, Adam with learned R (PSNR=15.4 dB), Adam with exact RMS (PSNR=19.2 dB), H-DC with learned R (PSNR=23.6 dB).
Figure 6: Objective value vs. time for H-DC and Adam with learned Rθ ( λ=0.02 ).
Method
Obj ↓
PSNR (dB) ↑
Adam
1.47±0.25
17.00±1.72
H-DC
1.19±0.24
23.53±1.81
Table 1: Average results over 10 CIFAR-10 test images, inpainting with 30% pixels kept, learned R ( λ=0.02 ).
Nonconvex optimization develops methods for problems whose objectives or constraints can have multiple local optima, saddle points, or nonsmooth structure.
School of Artificial Intelligence, Jilin University, China · Graduate School of Informatics, Kyoto University, JPN · Department of Civil and Systems Engineering, Johns Hopkins University, USA