Input convex neural networks provide a natural way to enforce convexity globally through architectural constraints. We study how this structure-preserving approximation framework extends to semiconvex, quasiconvex, and convex transformable functions. For C1,1 convex functions on compact convex domains, we establish the sharp W1,p minimax rate m−1/d under max-affine approximation with m affine pieces, and show that every max-affine function admits an exact bounded-width ICNN representation with complexity linear in m. The same W1,p approximation rate extends to semiconvex functions through quadratic lifting and to convex transformable functions through monotone composition. For general Lipschitz quasiconvex functions, we develop a quantitative finite-probe approximation theory based on monotone responses to affine projections and obtain a structure-preserving IqCNN with an L∞ approximation guarantee. We further show that these approximation results extend to ReLU-transferable smooth activations, including Softplus and ELU. Numerical experiments on PDE-based approximation problems compare the proposed architectures with parameter-matched MLPs and validation-tuned shape penalties, demonstrating global structural validity without penalty tuning while maintaining competitive approximation accuracy. Overall, the results identify recoverable convex latent structure as the key mechanism enabling gradient-level structure-preserving approximation beyond convexity.
Figures & tables
Function class
Latent structure
Structured model
Norm
Complexity
Convex
Convex
Max-affine / ICNN
W1,p
O(ε−d)
Semiconvex
Quadratic lift to convex
IsCNN
W1,p
O(ε−d)
Convex transformable
Monotone transform of convex
IctNN
W1,p
O(ε−d)
General quasiconvex
Family of affine probes
IqCNN
L∞
O(ε−(d+1))
Table 1: Structure-preserving approximation regimes considered in this paper. Each neural-network result holds for ReLU and for ReLU-transferable activations, including Softplus and ELU, with the same approximation order. Here ε denotes the target approximation error.
d
model
L2
grad. L2
W1,2
λmin
2
ICNN
.176 (.004)
.319 (.010)
.364 (.010)
+.306 (.011)
MLP
.400 (.008)
.921 (.014)
1.004 (.016)
−.696 (.045)
MLP+convex-reg
.221 (.051)
.745 (.154)
.777 (.162)
−.121 (.004)
3
ICNN
.195 (.014)
.334 (.021)
.386 (.025)
+.297 (.023)
MLP
.676 (.161)
1.672 (.296)
1.804 (.335)
−1.016 (.130)
MLP+convex-reg
.428 (.148)
1.226 (.164)
1.300 (.204)
−.251 (.048)
Table 2: Monge–Ampère test errors and sampled minimum Hessian eigenvalue, reported as mean (standard deviation) over three runs. Training uses only PDE and boundary residuals.
Figure 1: Monge–Ampère accuracy (left) and the 1% quantile of the samplewise minimum Hessian eigenvalue (right). ICNN remains on the elliptic, convex branch by construction; a Hessian-regularized MLP reduces but does not eliminate violations.
d
model
L2
grad. L2
W1,2
violation
2
IsCNN
.0072 (.0009)
.0298 (.0042)
.0307 (.0042)
0
MLP
.0025 (.0008)
.0123 (.0033)
.0126 (.0034)
8.01 (1.74)%
MLP+Hessian-reg
.0080 (.0014)
.0321 (.0048)
.0330 (.0050)
.33 (.30)%
3
IsCNN
.0117 (.0012)
.0476 (.0041)
.0490 (.0042)
0
MLP
.0089 (.0030)
.0352 (.0100)
.0363 (.0104)
20.18 (10.16)%
MLP+Hessian-reg
.0175 (.0080)
.0592 (.0213)
.0618 (.0226)
.46 (.63)%
Table 3: Hamilton–Jacobi snapshot errors and fraction of test Hessians that violate D2F+ρtI⪰0 , as mean (standard deviation).
Figure 2: HJ Sobolev error (left) and the fraction of sampled lifted Hessians that violate semiconvexity (right). IsCNN lies exactly at zero violation; the regularized MLP approaches zero only on the sampled diagnostic.
d
model
L2
grad. L2
W1,2
random fail.
Δqc
2
IqCNN
.0196 (.0099)
.0974 (.0284)
.0994 (.0299)
0
0
MLP
.0244 (.0012)
.1168 (.0052)
.1193 (.0053)
.025 (.030)%
.0376 (.0374)
MLP+levelset-reg
.0199 (.0022)
.1011 (.0069)
.1030 (.0071)
0
0
3
IqCNN
.0187 (.0092)
.0914 (.0277)
.0933 (.0290)
0
0
MLP
.0205 (.0020)
.1017 (.0076)
.1038 (.0078)
.0028 (.0048)%
.0609 (.0160)
MLP+levelset-reg
.0195 (.0009)
.0939 (.0024)
.0959 (.0025)
0
0
Table 4: Brascamp–Lieb profile evaluation on the larger cube [−1.5,1.5]d . Entries are mean (standard deviation); random failure rates and Δqc measure quasiconvex/sublevel-set violation.
Figure 3: Accuracy and adversarial sublevel-set violation on the Brascamp–Lieb profiles outside the training cube. The right panel uses an ordinary linear axis, so exact zeros for IqCNN are displayed at zero.
d
model
params.
WR=11,2
WR=21,2
negative D2eF
8
IctNN
1069
.0458 (.0017)
.165 (.017)
latent cert.
MLP
1041
.312 (.131)
.320 (.148)
68.10 (14.42)%
MLP+CT-reg
1041
.177 (.114)
.193 (.071)
0
16
IctNN
1717
.0672 (.0017)
.341 (.002)
latent cert.
MLP
1751
.301 (.040)
.375 (.055)
84.11 (7.05)%
MLP+CT-reg
1751
.237 (.047)
.266 (.065)
.52 (.90)%
Table 5: Convex-transformable target: parameter-matched W1,2 errors on the training cube and the larger cube, with the fraction of negative eigenvalues of D2exp(F) at R=2 . Entries are mean (standard deviation) over three independent trainings. “latent cert.” denotes IctNN’s global learned certificate; exp need not be its learned transform.
Figure 4: Convex-transformable target in dimensions 16 and 32 (mean and one standard deviation over three independent trainings). The dashed line is the training boundary. IctNN is most accurate in-domain and retains a clear advantage through radius 2 in d=32 .
Figure 5: Validation selection of the supervised shape penalties in d=8 . Each quantity is divided by its value for the unregularized MLP ( λ=0 ), so lower is better and one means no change. Curves and error bars are the mean and one standard deviation from three independently trained validation runs. Orange circles show W1,2 error, green squares show the remaining structural violation, and blue rings mark the frozen choices: λ=100 in B–C and λ=1000 in D.
School of Future Technology Xi’an Jiaotong University Xi’an, China · School of Future Technology School of Automation Science and Engineering Xi’an Jiaotong University Xi’an, China · School of Automation Science and Engineering Xi’an Jiaotong University Xi’an, China