Conv-Reluplex: A Verification Framework For Convolution Neural Networks
Authors: Jin Xu, Zishan Li, Miaomiao Zhang, Bowen Du
Organizations: School of Software Engineering, Tongji University, Shanghai, China · Department of Computer Science, University of Warwick, Coventry, United Kingdom
In recent years, machine learning has demonstrated impressive performance in many real-world tasks, especially in computer vision and natural language processing. However, to apply them in safety-critical systems one needs formal guarantees on the neural network outputs. The Reluplex tool is proposed to verify the safety of deep neural networks (DNNs), and in case the DNN fails to give a correct output, can generate adversarial examples. Since the tool can only handle DNNs, it is necessary to extend the tool to process image data. Therefore, in this paper, we propose the Conv-Reluplex framework, which is designed to verify the convolutional layer and pooling layer in convolutional neural networks (CNNs), and generate adversarial examples when classification is misguided. We conduct several experiments on MNIST to evaluate our approaches. The results show that the original CNN is improved using the adversarial examples generated by our tool, and the precision of classification can be increased significantly.
Figures & tables
Fig. 1: Main steps of the Conv-Reluplex framework
Fig. 2: Max-pooling calculation
Fig. 3: Unpooling calculation of Max-pooling
Fig. 4: Convolution calculation
Fig. 5: Deconvolution calculation
Fig. 6: The structure of the CNN model in the experiment
TABLE I: Verification results of a test sample of number 6 in different perturbation neighborhoods
Fig. 7: Original picture and adversarial example of number 6
Fig. 8: Different adversarial examples
accuracy of original sample
accuracy of adversarial sample
Original model
94.98%
0%
Adversaria training
94.89%
99.65%
TABLE II: Accuracy comparison before and after adversarial training
classification
0
1
2
3
4
5
6
7
8
9
Original model
√
\
√
Fail
Fail
√
√
Fail
Fail
Fail
Adversaria training
√
\
√
√
√
√
√
Fail
√
Fail
TABLE III: Verification results of number 1’s test sample (δ=0.6)
classification
0
1
2
3
4
5
6
7
8
9
Original model
√
√
√
\
√
Fail
√
√
Fail
√
Adversaria training
√
√
√
\
√
√
√
√
√
√
TABLE IV: Verification results of number 3’s test sample (δ=0.6)
classification
0
1
2
3
4
5
6
7
8
9
Original model
√
√
√
√
√
Fail
\
√
Fail
√
Adversaria training
√
√
√
√
√
√
\
√
√
√
TABLE V: Verification results of number 6’s test sample (δ=0.6)
Many neural network (NN) verification systems represent the network's input-output relation as a constraint program. Sound and complete, representations involve integer constraints, for simulating the activations. Recent works convexly relax the integer constraints, improving performance, at the cost of soundness. Convex relaxations consider outputs that are unreachable by the original network. We study the worst case divergence between the original network and its convex relaxations; both qualitatively and quantitatively. The relaxations' space forms a lattice, where the top element corresponds to a full relaxation, with every neuron linearized. The bottom element corresponds to the original network. We provide analytical upper and lower bounds for the ℓ∞-distance between the fully relaxed and original outputs. This distance grows exponentially, w.r.t. the network's depth, and linearly w.r.t. the input's radius. The misclassification probability exhibits a step-like behavior, w.r.t. input radius. Our results are supported by experiments on MNIST, Fashion MNIST and random networks.
Foundation for Research and Technology – Hellas, Heraklion, Greece · University of Crete, Heraklion, Greece · Catalan Institution for Research and Advanced Studies, Barcelona, Spain +1
Black-box adversarial attacks are widely used as tools to test the robustness of deep neural networks against malicious perturbations of input data aimed at a specific change in the output of the model. Such methods, although they remain empirically effective, usually do not guarantee that an adversarial example can be found for a particular model. In this paper, we propose Contract And Conquer (CAC), an approach to provably compute adversarial examples for neural networks in a black-box manner. The method is based on knowledge distillation of a black-box model on an expanding distillation dataset and precise contraction of the adversarial example search space. CAC is supported by the transferability guarantee: we prove that the method yields an adversarial example for the black-box model within a fixed number of algorithm iterations. Experimentally, we demonstrate that the proposed approach outperforms existing state-of-the-art black-box attack methods on ImageNet dataset for different target models, including vision transformers.
Deep neural networks (DNNs) deployed on resource-constrained neuromorphic hardware face three concurrent challenges: the need for model compression through pruning, vulnerability to adversarial input perturbations, and susceptibility to hardware-induced weight faults such as stuck-at-zero errors. While each of these factors has been studied in isolation, their combined effects on model reliability have received little attention. This paper presents an empirical investigation of how pruning, adversarial training, and hardware fault injection interact to affect the robustness of convolutional neural networks. Using a compact three-layer CNN trained on MNIST, we conduct three experiments: (1) comparing the fault tolerance of naturally and adversarially trained models under simultaneous hardware faults and adversarial attacks, (2) evaluating how pruning affects adversarial robustness, and (3) characterizing the joint accuracy surface across fault rates, adversarial perturbation magnitudes, and pruning levels. Our results show that adversarial training improves robustness against input perturbations but increases sensitivity to stuck-at-zero weight faults. Contrary to intuition, pruning did not significantly increase fault sensitivity, and varying the pruning level had little effect across fault rates and attack strengths. These results highlight the need to jointly consider adversarial robustness and hardware reliability.
Manali Dangarikar, Cory Merkel
Brain Lab Rochester Institute of Technology Rochester, NY, USA