Authors: Osama Orabi, Artur Zagitov, Hadi Salloum, Viktor A. Lobachev, Kasymkhan Khubiev, Yaroslav Kholodov
Organizations: Department of Computer Science and Engineering, Innopolis University, 420500 Innopolis, Russia · Q Deep, Laboratory of Quantum Computing, 420502 Innopolis, Russia · Research Center of the Artificial Intelligence Institute, Innopolis University, 420500 Innopolis, Russia · Moscow Institute of Physics and Technology, 141701 Dolgoprudny, Russia · Sirius University of Science and Technology, 354340 Sochi, Russia
Neural network pruning can be formulated as a combinatorial optimization problem, yet most existing approaches rely on greedy heuristics that ignore complex interactions between filters. Formal optimization methods such as Quadratic Unconstrained Binary Optimization (QUBO) provide a principled alternative but have so far underperformed due to oversimplified objective formulations based on metrics like the L1-norm. In this work, we propose a unified Hybrid QUBO framework that bridges heuristic importance estimation with global combinatorial optimization. Our formulation integrates gradient-aware sensitivity metrics - specifically first-order Taylor and second-order Fisher information - into the linear term, while utilizing data-driven activation similarity in the quadratic term. This allows the QUBO objective to jointly capture individual filter relevance and inter-filter functional redundancy. We further introduce a dynamic capacity-driven search to strictly enforce target sparsity without distorting the optimization landscape. Finally, we employ a two-stage pipeline featuring a Tensor-Train (TT) Refinement stage - a gradient-free optimizer that fine-tunes the QUBO-derived solution directly against the true evaluation metric. Experiments on the SIDD image denoising dataset demonstrate that the proposed Hybrid QUBO significantly outperforms both greedy Taylor pruning and traditional L1-based QUBO, with TT Refinement providing further consistent gains at appropriate combinatorial scales. This highlights the potential of hybrid combinatorial formulations for robust, scalable, and interpretable neural network compression.
Figures & tables
Figure 1: Overview of the proposed hybrid pruning pipeline. The process begins by analyzing a dense baseline model to compute gradient sensitivities and activation redundancy (Stages 1-2). These heuristics populate a Hybrid QUBO matrix, which is solved using a dynamic capacity search (Stage 3) to navigate the discrete energy landscape (Stage 4). The highest-quality mask seeds a Tensor-Train (PROTES) refinement stage (Stage 5) to directly optimize the true non-differentiable validation metric, yielding the final optimized network (Stages 6-7).
Figure 2: Hyperparameter search landscape showing validation PSNR across different coefficient samplings (e.g., log10(αT) vs. log10(αF) ). Each point represents a sampled configuration evaluated at a fixed sparsity. Strong solutions form continuous manifolds, indicating that performance relies fundamentally on the balance between competing energy terms.
Method
PSNR (dB)
SSIM
Baseline (Unpruned)
37.2632
0.9327
Classic QUBO (L1)
24.7963
0.6346
Taylor (Greedy)
34.8082
0.8974
Hybrid QUBO (Ours)
35.0715
0.9140
TT Refinement (Ours)
35.0715
0.9140
Table 1: Results for Full Model (36% Pruning). The Hybrid QUBO significantly outperforms the Taylor baseline.
Method ( N=4 Layers)
PSNR (dB)
SSIM
Baseline (Unpruned)
37.2632
0.9316
Taylor (Greedy)
37.0546
0.9284
Hybrid QUBO
37.1742
0.9300
TT Refinement
37.1893
0.9300
Table 2: Results for pruning 4 layers (60 total filters pruned).
Method ( N=8 Layers)
PSNR (dB)
SSIM
Baseline (Unpruned)
36.6977
0.9258
Taylor (Greedy)
33.9939
0.8912
Hybrid QUBO
35.4934
0.9091
TT Refinement
35.5075
0.9093
Table 3: Results for pruning 8 layers (200 total filters pruned).
Method ( N=16 Layers)
PSNR (dB)
SSIM
Baseline (Unpruned)
36.0683
0.9163
Taylor (Greedy)
33.8815
0.8836
Hybrid QUBO
34.4419
0.8953
TT Refinement
34.5659
0.8973
Table 4: Results for pruning 16 layers (480 total filters pruned).
Figure 3: PSNR comparison across 7 independent optimization runs. The TT Refinement consistently maintains a measurable advantage over the single-stage QUBO and the baseline Taylor methods.
Figure 4: Training and validation history of the baseline Half-UNet model over 100 epochs. The plots display the stable convergence of the custom Loss (top), Peak Signal-to-Noise Ratio (middle), and Structural Similarity Index Measure (bottom).
Figure 5: Qualitative comparison of image denoising results. Our highly compressed model (Hybrid QUBO + TT Refinement) maintains the visual structure and fidelity of the unpruned baseline without introducing pruning artefacts.