Where Global Context and Weight Sharing Work in Compact Vision Backbones
Organizations: GE Appliances, a Haier company
Abstract
Compact convolutional backbones such as EfficientNet achieve strong parameter efficiency through local depthwise separable convolutions, yet two structural questions remain central to hybrid design: where does global self-attention improve recognition over pure convolution, and can repeated blocks share weights without losing representational capacity? We investigate both questions on ImageNet-1k (50,000-image validation) using controlled EfficientNet-B0 experiments. First, we show that wrapping a classification backbone in multi-scale feature pyramids (BiFPN) fails to outperform a plain MBConv trunk (69.99% - 71.81% vs. 72.86% plain B0) until paired with stage attention (73.15%). In contrast, keeping the MBConv trunk intact and inserting lightweight zero-gated residual self-attention adapters at stages P3 (28x28 windowed), P4 (14x14 global), and P5 (7x7 global) reaches 74.02% standalone and outperforms a bit-for-bit matched plain-B0 control by +0.954 percentage points (73.88% vs. 72.92%) across an 88-epoch stateless replay schedule, with learned attention amplitude rising monotonically with semantic depth (27.2% at P3 -> 45.4% at P4 -> 66.3% at P5). Second, we evaluate recurrent weight sharing across repeated MBConv blocks with pass-specific BatchNorm statistics (+1,536 parameters per pass). While post-hoc kernel averaging degrades accuracy to 17.03% and single-template stage tying (-27.6% parameters) plateaus at 66.10%, alternating two shared block templates ([B1, A, B, A]) in Stage 6 cuts total model parameters by 10.9% (582K weights) and recovers by +10.82 points in a single epoch to 70.92% (-1.84 points vs. unshared control).
Figures & tables
| Cohort / Group | Model Architecture | Input | Sel. Ep | Top-1 / Top-5 (%) | Key Architectural Comparison |
|---|---|---|---|---|---|
| Scale Baselines | Plain EfficientNet-B0 (A4 Ref) | e86 / 88 | 72.86 / 91.11 | Primary B0 reference ( params) | |
| (88-Epoch Ref) | Plain EfficientNet-B1 (A7 Ref) | e87 / 88 | 74.91 / 92.19 | Compound scale ( params) | |
| Plain EfficientNet-B2 (A5 Ref) | e88 / 88 | 75.95 / 92.75 | Largest reference scale ( params) | ||
| BiFPN Hybrids | B0 + Fresh BiFPN Neck (A2) | e90 / 90 | 69.99 / 89.29 | Warm trunk + fresh BiFPN lags plain B0 | |
| B0 + BiFPN Joint Recovery (A3) | e87 / 87 | 71.81 / 90.38 | Joint BiFPN trails plain B0 ( pp) | ||
| B0 + BiFPN + Stage Attention (A1) | e86 / 87 | 73.15 / 91.28 | Adding attention lifts BiFPN by pp |
| Weight-Sharing Architecture | Stage 6 Pattern | Params | Param | Zero-Shot | + 1K BN | 1-Ep Cont. | vs. Ctrl |
|---|---|---|---|---|---|---|---|
| Unshared FP32 Baseline / Control | 5.320M | 72.87% | 70.52% | 72.76% | pp | ||
| Stages 4–6 Single-Template (Avg Init) | 3.851M | 0.72% | 17.03% | — | — | ||
| Stages 4–6 Single-Template (Copy ) | 3.851M | 7.36% | 47.86% | 66.10% | pp | ||
| Stage 6 Single-Template (Avg Init) | 4.155M | 14.84% | 52.53% | — | — | ||
| Stage 6 Alternating Pair (4-Block) | 4.738M | 28.34% | 60.10% | 70.92% | pp | ||
| Stage 6 Alternating Pair (5-Block Depth) | 4.739M | 14.37% | 58.49% | 70.83% | pp |
Lightweight CNNs
Lightweight convolutional neural networks reduce model size and computation for efficient inference on mobile, embedded, and other constrained devices.
See all 62 papers