BDNet: Fast-slow thinking is all you need

Chenzhi Zheng; Zhuoran Zheng

cs.CVOct 30, 2026

BDNet: Fast-slow thinking is all you need

Authors: Chenzhi Zheng, Zhuoran Zheng

Organizations: 1Jinan University · 2National University of Defense Technology

Abstract

We present BDNet, a hierarchical image-classification backbone built around fast image features and slow statistical references. Its baseline deformation operator (BDO) transforms the deviation of a feature from a stage-specific reference through a low-rank projection and a learned channel scale. Rather than computing pairwise token similarity, the branch operates directly on baseline-relative residuals with complexity linear in the token count. Each of four reference buffers accumulates detached stage statistics during training and remains frozen at inference. A four-stage pyramid with depths $(2,2,6,2)$ and widths $(96,192,384,768)$ yields a 20.70M-parameter classifier at $224\times224$ resolution. We organize an ImageNet-1K study around clean classification, eight controlled ablations, and Gaussian-noise, contrast, and brightness perturbations. The central hypothesis is that a slow reference can improve the robustness of a compact representation while retaining a simple tensor implementation. We distinguish this training-time statistical mechanism from input-dependent attention and make its linear inference behavior explicit.

Figures & tables

Figure 1: Motivation of BDNet. (a) The residual is defined relative to a slow reference: alignment produces no BDO correction. (b) Image features and reference statistics have different update rates. (c) Inference processes one image using frozen stage references.

Figure 2: Overview of BDNet. (a) Four stages maintain independent slow references; the three inter-stage transitions use

2\times 2

stride-two convolutions. (b) A Pre-LN block adds the BDO correction once, followed by a GELU MLP residual. (c) The BDO branch projects a baseline-relative residual from

D

r

and back, then scales channels by

\gamma

. (d) A detached stage-end EMA updates each reference only during training. Solid arrows denote features; dashed arrows denote statistical/reference paths.

Stage	Grid	$D$	Blocks	$r$
S1	$56\times 56$	96	2	24
S2	$28\times 28$	192	2	48
S3	$14\times 14$	384	6	96
S4	$7\times 7$	768	2	192

Table 1: BDNet specification for a

224\times 224

image. Each transition after S1 is a

2\times 2

stride-two convolution.

Model	Backbone family	Params (M)	Top-1 (%)	Clean-value provenance
DeiT-S [ 10 ]	Dot-product attention	22.05	79.8	Paper, Table 5; no distillation
VMamba-S [ 4 ]	Visual selective state space	50.00	83.6	Official s2l15 small release
ConvNeXt-T [ 7 ]	Local convolution	28.59	82.1	Official ImageNet-1K release
Mixer-B/16 [ 9 ]	Token/channel MLP mixing	59.88	76.44	Paper, Table 3; ImageNet-only
BDNet	Fast–slow baseline deformation	20.70	80.2	Planning scenario

Table 2: ImageNet-1K classification at

224\times 224

, without external pretraining.

ID	Branch	Slow $s$	$\alpha$	Rank	Params (M)	Top-1	Top-5	Noise	Contrast	Bright.
A	BDO	Yes	0.95	D/4	20.70	80.2	95.3	65.6	71.8	75.1
B	Zero-reference BDO	No	–	D/4	20.70	78.0	94.2	59.3	66.8	70.9
C	CBAM	No	–	–	20.70	79.1	94.8	62.1	69.2	72.5
D	Mamba branch	No	–	–	33.58	80.1	95.2	62.7	68.9	72.4
E	Budgeted MHA	No	–	–	20.70	80.4	95.4	61.9	68.1	71.7
F	BDO	Yes	0.90	D/4	20.70	79.7	95.0	64.9	70.8	74.1

Table 3: A–H controlled design and classification/robustness scenario. Noise, contrast, and brightness are mean Top-1 across the severities in Table 4 . Parameters follow the configured modules; D is an analytic Mamba-branch parameter count.

		Gaussian noise $\sigma$			Contrast $c$			Brightness $b$
ID	Clean	0.05	0.10	0.20	0.75	0.50	0.25	$-0.20$	$-0.10$	$+0.10$	$+0.20$
A	80.2	76.4	68.3	52.1	77.6	73.2	64.6	72.8	77.8	77.6	72.2
B	78.0	72.1	61.6	44.2	74.4	68.7	57.3	68.2	74.5	73.9	67.0
C	79.1	73.8	64.5	47.9	75.9	70.7	61.0	69.8	75.8	75.5	68.9
D	80.1	74.9	65.3	47.9	76.4	70.6	59.7	70.1	76.2	75.8	67.5
E	80.4	74.7	64.6	46.4	76.2	69.8	58.3	69.5	75.9	75.3	66.1

Table 4: Per-severity Top-1 scenario for the prescribed controlled corruptions. Gaussian noise uses standard deviation

\sigma

; contrast uses multiplier

c

; brightness uses additive shift

b

[0,1]

pixels. All slow references remain frozen.

Image Classification

Image classification assigns semantic labels to images, enabling recognition of objects, scenes, conditions, and fine-grained categories.

See all 417 papers

AnyDepth-DETR/-YOLO: Any-depth object detection with a single network
May 10, 2026Woochul Kang, Hyungseop Lee, Jiho LeeEfficient Neural Network Inference Dynamic Neural Networks
Incheon Nat’l Univ.
PPS: Plug-and-Play Saccadic Vision for Fine-Grained Classification
Sep 19, 2025Johann Schmidt, Sebastian Stober, Joachim Denzler +1Fine-Grained Image Classification Visual Attention
Artificial Intelligence Lab Otto-von-Guericke University Magdeburg Germany · Computer Vision Group Friedrich Schiller University Jena Germany
Green-ELM: Efficient Analytic Learning via High-Dimensional Random Projections
Apr 17, 2026Wladimir SilvaNeural Network Optimization Random Feature Methods
Department of Computer Engineering North Carolina State University

May 10, 2026J/K move · Enter open · S save

AnyDepth-DETR/-YOLO: Any-depth object detection with a single network

Woochul Kang, Hyungseop Lee, Jiho Lee

Incheon Nat’l Univ.

Efficient Neural Network Inference Dynamic Neural Networks Efficient Object Detection Object Detection Real-Time Object Detection

Open paper AskPDF arXiv

Modern object detectors are static, fixed-depth networks optimized for a single operating point, requiring separate models for different deployment scenarios. We present an any-depth detection framework that enables a single network to span a continuous range of accuracy--efficiency trade-offs by controlling depth at inference time without retraining. Each backbone and neck stage is divided into an essential path, which always executes, and a skippable refinement path; this decomposition preserves the full multi-scale feature hierarchy at every depth configuration, unlike conventional early exiting that discards entire stages. To train such a network, jointly optimizing many sub-networks of varying depth introduces conflicting gradient signals. We address this via self-distillation between only the two extremes, with prediction-level and feature-level alignment losses that enforce stage-wise modularity, ensuring the outputs of each stage remain compatible regardless of the paths taken. Instantiated on RT-DETR and YOLOv12, our full-depth configurations match or surpass their respective SOTA baselines with negligible parameter overhead, while the most efficient configurations achieve up to

1.82\times

speedup at a cost of only 2.0 AP, all from a single set of weights.

Abstract

Image Classification

Explore similar work

AnyDepth-DETR/-YOLO: Any-depth object detection with a single network

PPS: Plug-and-Play Saccadic Vision for Fine-Grained Classification

Green-ELM: Efficient Analytic Learning via High-Dimensional Random Projections