cs.CVNov 19, 2026

Backbone Matters: Benchmarking Encoder-Decoder Networks for Semantic Segmentation of Driving Scenes on BDD100K

Authors: Ehsan Rassekh

Organizations: Institute for Advanced Studies in Basic Sciences (IASBS), Zanjan, Iran

Abstract

Semantic segmentation gives an autonomous vehicle a dense, pixel-level description of its surroundings and is a core part of scene understanding. Most segmentation networks consist of an encoder (the backbone) and a decoder, yet the backbone is often chosen without a systematic comparison. This work studies this choice on the BDD100K driving dataset. Four widely used decoders (U-Net, FPN, LinkNet and PSPNet) are combined with 13 backbones from six families (ResNet, VGG, DenseNet, Inception, MobileNet and EfficientNet), and all 52 models are trained with one common protocol. The best model, an FPN decoder with an EfficientNet-B4 backbone, reaches 62.6% mean intersection over union (mIoU), 89.8% accuracy and a loss of 0.092. Only two of the 52 models, both FPN with EfficientNet, exceed the 56.9% mIoU reported with the BDD100K benchmark. The effect of the backbone depends strongly on the decoder: the same EfficientNet-B4 gives 42.6% mIoU with PSPNet but 62.6% with FPN, and the gap between the best and the worst backbone ranges from 12.0 points (U-Net) to 42.5 points (FPN). The results and the scope of this study are discussed.

Figures & tables

Explore similar work

CardsList
  1. AD-SAM: Adapting the Segment Anything Model for Semantic Segmentation in Autonomous Driving

    Oct 30, 2025Het Patel, Mario Camarena, Fatemeh Nazari +3Vision Foundation Model AdaptationAutonomous Driving Perception

  2. What Does the Encoder Actually Decide? A Controlled Comparison of Vision Backbones on Joint Tree Segmentation and Stereo Depth

    Sep 2, 2026Yida Lin, Bing Xue, Mengjie Zhang +2Stereo Depth EstimationImage Segmentation