Distributed Semantic Segmentation With Improved Rate-Distortion Trade-Off

Danish Nazir; Timo Bartels; Thorsten Bagdonat; Tim Fingscheidt

cs.CVAug 26, 2026

Distributed Semantic Segmentation With Improved Rate-Distortion Trade-Off

Authors: Danish Nazir, Timo Bartels, Thorsten Bagdonat, Tim Fingscheidt

Organizations: Group Innovation Volkswagen AG Wolfsburg, Germany · Institute for Communications Technology TU Braunschweig Braunschweig, Germany

Abstract

Distributed deep neural networks (DNNs) for dense perception tasks such as semantic segmentation execute an encoder DNN on edge devices, and a decoder DNN typically on a large-scale cloud platform with a particular constraint on transmission bitrate. Recent works employ source codecs to enable bitrate-efficient transmission between the edge device and the cloud. However, as these approaches are typically bound to a particular type of source codec and alternative network architectures are often not explored, this results in a suboptimal rate-distortion (RD) trade-off in the low-bitrate regime. In this work, we propose two novel source codecs that enable extremely low bitrates, while improving RD performance. We demonstrate the effectiveness of our proposed source codecs by achieving state-of-the-art performance in distributed semantic segmentation at below 0.2 (0.03) bits per pixel, measured using the mean intersection-over-union metric on ADE20K (Cityscapes).

Figures & tables

Panel of Figure 1 — Figure 1: High-level overview of state-of-the-art distributed semantic segmentation [ 38 , 37 ] , applicable also to our proposed methods. Here, $\mathbf{E}$ and $\mathbf{JD}$ are the image encoder and the joint feature and task decoder, respectively. Block $\mathbf{FE}$ represents the feature encoder. We propose improved compression encoders and decoders ( $\mathbf{CE}$ and $\mathbf{CD}$ ), respectively, whose network architectures are detailed in Figs. 3 and 6 .

Panel of Figure 2 — Figure 2: Probability density function (PDF) p( $\overline{{\sigma}^{2}}$ ) of the mean variance $\overline{{\sigma}^{2}}=\overline{{\sigma}^{2}_{i,f}}$ , with pixel index $i$ and feature index $f$ , computed over all images in $\mathcal{D}^{\mathrm{val}}_{\mathrm{ADE20K}}$ , from the latent representation $\hat{\mathbf{r}}$ of the proposed HSM source codec vs. the state-of-the-art HS baseline source codec [ 37 , 38 ] , whose network architectures are detailed in Figs. 3 and 4 .

Panel of Figure 3 — Figure 3: Proposed HSM source codec (with $\bm{\sigma}$ and $\bm{\mu}$ ) and baseline (with $\mathbf{\bm{\mu}}$ ) network architecture with compression encoder $\mathbf{CE}$ and compression decoder $\mathbf{CD}$ , utilized in Fig. 1 . Details of the hyperprior encoder ( $\mathbf{HE}$ ) are shown in Fig. 5 (a), details of the baseline and proposed hyperprior decoder ( $\mathbf{HD}$ ) blocks are shown in Figs. 4 (a) and Figs. 4 (b), respectively.

Panel of Figure 4 — Figure 4: Details of hyperprior decoder ( $\mathbf{HD}$ ) architectures for distributed semantic segmentation. Fig. 4 (a) shows the baseline hyperprior decoder ( $\mathbf{HD}$ ), taken from [ 37 , 38 ] . Fig. 4 (b) illustrates the proposed $\mathbf{HD}$ architecture block, utilized in Fig. 3 ( HSM ) and Fig. 6 ( AR-HSM ). Details of the hyperprior standard deviation decoder ( $\mathbf{HSD}$ ) and the hyperprior mean decoder ( $\mathbf{HMD}$ ) are shown in Fig. 5 (b).

Panel of Figure 5 — Figure 5: Network architecture of the hyperprior encoder $\mathbf{HE}$ , hyperprior standard deviation decoder $\mathbf{HSD}$ (both adopted from [ 1 ] ), and the proposed hyperprior mean decoder $\mathbf{HMD}$ , used in Figs. 3 and 6 .

Panel of Figure 6 — Figure 6: Proposed AR-HSM source codec network architecture with compression encoder $\mathbf{CE}$ and compression decoder $\mathbf{CD}$ , utilized in Fig. 1 . Details of the hyperprior encoder ( $\mathbf{HE}$ ) are shown in Fig. 5 (a), details of the conventional and proposed hyperprior decoder ( $\mathbf{HD}$ ) blocks are shown in Figs. 4 (a) and Figs. 4 (b), respectively. Further details of the standard deviation refinement block $\mathbf{SDR}$ and mean refinement block $\mathbf{MDR}$ are shown in Fig. 7 .

Panel of Figure 7 — Figure 7: Standard deviation and mean refinement blocks ( $\mathbf{SDR}$ , $\mathbf{MDR}$ ) used in Fig. 6 , adopted from [ 27 ] . Swin transformer layer ( $\mathbf{STL}$ ) and convolutional block ( $\mathbf{CB}$ ) are detailed in Figs. 8 (a) and (b), respectively.

Panel of Figure 8 — Figure 8: Swin transformer layer $(\mathbf{STL})$ and convolutional block $(\mathbf{CB})$ employed in Fig. 7 , adopted from [ 27 ] .

Dataset	Official subsets	#images	Symbol
ADE20K [ 52 ]	train	20,210	$\mathcal{D}_{\mathrm{ADE20K}}^{\mathrm{train}}$
	val	2,000	$\mathcal{D}_{\mathrm{ADE20K}}^{\mathrm{val}}$
Cityscapes [ 14 ]	train	2,975	$\mathcal{D}_{\mathrm{CS}}^{\mathrm{train}}$
	val	500	$\mathcal{D}_{\mathrm{CS}}^{\mathrm{val}}$

Table 1: Datasets & splits used in our experiments.

Panel of Figure 9 — Figure 9: Illustration of our mIoU-adapted BD-rate metric , inspired from [ 8 ] , employed in Table 2 for the comparison of our proposed methods vs. various baselines.

Panel of Figure 10 — Figure 10: Performance (mIoU) of the proposed HSM and AR-HSM source codecs vs. the state-of-the-art HS source codec [ 37 , 38 ] , the ablated variant AR-HS ( $\bm{\mu}=\mathbf{0}$ ), reference method AR-HS , and no compression for distributed semantic segmentation on (a) $\mathcal{D}_{\mathrm{ADE20K}}^{\mathrm{val}}$ and on (b) $\mathcal{D}_{\mathrm{CS}}^{\mathrm{val}}$ .

Enc.	Source Codec	ADE20K $\mathcal{D}_{\mathrm{ADE20K}}^{\mathrm{val}}$	Cityscapes $\mathcal{D}_{\mathrm{CS}}^{\mathrm{val}}$
Enc.	Source Codec	BD-rate $\downarrow$ (%)	BD-rate $\downarrow$ (%)
RN-50	HS Baseline [ 37 ]	$0.0$	$0.0$
	HSM (ours)	$-\underline{17.6}$	$-\underline{41.0}$
	AR-HS ( $\bm{\mu}=0$ )	$+706.9$	$+621.9$
	AR-HS (reference)	$-7.4$	$+15.5$
	AR-HSM (ours)	$\mathbf{-74.3}$	$\mathbf{-67.3}$

Table 2: BD-rate savings measured with an input resolution of

512\times 512

for

\mathcal{D}^{\mathrm{val}}_{\mathrm{ADE20K}}

and

2048\times 1024

for

\mathcal{D}^{\mathrm{val}}_{\mathrm{CS}}

for the proposed HSM and AR-HSM vs. the state-of-the-art ( HS ) baseline [ 37 , 38 ] , the ablated variant AR-HS (

\bm{\mu}=0

), and the reference method ( AR-HS ). BD-rate is reported relative to the HS baseline. In each table segment, best results bold, second-best underlined.

Panel of Figure 11 — Figure 11: BD-rate savings of our proposals HSM and AR-HSM in comparison to the HS baseline and the AR-HSM reference, in dependence on total GFLOPs (top two plots) and $\#$ params (lower two plots), measured with an input resolution of $512\times 512$ for $\mathcal{D}^{\mathrm{val}}_{\mathrm{ADE20K}}$ and $2048\times 1024$ for $\mathcal{D}^{\mathrm{val}}_{\mathrm{CS}}$ .

Explore similar work

On the Rate-Distortion-Complexity Tradeoff for Semantic Communication
Feb 16, 2026Jingxuan Chai, Yong Xiao, Guangming ShiSemantic Communication Rate-Distortion Theory
the School of Artificial Intelligence at the Xidian University, Shaanxi 710071, China · the School of Electronic Information and Communications at the Huazhong University of Science and Technology, Wuhan 430074, China · the Pengcheng Laboratory, Shenzhen, Guangdong 518055, China +2
Dual-Constrained Diffusion Image Compression for Operational Rate-Distortion-Perception Optimization
Jun 11, 2026Sanxin Jiang, Jiro Katto, Heming SunLearned Image Compression Rate-Distortion Optimization
Department of Information Engineering Shanghai University of Electric Power Shanghai, China · Department of Computer Science and Communication Engineering Waseda University Tokyo, Japan · Faculty of Engineering Institute of Science Tokyo Tokyo, Japan
Semantic-Aware Joint Source-Channel Optimization for Encoder-Agnostic Digital Video Communication
Sep 30, 2026Xiangben Zhu, Caili Guo, Yang Yang +2Semantic Communication Rate-Distortion Optimization
Beijing Key Laboratory of Network System Architecture and Convergence, School of Information and Communication Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China · China Mobile (Suzhou) Software Technology Company Limited, Suzhou 215011, China · Department of Engineering, King’s College London, London WC2R 2LS, U.K.

Feb 16, 2026J/K move · Enter open · S save

On the Rate-Distortion-Complexity Tradeoff for Semantic Communication

Jingxuan Chai, Yong Xiao, Guangming Shi

the School of Artificial Intelligence at the Xidian University, Shaanxi 710071, China · the School of Electronic Information and Communications at the Huazhong University of Science and Technology, Wuhan 430074, China · the Pengcheng Laboratory, Shenzhen, Guangdong 518055, China+2

Semantic Communication Rate-Distortion Theory Information-Theoretic Lower Bounds

Open paper AskPDF arXiv

Semantic communication is a novel communication paradigm that focuses on conveying the user's intended meaning rather than the bit-wise transmission of source signals. One of the key challenges is to effectively represent and extract the semantic meaning of any given source signals. While deep learning (DL)-based solutions have shown promising results in extracting implicit semantic information from a wide range of sources, existing work often overlooks the high computational complexity inherent in both model training and inference for the DL-based encoder and decoder. To bridge this gap, this paper proposes a rate-distortion-complexity (RDC) framework which extends the classical rate-distortion theory by incorporating the constraints on semantic distance, including both the traditional bit-wise distortion metric and statistical difference-based divergence metric, and complexity measure, adopted from the theory of minimum description length and information bottleneck. We derive the closed-form theoretical results of the minimum achievable rate under given constraints on semantic distance and complexity for both Gaussian and binary semantic sources. Our theoretical results show a fundamental three-way tradeoff among achievable rate, semantic distance, and model complexity. Extensive experiments on real-world image and video datasets validate this tradeoff and further demonstrate that our information-theoretic complexity measure effectively correlates with practical computational costs, guiding efficient system design in resource-constrained scenarios.

Abstract

Explore similar work

On the Rate-Distortion-Complexity Tradeoff for Semantic Communication

Dual-Constrained Diffusion Image Compression for Operational Rate-Distortion-Perception Optimization

Semantic-Aware Joint Source-Channel Optimization for Encoder-Agnostic Digital Video Communication