TwinMixing: A Shuffle-Aware Feature Interaction Model for Multi-Task Segmentation

Minh-Khoi Do; Huy Che; Dinh-Duy Phan; Duc-Khai Lam; Duc-Lung Vu

cs.CVMar 30, 2026

TwinMixing: A Shuffle-Aware Feature Interaction Model for Multi-Task Segmentation

Authors: Minh-Khoi Do, Huy Che, Dinh-Duy Phan, Duc-Khai Lam, Duc-Lung Vu

Organizations: University of Information Technology, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam

Abstract

Accurate and efficient perception is essential for autonomous driving, where segmentation tasks such as drivable-area and lane segmentation provide critical cues for motion planning and control. However, achieving high segmentation accuracy while maintaining real-time performance on low-cost hardware remains a challenging problem. To address this issue, we introduce TwinMixing, a lightweight multi-task segmentation model designed explicitly for drivable-area and lane segmentation. The proposed network features a shared encoder and task-specific decoders, enabling both feature sharing and task specialization. Within the encoder, we propose an Efficient Pyramid Mixing (EPM) module that enhances multi-scale feature extraction through a combination of grouped convolutions, depthwise dilated convolutions and channel shuffle operations, effectively expanding the receptive field while minimizing computational cost. Each decoder adopts a Dual-Branch Upsampling (DBU) Block composed of a learnable transposed convolution-based Fine detailed branch and a parameter-free bilinear interpolation-based Coarse grained branch, achieving detailed yet spatially consistent feature reconstruction. Extensive experiments on the BDD100K dataset validate the effectiveness of TwinMixing across three configurations - tiny, base, and large. Among them, the base configuration achieves the best trade-off between accuracy and computational efficiency, reaching 92.0% mIoU for drivable-area segmentation and 32.3% IoU for lane segmentation with only 0.43M parameters and 3.95 GFLOPs. Moreover, TwinMixing consistently outperforms existing segmentation models on the same tasks, as illustrated in Fig. 1. Thanks to its compact and modular design, TwinMixing demonstrates strong potential for real-time deployment in autonomous driving and embedded perception systems. The source code: https://github.com/Jun0se7en/TwinMixing.

Figures & tables

Figure 1: The horizontal axis represents FLOPs, the vertical axis denotes mIoU, and the circle radius corresponds to IoU.

Figure 2: Visual comparison of semantic scene understanding versus drivable area and lane segmentation, highlighting the focus on safety-critical and navigable regions for autonomous driving.

Figure 3: The architecture of TwinMixing. The model consists of a shared encoder and two task-specific decoders. The encoder integrates the proposed Efficient Pyramid Mixing (EPM) modules to enhance multi-scale feature extraction and contextual representation. Each decoder adopts a Dual Branch Upsampling Block (DBU) composed of Fine detailed branch and Coarse grained branch . The two decoders independently generate segmentation masks for lane lines and drivable areas, respectively.

Figure 4: Overview of the encoder in TwinMixing. The encoder extracts hierarchical multi-scale features from the input image through a combination of standard convolutional layers, Efficient Pyramid Mixing (EPM) modules, and Partial Class Activation Attention (PCAA) [ 29 ] , producing the shared representation $\mathcal{F}_{e}$ for subsequent decoding.

Figure 5: Illustration of the proposed Efficient Pyramid Mixing (EPM) module. The design is inspired by the ESP [ 22 ] , where the EPM performs a reduction step using an EPM Unit with a kernel 1

\times

1 before splitting features into multiple parallel branches. Each branch transforms the reduced feature through an EPM Unit with a different dilation rate to capture multi-scale spatial information. The outputs of all branches are then merged through the Hierarchical Feature Fusion (HFF) mechanism [ 22 ] . In the Stride EPM variant, the reduction step employs an EPM Unit with a kernel 1

\times

1 and a stride of 2 to achieve downsampling.

Panel of Figure 6 — Figure 6: Evolution of transformation designs in approaches derived from the Efficient Spatial Pyramid (ESP) framework [ 22 ] . (a) Dilated Convolution (D-Conv) in ESPNet [ 22 ] ; (b) Depth-wise Dilated Separable Convolution (DDS-Conv) in [ 14 , 30 ] ; (c) EPM Unit proposed; (d) Stride EPM Unit variant for downsampling.

Figure 7: Architecture of the proposed model with two parallel decoders sharing the same structural design for different segmentation tasks.

Panel of Figure 8 — Figure 8: Illustration of the proposed Dual Branch Upsampling Block (DBU). It consists of two complementary upsampling paths: the Fine detailed branch and the Coarse grained branch .

Config	FPS w/ batch size $\uparrow$			Params	FLOPs batch=1
Config	1	4	16	Params	FLOPs batch=1
Tiny	83 ${}_{\pm{\text{0.89}}}$	335 ${}_{\pm{\text{3.37}}}$	1302 ${}_{\pm{\text{12.71}}}$	0.10M	1.08G
Base	67 ${}_{\pm{\text{0.47}}}$	255 ${}_{\pm{\text{2.68}}}$	724 ${}_{\pm{\text{0.94}}}$	0.43M	3.95G
Large	56 ${}_{\pm{\text{0.51}}}$	218 ${}_{\pm{\text{2.42}}}$	301 ${}_{\pm{\text{0.64}}}$	1.50M	14.25G

Table 1: Throughput complexity comparison among configurations at various batch sizes

	Parameters	FLOPs	Training times
TwinLiteNet	0.44M	3.9G	567.1s
TwinLiteNet ${}^{+}_{\text{Medium}}$	0.48M	4.63G	799.8s
TwinMixing ${}_{\text{base}}$	0.43M	3.95G	1070.2s

Table 2: Comparison of per-epoch training time across TwinLiteNet, TwinLiteNet + , and TwinMixing.

Model	Drivable area segmentation	Lane segmentation		FLOPs	Parameters
Model	mIoU (%)	Acc (%)	IoU (%)	FLOPs	Parameters
DeepLabV3+ [ 17 ]	90.9	–	29.8	30.7G	15.4M
SegFormer [ 18 ]	92.3 ${}^{{\color[rgb]{1,0,0}\textbf{4}}}$	–	31.7	12.1G	7.2M
R-CNNP [ 37 ]	90.2	–	24.0	–	–
YOLOP [ 37 ]	91.6	–	26.5	8.11G	5.53M
YOLOv8 (multi-seg) [ 38 ]	84.2	81.7 ${}^{{\color[rgb]{1,0,0}\textbf{3}}}$	24.3	–	–

Table 3: Quantitative comparison of models on the drivable-area and lane-segmentation tasks. Results are reported as mIoU (%) for drivable area segmentation, and as Acc (%) and IoU (%) for lane segmentation, together with model complexity measured by FLOPs and parameter count.

Figure 9: Qualitative comparison of segmentation results under normal driving conditions.

Figure 10: Qualitative comparison of segmentation results under challenging driving conditions.

Figure 11: Qualitative visualization of cross-dataset generalization.

Method	Drivable area segmentation	Lane segmentation		Parameters	FLOPs
Method	mIoU (%)	Acc (%)	IoU (%)	Parameters	FLOPs
Single-task	92.3	✘	✘	0.41M	3.50G
Single-task	✘	81.1	33.5	0.41M	3.50G
Multi-task	92.4 ${}_{{\color[rgb]{0,0,1}\uparrow\text{0.1}}}$	80.7 ${}_{{\color[rgb]{1,0,0}\downarrow\text{0.4}}}$	33.2 ${}_{{\color[rgb]{1,0,0}\downarrow\text{0.3}}}$	0.43M ${}_{{\color[rgb]{1,0,0}\uparrow\text{0.02M}}}$	3.95G ${}_{{\color[rgb]{1,0,0}\uparrow\text{0.45G}}}$

Table 4: Comparative evaluation of the TwinMixing model under multi-task and single-task learning settings, reporting segmentation accuracy and computational efficiency metrics.

Environmental conditions	Drivable area segmentation	Lane segmentation
Environmental conditions	mIoU (%)	IoU (%)
Daytime	93.2	35.1
Night	92.5	32.9
Snowy	91.6	31.6
Rainy	90.2	32.0
Foggy	92.4	29.6

Table 5: Performance across various environmental conditions on both task drivable area segmentation and lane segmentation

Methods	Drivable area segmentation	Lane segmentation
Methods	mIoU (%)	Acc (%)	IoU (%)
DBU	92.4	80.7	33.2
w/o fine detailed	92.2 ${}_{{\color[rgb]{1,0,0}\downarrow\text{0.2}}}$	80.3 ${}_{{\color[rgb]{1,0,0}\downarrow\text{0.4}}}$	32.9 ${}_{{\color[rgb]{1,0,0}\downarrow\text{0.3}}}$
w/o coarse grained	92.3 ${}_{{\color[rgb]{1,0,0}\downarrow\text{0.1}}}$	80.3 ${}_{{\color[rgb]{1,0,0}\downarrow\text{0.4}}}$	33.0 ${}_{{\color[rgb]{1,0,0}\downarrow\text{0.2}}}$

Table 6: Ablation of the Dual Branch Upsampling (DBU) module. Removing either branch degrades performance, confirming their complementary effect on segmentation accuracy.

Dilated rates	Drivable area segmentation	Lane segmentation
Dilated rates	mIoU (%)	Acc (%)	IoU (%)
$\bigstar$	90.9	76.4	29.4
1	89.6	75.4	28.2
4	90.2	74.8	27.7
16	89.8	74.8	27.7

Table 7: Sensitivity analysis of dilation-rate configurations in the EPM module.

Groups	Parameters	FLOPs	Drivable area	Lane
Groups	Parameters	FLOPs	mIoU (%)	IoU (%)
$\bigstar$	98.2K	1,084G	90.9	29.4
1	115.0K	1.222G	91.3	30.4
2	104.5K	1.118G	91.2	29.5
4	100.7K	1.094G	90.8	29.3
8	99.2K	1.088G	90.9	29.3

Table 8: Sensitivity analysis of group-size configurations in the EPM module.

Config	Performance
Config	FP32	FP16	INT8 (QAT)
Tiny	91.1 / 29.8	91.1 / 29.8	90.3 / 29.2
Base	92.4 / 33.2	92.4 / 33.2	92.2 / 32.7
Large	92.8 / 34.2	92.8 / 34.2	92.6 / 33.9

Table 9: Quantization results of TwinMixing across different configurations, including FP32, FP16, and INT8. Results are presented in the format mIoU (for drivable area segmentation) / IoU (for lane segmentation).

Figure 12: Visualization of challenging failure cases for TwinMixing ${}_{\text{large}}$ .

Device	AGX Orin	Xavier	Orin Nano	TX2
Latency	21.96 ${}_{\pm{\text{0.23}}}$	26.78 ${}_{\pm{\text{0.38}}}$	27.53 ${}_{\pm{\text{0.08}}}$	60.81 ${}_{\pm{\text{0.13}}}$

Table 10: Inference latency (in milliseconds) of TwinMixing

{}_{\text{tiny}}

on various NVIDIA Jetson devices, reported as the mean and standard deviation over 500 independent runs.

Semantic Segmentation

Semantic segmentation assigns a class label to every image pixel, enabling detailed scene understanding across domains and sensors.

See all 414 papers

May 20, 2026

LiteViLNet: Lightweight Vision-LiDAR Fusion Network for Efficient Road Segmentation

Road segmentation is a fundamental perception task for autonomous driving and mobile robotics, where both appearance and geometric cues must be processed under edge-computing constraints. Existing multi-modal approaches often improve accuracy with large encoders or expensive global interaction, which limits their use on embedded platforms. We present LiteViLNet, a lightweight RGB-geometry fusion network that combines a MobileNetV3 RGB encoder with a 0.12M-parameter depth-wise-separable geometry encoder. A multi-scale feature fusion module performs modality-specific enhancement, global-query cross-modal interaction, and adaptive gating, while a depth-wise large-kernel bridge enlarges the contextual support of the deepest representation with low overhead. The resulting U-Net-style decoder uses deep supervision only during training. On the KITTI Road benchmark, the 14.04M-parameter full model obtains

97.23\pm0.15\%

MaxF. On the held-out ORFD test set under the released OFF-Net evaluation protocol, the full model achieves

96.74\pm0.09\%

F-score and

93.68\pm0.18\%

IoU. On a Jetson Orin NX, model-only PyTorch FP16 inference reaches

22.18\pm0.21

FPS; a separate TensorRT FP16 measurement reaches

68.73\pm0.06

FPS on the Jetson. Camera-depth adaptations and perception-and-control demonstrations on three heterogeneous robot platforms further illustrate the portability of the dual-stream design.

Daojie Peng, Bingtao Wang, Fulong Ma +2

The Hong Kong University of Science and Technology (Guangzhou) · The Shandong University

Edge Inference Camera-LiDAR Fusion Autonomous Driving Autonomous Driving Perception

Ask PDF

Oct 30, 2025

AD-SAM: Adapting the Segment Anything Model for Semantic Segmentation in Autonomous Driving

This paper presents the Autonomous Driving Segment Anything Model (AD-SAM), a foundation-model adaptation framework for semantic segmentation in autonomous driving. AD-SAM combines a frozen Segment Anything Model (SAM) Vision Transformer (ViT-H) encoder with a trainable ResNet-50 encoder to integrate general visual representations with multi-scale, domain-specific spatial features. Features from the two encoders are integrated through deformable convolution and channel attention, followed by a multi-stage deformable decoder for semantic prediction. Training employs a hybrid objective combining Focal, Dice, Lovász-Softmax, and Surface losses. Experiments on Cityscapes and Berkeley DeepDrive 100K (BDD100K) show that AD-SAM outperforms SAM, Generalized SAM (G-SAM), and DeepLabV3 under a controlled training protocol. AD-SAM achieves 76.27% mIoU on Cityscapes and 64.74% on BDD100K, exceeding DeepLabV3 by 3.45 and 5.00 percentage points, respectively, with larger gains over the SAM-based baselines. Sample-size experiments reveal dataset-dependent behavior. In particular, AD-SAM performs strongly across training sizes on Cityscapes, while its advantage on the more heterogeneous BDD100K becomes pronounced with increased training data. When trained on Cityscapes and directly evaluated on BDD100K, AD-SAM achieves the highest cross-dataset retention (84.88%) among the evaluated models. AD-SAM also converges rapidly, while precomputed frozen SAM embeddings reduce training memory requirements. These findings demonstrate the potential of combining general foundation-model representations with domain-specific multi-scale features for accurate and robust autonomous-driving semantic segmentation.

Het Patel, Mario Camarena, Fatemeh Nazari +3

Department of Computer Science, The University of Texas Rio Grande Valley, Edinburg, TX 78539 USA · Department of Civil Engineering, The University of Texas Rio Grande Valley, Edinburg, TX 78539 USA · Department of Computer Science and Engineering, University of California Riverside, Riverside, CA 92521 USA

Vision Foundation Model Adaptation Autonomous Driving Perception Semantic Segmentation

Ask PDF

Oct 24, 2025

AURASeg: Attention-Guided Upsampling with Residual-Assisted Boundary Refinement for Drivable-Area Segmentation

Free-space segmentation is essential for autonomous robots to identify drivable regions and navigate safely across indoor, outdoor, and road-scene environments. However, conventional encoder-decoder models often recover coarse region masks while losing the fine spatial information needed to localize drivable-area boundaries accurately. We propose Attention-Guided Upsampling with Residual-Assisted Boundary Refinement (AURASeg), a segmentation framework designed to preserve region-level accuracy while improving boundary quality. Built on a ResNet-18 encoder, AURASeg introduces an Attention Progressive Upsampling Decoder (APUD) that progressively combines semantic context with high-resolution spatial detail, together with a Residual Boundary Refinement Module (RBRM) that explicitly refines contour-sensitive features before final prediction. We evaluate AURASeg across indoor simulation, ground-robot imagery, and road-driving benchmarks. The results show that our proposed model remains competitive with established segmentation models on region-level metrics while providing particularly strong boundary localization, including in comparison with boundary-focused methods. Detailed ablations further demonstrate the role of the proposed decoding and refinement modules.

Narendhiran Vijayakumar

Autonomous Driving Perception Boundary-Aware Segmentation Semantic Segmentation

Ask PDF

Abstract

Semantic Segmentation

Explore similar work

LiteViLNet: Lightweight Vision-LiDAR Fusion Network for Efficient Road Segmentation

AD-SAM: Adapting the Segment Anything Model for Semantic Segmentation in Autonomous Driving

AURASeg: Attention-Guided Upsampling with Residual-Assisted Boundary Refinement for Drivable-Area Segmentation