Innovative Tooth Segmentation Using Hierarchical Features and Bidirectional Sequence Modeling
Authors: Xinxin Zhao, Jian Jiang, Yan Tian, Liqin Wu, Zhaocheng Xu, Teddy Yang, Yunuo Zou, Xun Wang
Organizations: School of Computer Science and Technology, Zhejiang Gongshang University, 310018, Hangzhou, China · School of Computer and Cyber Sciences, Communication University of China, 100024, Beijing, China · Center of Big Data, China Digital Culture Group Co., Ltd, 100176, Beijing, China · Shining3D Tech Co., Ltd., 311258, Hangzhou, China · Zhejiang Key Laboratory of Big Data and Future E-Commerce Technology, 310018, Hangzhou, China · Department of Stomatology, Tongxiang Hospital of Traditional Chinese Medicine, 100024, Tongxiang, China · School of Mathematical and Computational Sciences, Massey University, 100024, Auckland, New Zealand · Oral and Maxillofacial Surgery at the Faculty of Dentistry,University of Hong Kong, 999077, Hong Kong, China · School of Art Design, Zhejiang Gongshang University, 310018, Hangzhou, China
Tooth image segmentation is a cornerstone of dental digitization. However, traditional image encoders relying on fixed-resolution feature maps often lead to discontinuous segmentation and poor discrimination between target regions and background, due to insufficient modeling of environmental and global context. Moreover, transformer-based self-attention introduces substantial computational overhead because of its quadratic complexity (O(n^2)), making it inefficient for high-resolution dental images. To address these challenges, we introduce a three-stage encoder with hierarchical feature representation to capture scale-adaptive information in dental images. By jointly leveraging low-level details and high-level semantics through cross-scale feature fusion, the model effectively preserves fine structural information while maintaining strong contextual awareness. Furthermore, a bidirectional sequence modeling strategy is incorporated to enhance global spatial context understanding without incurring high computational cost. We validate our method on two dental datasets, with experimental results demonstrating its superiority over existing approaches. On the OralVision dataset, our model achieves a 1.1% improvement in mean intersection over union (mIoU).
Figures & tables
Figure 1: For the dental segmentation dataset, we conducted a comprehensive comparison between our method and the advanced segmentation approach HQ-SAM in terms of architectural design. Across image processing tasks with varying resolutions, our method leverages optimized feature extraction strategies and efficient architectural design to maintain low latency while delivering high-quality segmentation results. The blue area represents the mask generated after image segmentation. The white dashed box visually highlights the differences between the segmentation masks generated by our method and those produced by HQ-SAM.
Figure 2: Overall structure of the proposed approach. The proposed approach adopts a classic encoder-decoder architecture. First, the input dental images undergo a three-stage downsampling process to capture image features at multiple scales. These extracted features are then combined with prompt vectors created by the prompt encoder and processed together by the mask decoder to generate segmentation masks that correspond to the original images. In BSB, the sigmoid linear unit (Silu) activation function combines the smoothness of the sigmoid with a linear component to enhance feature representation. The blue region indicates the mask generated following the image segmentation process.
Figure 3: Illustration of the comparison between the vanilla mamba block and the bidirectional sequence block. (a) Vanilla mamba blocks are scanned in a sequential order from start to finish. (b) The bidirectional sequence module scans in a forward and backward order. The green and purple dots represent patch blocks at different positions, while the arrows in different colors indicate various scanning orders.
Bidirectional Strategy
DSD(mIoU)
None
89.1 ± 0.4
Bidirectional SSM
90.7 ± 0.3
Bidirectional SSM + Conv1d
90.9 ± 0.3
Table 1: Ablation study on the bidirectional design. We highlighted our block configuration in blue.
Figure 4: In (a), we compare the FPS performance of our method with SAM, HQ-SAM, and EfficientSAM across different input image sizes. In (b), we present a quantitative evaluation of the impact of low-level detailed feature (LDF) aggregation on the model’s mIoU. In (c), we analyze the comparative mIoU performance of our method and SAM under Gaussian noise conditions (standard deviation of 25). In (d), we illustrate the trend in mIoU variations between our method and SAM under random rotation conditions (angle range from -30° to 30°).
Variant
mIoU
mBIoU
No Gate
90.8 ± 0.3
87.2 ± 0.3
Shared Gate
91.4 ± 0.2
87.9 ± 0.3
Dual Gate (Ours)
91.9 ± 0.2
88.7 ± 0.3
Table 2: Ablation study on the gating mechanism. Results are reported as mean ± standard deviation across 5 folds.
Approach
FPS ↑
Memory(MB) ↓
Flops(G) ↓
MedSAM [ 10 ]
28.5
3100
34.2
Adapter-SAM [ 37 ]
31.5
3050
33.2
SegFormer [ 38 ]
43.1
2620
22.6
Swin-Unet [ 39 ]
22.7
3450
47.8
Ours
52.3
1860
12.5
Table 3: Efficiency comparison of different methods in terms of FPS, GPU memory usage, and FLOPs on the DSD.
Module
FLOPs (G)
Percentage of Total
Theoretical Complexity
Encoder (Stage 1 & 2)
2.1
16.8%
O(N)
Encoder (Stage 3 – BSB)
7.8
62.4%
O(N)
Decoder (Fusion & Upsampling)
1.9
15.2%
O(NlogN)
Decoder (Mask Prediction)
0.7
5.6%
O(N3/2)
Total
12.5
100 %
∼O(N)
Table 4: Theoretical FLOPs breakdown of different modules (input resolution 640×480 ).
Resolution
Pixels (M)
FLOPs (G)
Runtime (ms)
320×240
0.08
3.2
2.1
640×480
0.31
12.5
19.1
1024×768
0.79
31.6
49.8
1536×1152
1.77
78.2
126.4
Table 5: Empirical scaling of FLOPs and runtime for our method at different input resolutions.
Figure 5: Comparison of aggregation of low-level detailed features (LDF). The results show that aggregating low-level detail features effectively mitigates noise from artifacts such as dental calculus and food residues, producing segmentation masks that are closer to the ground truth (GT). The blue regions represent the generated masks after segmentation, and the white dashed boxes highlight the differences in the segmentation results.
Figure 6: We conducted a comparative analysis with SAM under conditions of Gaussian noise and random rotation. The results indicate that on dental images subjected to Gaussian noise and random rotation, the segmentation masks generated by our method are more complete and refined. The blue regions represent the generated masks after segmentation, and the white dashed boxes highlight the differences in the segmentation results.
Approach
DSD
OralVision
mIoU
mBIoU
mIoU
mBIoU
SAM [ 5 ]
88.7 ± 0.4
85.3 ± 0.5
88.1 ± 0.3
85.0 ± 0.4
HQ-SAM [ 12 ]
91.2 ± 0.3
88.1 ± 0.4
90.3 ± 0.3
87.2 ± 0.3
EfficientSAM [ 7 ]
87.2 ± 0.5
84.4 ± 0.5
86.9 ± 0.4
83.8 ± 0.3
GroundedSAM [ 6 ]
88.6 ± 0.4
85.5 ± 0.3
87.6 ± 0.3
84.7 ± 0.3
MedSAM [ 10 ]
89.0 ± 0.3
86.0 ± 0.4
88.5 ± 0.3
85.8 ± 0.3
Table 6: Experimental results on the test sets of the DSD and OralVision datasets. All values are reported as mean ± standard deviation.
Class
SAM
HQ-SAM
Ours
Teeth
90.2 ± 0.3
92.5 ± 0.2
93.6 ± 0.3
Gums
86.3 ± 0.2
88.7 ± 0.3
89.8 ± 0.2
Lips
87.4 ± 0.3
89.9 ± 0.2
91.0 ± 0.2
Cheeks
87.0 ± 0.2
89.5 ± 0.3
90.6 ± 0.3
Tongue
89.5 ± 0.2
91.0 ± 0.3
92.0 ± 0.2
mIoU (overall)
88.1 ± 0.3
90.3 ± 0.3
91.4 ± 0.3
Table 7: Per-class IoU on OralVision.
Figure 7: The visualization results of SAM, HQ-SAM, and our method on the DSD. Compared to other methods, our approach generates more fine-grained segmentation masks. The blue area represents the mask generated after image segmentation. The white dashed box highlights the differences between the segmentation masks generated by our method and those produced by other methods.
Figure 8: Failure cases of our approach in DSD. The blue area represents the mask generated after image segmentation. The red dashed box highlights areas where incomplete segmentation occurred due to poor image brightness. The green dashed box indicates segmentation errors caused by the similar color between the gums and cheek tissue. The blue dashed box marks regions where the model failed to segment correctly due to saliva noise and misleading folded surfaces.
Institute of Software, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Department of Computer Science and Technology, Tsinghua University +2
School of Software Technology, Zhejiang University · School of Computer and Computing Science, Hangzhou City University · The Affiliated Stomatology Hospital, Zhejiang University School of Medicine