Detection is solved, delineation is not: what governs tooth segmentation on panoramic radiographs

Muhammad Rehan; Moaz Amjad; Syed Danial Ahmed; Mariam Adnan; Haider Ali

cs.CVSep 18, 2026

Detection is solved, delineation is not: what governs tooth segmentation on panoramic radiographs

Authors: Muhammad Rehan, Moaz Amjad, Syed Danial Ahmed, Mariam Adnan, Haider Ali

Organizations: AI Dentify, Pakistan

Abstract

Automatic tooth segmentation and FDI numbering on panoramic radiographs underpins computer-assisted dental diagnosis, yet which factors govern performance remains unclear. We assemble a corpus of 1,422 panoramic radiographs containing 42,142 expert-delineated tooth polygons across the 32-class FDI taxonomy, annotated by 30 dental practitioners and independently reviewed by two others, and use it to isolate input resolution, architecture and anatomical priors under a single evaluation protocol. First, resolution dominates: across a controlled 640/1024/1280 ablation, mask mAP50-95 rises 0.656 -> 0.710 -> 0.717 while mAP50 stays flat at ~0.982. Both gains are significant under a paired bootstrap over images (p < 0.001, p = 0.024); neither mAP50 change is distinguishable from zero. Added resolution buys boundary precision, not detection. Second, architecture is nearly irrelevant in-domain: a query-based transformer with 2.1x the parameters is statistically equivalent to a one-stage detector (95% CI [-0.0064, +0.0064]), only marginally better under domain shift, 5.5x slower on CPU and not executable under standard ONNX runtimes. Third, three targeted interventions fail: a LoRA-adapted self-supervised encoder underperforms, a promptable foundation segmenter degrades masks by 39%, and globally optimal anatomical label assignment yields +0.0007 despite correcting a constraint violated in 40% of out-of-domain predictions. Zero-shot transfer to an independent multi-centre cohort, verified overlap-free, costs 62% of mask mAP50-95 but only 18% of mAP50, reproducing the dissociation. Decomposing masks along the tooth axis localises the residual error to the apical third. Boundary precision is therefore the binding constraint, and effort is better directed at resolution and acquisition diversity than at architectural novelty.

Figures & tables

Figure 1: The annotation layer. (a) A complete dentition, coloured by FDI quadrant and numbered; fill colour encodes quadrant, a property of the taxonomy rather than arbitrary identity. (b) Polygon granularity for a tooth at the corpus mean vertex count; the bifurcated roots of the first molar are traced individually. (c) Per-class instance counts: all 32 classes are represented, with third molars the only markedly less frequent group.

Figure 2: Input resolution acts on boundary precision, not detection. mAP50 is flat to three decimal places across the range while mAP50-95 rises by

0.061

. Error bars are 95% bootstrap intervals over images; the mAP50 intervals overlap almost entirely.

Input	mAP50	mAP50-95	$\Delta$ mAP50-95
640	0.983	0.656	—
1024	0.982	0.710	$+0.054$
			$[+0.048,+0.060]$
1280	0.982	0.717	$+0.007$
			$[+0.002,+0.011]$

Table 1: Input resolution ablation, identical protocol.

\Delta

is the change in mask mAP50-95 from the preceding row, with a 95% paired bootstrap interval over images. Best epoch rises

29\rightarrow 40\rightarrow 81

across the three arms.

Model	Params	In-domain mAP50-95	Zero-shot mAP50-95	CPU (ms)	ONNX
YOLO11m-seg	22.4M	0.7166	0.2740	749	yes
Mask2Former swin-T	47.4M	0.7163	0.2823	4105	no
DINOv3-B + LoRA	93.0M	0.6470	0.2309	—	—

Table 2: Architectures under an identical evaluation protocol. The in-domain difference between the top two is

0.0003

(95% CI

[-0.0064,+0.0064]

): not merely non-significant, but bounded inside

\pm 0.007

mAP.

Figure 3: Domain shift reproduces the dissociation from a second direction. Zero-shot transfer to DENTEX costs 62% of mask mAP50-95 but only 18% of mAP50.

Figure 4: Per-class IoU against mean annotated tooth area (

r{=}0.59

). The smallest teeth (lower incisors 41, 31, 32) are weakest. Third molars (18, 28) are a documented exception: large, but frequently impacted or partially erupted.

Band	IoU	Share of union area
Crown third	0.905	39.5%
Middle third	0.906	35.2%
Apical third	0.776	25.4%

Table 3: Segmentation accuracy by position along the tooth axis, over 4,207 detected teeth. Error is concentrated in the apical third.

Figure 5: Segmentation error is concentrated at the root apex. (a) Two teeth with the annotation, the prediction, and their disagreement; the apical third is tinted. Panels are selected automatically as the instances whose crown and apical IoU lie closest to the corpus means, so they are typical rather than extreme. (b) IoU against position along the tooth axis, pooled over 4,208 teeth. Both tips fall away—thin end slices are unforgiving—but only the apical side keeps falling, reaching $0.55$ in the final slice against $0.79$ at the occlusal tip. (c) Every FDI class lies below the diagonal, so the effect is not driven by a subset of classes.

Intervention	In-domain	Zero-shot
SAM2 refinement	$-0.278$	$-0.054$
	$[-0.289,-0.262]$	$[-0.057,-0.050]$
Hungarian assignment	$+0.0001$	$+0.0007$
	$[-0.0000,+0.0003]$	$[+0.0001,+0.0011]$

Table 4: Targeted interventions, change in mask mAP50-95 with 95% paired bootstrap intervals. Hungarian assignment is statistically detectable out of domain and practically negligible.

Figure 6: Why promptable refinement fails on radiographs. Panels are selected automatically as the teeth with the largest area reduction under refinement, one per class. SAM2’s region (orange) covers the crown and stops at the cervical margin, while ground truth (black) and the detector’s mask (blue, dashed) continue along the root. Retained area is 58–67% of the detector’s mask.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

		Corpus		In-domain (val)			Zero-shot (DENTEX)
FDI	Tooth	$n$	Area	$n$	IoU	AP	$n$	IoU	AP	$\Delta$ AP
18	UR 3rd molar	829	4.0	80	0.782	0.681	341	0.484	0.224	-0.457
17	UR 2nd molar	1367	4.9	136	0.858	0.751	576	0.532	0.256	-0.495
16	UR 1st molar	1347	5.4	139	0.860	0.744	563	0.511	0.258	-0.486
15	UR 2nd premolar	1344	3.6	139	0.840	0.737	563	0.470	0.218	-0.519
14	UR 1st premolar	1279	4.0	125	0.814	0.690	586	0.446	0.164	-0.526

Appendix

Table 5: Per-class results for the 1280 px YOLO11m-seg model. Prevalence and mean annotated area are computed over all 1,422 annotated images; accuracy columns use the common evaluation harness.

\Delta

is the zero-shot minus in-domain change in AP50-95. Classes are ordered by FDI quadrant.

Explore similar work

Apr 17, 2026

Dental Panoramic Radiograph Analysis Using YOLO26 From Tooth Detection to Disease Diagnosis

Panoramic radiography is a fundamental diagnostic tool in dentistry, offering a comprehensive view of the entire dentition with minimal radiation exposure. However, manual interpretation is time-consuming and prone to errors, especially in high-volume clinical settings. This creates a pressing need for efficient automated solutions. This study presents the first application of YOLOv26 for automated tooth detection, FDI-based numbering, and dental disease segmentation in panoramic radiographs. The DENTEX dataset was preprocessed using Roboflow for format conversion and augmentation, yielding 1,082 images for tooth enumeration and 1,040 images for disease segmentation across four pathology classes. Five YOLOv26-seg variants were trained on Google Colab using transfer learning at a resolution of 800x800. Results demonstrate that the YOLOv26m-seg model achieved the best performance for tooth enumeration, with a precision of 0.976, recall of 0.970, and box mAP50 of 0.976. It outperformed the YOLOv8x baseline by 4.9% in precision and 3.3% in mAP50, while also enabling high-quality mask-level segmentation (mask mAP50 = 0.970). For disease segmentation, the YOLOv26l-seg model attained a box mAP50 of 0.591 and a mask mAP50 of 0.547. Impacted teeth showed the highest per-class average precision (0.943), indicating that visual distinctiveness influences detection performance more than annotation quantity. Overall, these findings demonstrate that YOLOv26-based models offer a robust and accurate framework for automated dental image analysis, with strong potential to enhance diagnostic efficiency and consistency in clinical practice.

Khawaja Azfar Asif, Rafaqat Alam Khan

Department of Software Engineering, Lahore Garrison University, DHA Phase 6, Lahore, Pakistan

Medical Imaging Medical Image Segmentation Object Detection

Ask PDF

Feb 25, 2026

Innovative Tooth Segmentation Using Hierarchical Features and Bidirectional Sequence Modeling

Tooth image segmentation is a cornerstone of dental digitization. However, traditional image encoders relying on fixed-resolution feature maps often lead to discontinuous segmentation and poor discrimination between target regions and background, due to insufficient modeling of environmental and global context. Moreover, transformer-based self-attention introduces substantial computational overhead because of its quadratic complexity (O(n^2)), making it inefficient for high-resolution dental images. To address these challenges, we introduce a three-stage encoder with hierarchical feature representation to capture scale-adaptive information in dental images. By jointly leveraging low-level details and high-level semantics through cross-scale feature fusion, the model effectively preserves fine structural information while maintaining strong contextual awareness. Furthermore, a bidirectional sequence modeling strategy is incorporated to enhance global spatial context understanding without incurring high computational cost. We validate our method on two dental datasets, with experimental results demonstrating its superiority over existing approaches. On the OralVision dataset, our model achieves a 1.1% improvement in mean intersection over union (mIoU).

Xinxin Zhao, Jian Jiang, Yan Tian +5

School of Computer Science and Technology, Zhejiang Gongshang University, 310018, Hangzhou, China · School of Computer and Cyber Sciences, Communication University of China, 100024, Beijing, China · Center of Big Data, China Digital Culture Group Co., Ltd, 100176, Beijing, China +6

Medical Image Segmentation

Ask PDF

Aug 6, 2026

TLNM: Externally Validated Tooth Detection, Numbering and Segmentation from Smartphone Photographs Using Mask R-CNN

Oral health issues affect billions globally, but the cost and limited access to professional dental care hinder preventive oral healthcare. Research relies on clinical-grade radiographs or intraoral camera images, unavailable for public self-screening. This study introduces a tooth localisation and numbering model for smartphone photographs. We developed a customised Mask Region-based Convolutional Neural Network (Mask R-CNN) pipeline trained on 1,272 annotated smartphone images. To address variability in patient-generated health data, the pipeline incorporates two domain-informed mechanisms: a masked gray-world white-balancing algorithm to mitigate artificial colour casts and an anatomically constrained detection layer to enforce structural validity and suppress false positives. Evaluation comprised four stages: internal held-out testing, independent external testing, a descriptive ablation study, and fold-based training stability analysis using the same internal test set. On the internal test set, the model achieved an instance-mask AP@50 of 0.818, class-aware PQ of 0.780, and operational F1 of 0.884. Training stability showed limited between-model variation: across ten runs, instance-mask AP@50 had a standard deviation of 0.009. On the external dataset, the model achieved an instance-mask AP@50 of 0.901, class-aware PQ of 0.832, and operational F1 of 0.928 despite differences in population, sensors, and acquisition protocols. The inference pipeline is available as an open-source, containerised API. These results demonstrate that consumer-grade smartphone imagery can support automated tooth-level anatomical mapping, offering a scalable, potentially low-cost foundation for remote screening and tele-dentistry in resource-constrained environments.

Arash Nedaei, Henna Tiensuu, Elina Väyrynen +2

Biomimetics and Intelligent Systems Group, Faculty of Information Technology and Electrical Engineering, University of Oulu, Oulu, Finland · Research Unit of Population Health, Faculty of Medicine, University of Oulu, Oulu, Finland

Medical Imaging Instance Segmentation Medical Image Segmentation

Ask PDF