IOSVLM: A 3D Vision-Language Model for Unified Dental Diagnosis from Intraoral Scans
Organizations: ZJU-UIUC Institute, Zhejiang University, Haining, 314400, China · Stomatology Hospital, School of Stomatology, Zhejiang University School of Medicine, Hangzhou, 310058, China · Angelalign Research Institute, Angel Align Inc., Shanghai, 200011, China
Abstract
3D intraoral scans (IOS) are increasingly adopted in routine dentistry due to abundant geometric evidence, and unified multi-disease diagnosis is desirable for clinical documentation and communication. While recent works introduce dental vision-language models (VLMs) to enable unified diagnosis and report generation on 2D images or multi-view images rendered from IOS, they do not fully leverage native 3D geometry. Such work is necessary and also challenging, due to: (i) heterogeneous scan forms and the complex IOS topology, (ii) multi-disease co-occurrence with class imbalance and fine-grained morphological ambiguity, (iii) limited paired 3D IOS-text data. Thus, we present IOSVLM, an end-to-end 3D VLM that represents scans as point clouds and follows a 3D encoder-projector-LLM design for unified diagnosis and generative visual question-answering (VQA), together with IOSVQA, a large-scale multi-source IOS diagnosis VQA dataset comprising 19,002 cases and 249,055 VQA pairs over 23 oral diseases and heterogeneous scan types. To address the distribution gap between color-free IOS data and color-dependent 3D pre-training, we propose a geometry-to-chromatic proxy that stabilizes fine-grained geometric perception and cross-modal alignment. A two-stage curriculum training strategy further enhances robustness. IOSVLM consistently outperforms strong baselines, achieving gains of at least +9.58% macro accuracy and +1.46% macro F1, indicating the effectiveness of direct 3D geometry modeling for IOS-based diagnosis.
Figures & tables
| Dataset | Privacy | Source | IOS types | #SD | #OD | #cases | #VQA |
|---|---|---|---|---|---|---|---|
| MaloccIOS | Private | China | S+O | 2 | 11 | 14,630 | 208,370 |
| DiseaseIOS | Private | China | S | 8 | — | 4,172 | 33,376 |
| Bits2Bites | Public | Italy | O | — | 5 | 200 | 7,309 |
| IOSVQA | — | Mixed | S+O | 8 | 15 | 19,002 | 249,055 |
| Models | Input | Acc | F1 | Preci | Recall | PR |
|---|---|---|---|---|---|---|
| GPT-5 [ 17 ] | Multi-View Images | 62.26 | 44.73 | 49.41 | 49.20 | 99.84 |
| Gemini 3 Pro [ 7 ] | Multi-View Images | 67.65 | 48.93 | 56.78 | 51.85 | 99.71 |
| Qwen3VL-8B [ 1 ] | Multi-View Images | 57.88 | 36.46 | 45.21 | 46.58 | 100 |
| InternVL3.5-8B [ 18 ] | Multi-View Images | 56.23 | 34.77 | 40.36 | 45.47 | 100 |
| InternVL3.5-14B [ 18 ] | Multi-View Images | 35.68 | 24.61 | 29.14 | 46.87 | 100 |
| MedGemma-1.5 [ 16 ] | Multi-View Images | 52.21 | 31.31 | 36.58 | 45.25 | 100 |
| Ablation part | Method choice | Acc | F1 | Preci | Recall | PR |
|---|---|---|---|---|---|---|
| LLM | Qwen3-4B [ 21 ] | 71.10 | 39.22 | 35.55 | 45.15 | 100 |
| Qwen3-8B [ 21 ] | 70.73 | 38.84 | 36.42 | 45.26 | 100 | |
| Qwen3VL-8B | 67.02 | 43.10 | 48.94 | 46.70 | 95.08 | |
| GCP | ✗ | 67.02 | 43.10 | 48.94 | 46.70 | 95.08 |
| ✓ | 72.28 | 48.06 | 51.25 | 49.82 | 97.00 | |
| Training mode | Stage 1: V & P | 72.28 | 48.06 | 51.25 | 49.82 | 97.00 |
3D VLMs
Three-dimensional vision-language models connect language with spatially grounded visual data for 3D understanding, reasoning, and action.
See all 204 papers