NativeSeg: Native Vision Language Segmentation of 3D Computed Tomography via Spatial Visual Tokens and Verified Local Zoom
Organizations: Peking University · Tsinghua University · Institute of Automation, Chinese Academy of Sciences · Nanyang Technological University ·
Abstract
Text prompted 3D computed tomography (CT) segmentation requires both full slice context and fine boundary precision across volumes where over of organ prompted slices lack the queried target. Existing medical vision language models (VLMs) are largely non native segmenters: they delegate pixel delineation outside the VLM by compressing reasoning into a 1D [SEG] vector for an external Segment Anything Model (SAM) decoder, quantizing masks into lossy discrete codebooks, or issuing box prompts to stateless external SAM tools without empty slice screening or spatial state memory. We introduce NativeSeg, a native vision language segmentation framework on Low Rank Adaptation (LoRA) adapted Qwen3.8-27B that performs dense mask decoding, volumetric triage, spatial error self diagnosis, and multiround local zoom directly over the VLM's 2D spatial visual tokens without external SAM decoders. NativeSeg decodes confidence maps from upper layer 2D spatial tokens fused with a five slice U-Net pathway, halts at zero zooms on of standalone evaluations via a native presence gate (<pres>), localizes uncertain boundaries via a 20 cell multiscale error gate (<err>), and accepts zoomed updates only when predicted error decreases. Trained solely on adult CT, NativeSeg improves Round DSC by and points over discrete codebook and 1D [SEG] readouts on the identical backbone, reaches adult and zero shot pediatric DSC, and safely refines 2D nnU-Net ( adult; pediatric), 3D SegVol (), and 3D VoxTell ().
Figures & tables
| Method | Spl | RKid | LKid | Gall | Liv | Sto | Pan | Mean DSC (%) | NSD (%) | HD95 (mm) | Miss / FP |
|---|---|---|---|---|---|---|---|---|---|---|---|
| (a) Standalone VLM segmentation (all models trained on the 1,081 adult CT scans of TotalSegmentator) | |||||||||||
| LISA † [ 23 ] | 93.40 | 86.03 | 86.04 | 70.84 | 91.67 | 73.96 | 59.79 | 80.25 | 76.37 | 21.24 | 1483 / 1134 |
| DenseMLLM † [ 25 ] | 94.22 | 87.37 | 86.87 | 72.60 | 92.38 | 78.90 | 63.18 | 82.22 | 79.47 | 19.26 | 1304 / 957 |
| F-LMM † [ 43 ] | 95.85 | 88.30 | 87.11 | 75.42 | 93.34 | 83.62 | 70.19 | 84.83 | 83.79 | 14.72 | 1195 / 879 |
| Single Slice Multiround VLM (Ours) | 96.13 | 88.62 | 87.47 | 78.40 | 93.43 | 85.34 | 71.11 | 85.78 | 86.25 | 8.93 | 1022 / 533 |
| NativeSeg Round 0 (Ours) | 96.52 | 89.63 | 87.19 | 75.21 | 93.51 | 85.77 | 74.22 | 86.01 | 85.76 | 13.13 | 915 / 801 |
| Method | Spl | RKid | LKid | Gall | Liv | Sto | Pan | Mean DSC (%) | NSD (%) | HD95 (mm) | Miss / FP |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Reference: External multi dataset 2D/3D foundation models trained on large scale extra corpora (up to 22K+ volumes; not TotalSeg only) | |||||||||||
| BiomedParse ∗ [ 49 ] | 36.20 | 18.77 | 22.86 | 8.74 | 35.62 | 14.47 | 11.66 | 21.19 | Not rep. | Not rep. | Not rep. |
| Text3DSAM ∗ [ 45 ] | 64.27 | 28.35 | 59.28 | 7.25 | 76.64 | 48.37 | 14.30 | 42.64 | Not rep. | Not rep. | Not rep. |
| 3D SegVol ‡ [ 10 ] | 88.95 | 89.00 | 92.46 | 65.22 | 95.75 | 86.13 | 76.89 | 84.91 | 86.01 | 10.05 | 720 / 167 |
| 3D SAT ∗ [ 51 ] | 93.10 | 94.26 | 96.12 | 80.25 | 95.65 | 77.87 | 78.81 | 88.01 | Not rep. | Not rep. | Not rep. |
| BiomedParseV2 ∗ [ 50 ] | 92.99 | 94.36 | 96.22 | 84.63 | 96.28 | 78.18 | 78.68 | 88.76 | Not rep. | Not rep. | Not rep. |
| Variant (Qwen3.8-27B) | DSC | NSD | HD95 | Miss /FP |
|---|---|---|---|---|
| (a) Round 0 Mask Readout: Non Native vs. Native Spatial | ||||
| Discrete Codebook (MedSIGHT † [ 2 ] ) | 68.91 | 60.84 | 48.9 | 56 /1044 |
| Single [SEG] Broadcast (LISA † [ 23 ] ) | 81.68 | 76.27 | 21.4 | 136/140 |
| Spatial Tokens Only (DenseMLLM † [ 25 ] ) | 83.73 | 79.54 | 19.1 | 153/119 |
| Spatial Tokens + Skip (F-LMM † [ 43 ] ) | 87.73 | 84.64 | 16.3 | 134/116 |
| Spatial Tokens + Five Slice U-Net | 88.72 | 86.39 | 17.0 | 113/ 97 |
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Method | Spl | RKid | LKid | Gall | Liv | Sto | Pan | Mean DSC (%) | NSD (%) | HD95 (mm) | Miss / FP |
|---|---|---|---|---|---|---|---|---|---|---|---|
| (a) Standalone VLM segmentation on the identical Qwen3.8-27B backbone (Validation Split, 57 patients) | |||||||||||
| LISA † [ 23 ] | 89.46 | 87.88 | 83.79 | 63.75 | 92.65 | 76.52 | 56.60 | 78.66 | 74.38 | 29.05 | 1057 / 898 |
| DenseMLLM † [ 25 ] | 89.69 | 87.64 | 86.14 | 64.31 | 93.40 | 80.87 | 62.08 | 80.59 | 77.61 | 25.53 | 1006 / 881 |
| F-LMM † [ 43 ] | 92.90 | 90.68 | 87.95 | 67.44 | 95.19 | 83.96 | 69.74 | 83.98 | 82.83 | 19.45 | 940 / 713 |
| NativeSeg Round 0 (Ours) | 93.31 | 92.47 | 90.63 | 69.96 | 95.24 | 85.24 | 72.73 | 85.65 | 84.79 | 18.95 | 712 / 728 |
| NativeSeg (Ours) | 94.37 | 93.25 | 91.45 | 73.44 | 95.45 | 86.21 | 73.46 | 86.80 | 87.67 | 10.15 | 576 / 475 |
3D Medical Image Segmentation
3D Medical Image Segmentation identifies organs, lesions, and fine structures in volumetric scans for measurement and clinical planning.
See all 236 papers