AgenTeeth: A Model-Agnostic Framework for Suppressing Hallucination in Frozen Vision-Language Models on Dental X-Rays via Tool Evidence Injection

Ahmed Rafid; Fariya Ahmed; Rumman Adib; Mehedi Ahamed; Ajwad Abrar; Tareque Mohmud Chowdhury

cs.CVSep 15, 2026

AgenTeeth: A Model-Agnostic Framework for Suppressing Hallucination in Frozen Vision-Language Models on Dental X-Rays via Tool Evidence Injection

Authors: Ahmed Rafid, Fariya Ahmed, Rumman Adib, Mehedi Ahamed, Ajwad Abrar, Tareque Mohmud Chowdhury

Organizations: Islamic University of Technology, Dhaka, Bangladesh · Southeast University, Dhaka, Bangladesh

Abstract

Vision-language models (VLMs) remain largely unreliable on panoramic dental radiographs and can rely on learned anatomical priors rather than evidence in the image. This is particularly problematic for tooth localization and spatial reasoning, and fine-tuned dental VLMs can retain the same spatial biases. We present AgenTeeth, a model-agnostic, tool-augmented framework that grounds frozen VLMs using seven specialized dental vision experts. A question-aware orchestrator selects the relevant tools, whose detections are mapped to FDI tooth numbers or anatomical regions and returned as structured findings together with annotated image overlays. A fresh synthesis call then answers the question using this evidence, without fine-tuning the underlying VLM. On MMOral-OPG-Bench, AgenTeeth improves four backbone VLMs by 12.9-23.0 percentage points over their baselines. Our strongest configuration reaches 65.66% on open-ended VQA, compared with 45.35% for OralGPT-Plus. The advantage also holds at matched scale: a frozen Qwen2.5-VL-7B-Instruct with AgenTeeth reaches 48.11%, exceeding OralGPT-Plus built on the same backbone after supervised fine-tuning and reinforcement learning for tool use. We release the framework, all seven expert models, and a dentist-annotated dataset for alveolar bone-loss detection in panoramic radiographs.

Figures & tables

Figure 1: AgenTeeth corrects a left–right spatial error. The baseline selects tooth #36 and invents supporting evidence, whereas AgenTeeth grounds its answer in an expert detection at tooth #46.

Tool	mAP@50	Prec.	Rec.	F1
Jaw structure	97.9	98.3	98.3	98.3
FDI numbering	92.4	91.0	90.1	90.5
Impacted tooth	91.0	74.7	83.8	79.0
Dental history	87.6	89.0	80.4	84.4
Periapical lesion	66.5	59.4	65.0	62.1
Bone loss	57.5	65.3	52.1	58.0

Table 1: Held-out performance of the domain experts, reported in percent.

Figure 2: The AgenTeeth framework. A frozen orchestrator VLM operates in two passes. First, it selects the relevant domain experts, with FDI numbering always enabled and all tools invoked when dispatch is uncertain; benchmark category labels are never used. Detector outputs are then grounded to FDI teeth by deterministic matching and converted into a coordinate-free evidence packet. A fresh, tool-less second pass answers from this packet together with the raw radiograph, selected annotated overlays, and any region crop, while prioritizing tool evidence over unsupported visual assumptions. The matched baseline uses the same pipeline without the seven domain experts.

Previously reported (open split only)
	Closed-ended VQA						Open-ended VQA
Model	Teeth	Patho	His	Jaw	Summ	Overall	Teeth	Patho	His	Jaw	Summ	Report	Overall
GPT-5 [ 5 ]	–	–	–	–	–	–	39.8	29.4	44.1	77.4	40.1	28.2	42.3
OralGPT-Plus-7B [ 5 ]	–	–	–	–	–	–	45.3	33.6	45.6	75.4	34.6	32.7	45.4
Previously reported closed-ended result
Ovis2-34B [ 8 ]	45.8	51.6	53.9	79.4	79.2	56.8	–	–	–	–	–	–	–

Table 2: MMOral-OPG-Bench results , in percent. Open-ended results are scored with the GPT-5-mini evaluation protocol used by OralGPT-Plus, while closed-ended results use exact-match accuracy and are judge-independent. Baseline rows use the identical AgenTeeth pipeline with the seven domain experts removed. Previously reported open-ended rows are from OralGPT-Plus; the strongest closed-ended MMOral result is included for comparison. Closed accuracy is capped at 93.5% by 32 unanswerable questions.

	GPT-4o judge			GPT-5-mini judge
Model	Base	Ours	$\Delta$	Base	Ours	$\Delta$
gemma4:31b	34.7	51.8	+17.1	35.7	56.0	+20.3
qwen3.5	45.1	60.6	+15.5	49.9	63.9	+14.0
mistral-large-3	38.3	59.0	+20.7	38.8	61.1	+22.4
minimax-3	35.9	55.9	+19.9	44.5	65.7	+21.2

Table 3: Open-ended MMOral-OPG-Bench Overall under two judges , in percent. Base denotes the matched detector-free baseline and Ours denotes AgenTeeth. AgenTeeth improves all four backbones under both GPT-4o and GPT-5-mini, showing that the performance gains are consistent across judge choice. Closed-ended results are judge-independent and are reported separately in Tab. 2 .

	Tool	Closed-ended VQA						Open-ended VQA
Config.	mAP@50	Teeth	Patho	His	Jaw	Summ	Overall	Teeth	Patho	His	Jaw	Summ	Report	Overall
AgenTeeth (full)	–	60.3	56.1	67.6	79.1	71.0	64.4	57.9	57.7	69.1	89.1	65.7	39.9	61.1
w/ weaker FDI	$92.4\!\rightarrow\!88.1$	58.3	54.1	64.8	79.1	64.5	62.9	49.8	54.0	60.3	89.1	63.6	35.8	56.0
difference	$-4.3$	–2.0	–2.0	–2.8	0.0	–6.5	–1.5	–8.0	–3.6	–8.8	0.0	–2.1	–4.1	–5.1
w/ weaker history	$87.6\!\rightarrow\!59.0$	58.3	56.1	60.6	77.5	71.0	62.3	53.7	56.3	56.2	88.6	64.0	36.5	57.8
difference	$-28.6$	–2.0	0.0	–7.0	–1.6	0.0	–2.1	–4.2	–1.4	–12.9	–0.5	–1.7	–3.4	–3.4

Table 4: Framework accuracy tracks detector accuracy. Replacing either the FDI or dental-history expert with a weaker version reduces downstream VQA performance while all other components remain fixed. Results use mistral-large-3 as the backbone.

Images	Jaw	Overall	His	Summ	Overall	Avg
	Closed		Open
1	78.3	64.6	64.4	59.4	58.5	61.5
4	79.1	64.4	69.1	65.7	61.1	62.8

Table 5: Effect of annotated overlays. Increasing synthesis input from one to four overlays improves open-ended performance by 2.6 points, with negligible change on the closed split. Results use mistral-large-3.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Source	Reference	Used by
Tufts Dental Database	[ 18 ]	FDI, bone loss, dental history
DUAL (Kaggle)	[ 25 ]	FDI
DENTEX 2023	[ 6 ]	FDI
Panoramic Dental X-ray	[ 1 ]	FDI, jaw structure
Annotated OPG (restorations)	[ 10 ]	Dental history
Panoramic radiographs with periapical lesions	[ 3 ]	Periapical

Appendix

Table 1: Source datasets and the detectors trained on them.

Detector	Images	Train	Valid	Test	Aug. train
FDI numbering	1,967	1,574	197	196	3,148
Bone loss	943	755	94	94	1,510
Dental history	2,456	1,935	261	260	3,870
Jaw structure	320	256	32	32	768
Impacted tooth	3,362	2,638	356	368	5,276
Periapical †	3,744	3,006	364	374	6,012

Appendix

Table 2: Per-detector dataset sizes. † Counts are tiles, not radiographs: periapical derives 3,744 tiles from 3,924 source radiographs, and caries derives 2,470 tiles from 1,277. Aug. train is the training split after augmentation.

Source	Images	Train	Valid	Test
Tufts	878	703	88	87
DENTEX 2023	621	489	70	62
DUAL (Kaggle)	470	381	40	49
Panoramic Dental X-ray	53	37	5	4

Appendix

Table 3: Per-source splits of the merged FDI training set.

Detector	Preprocessing	Augmentation	$\times N$
FDI numbering	auto-orient; fit within $768\times 512$	brightness $\pm 20\%$ , exposure $\pm 15\%$ , blur 1 px, rotate $\pm 3^{\circ}$	2
Bone loss	auto-orient; fit within $768\times 768$ ; histogram equalisation	brightness $\pm 15\%$ , exposure $\pm 10\%$ , rotate $\pm 7^{\circ}$ , horizontal flip	2
Dental history	auto-orient; stretch to $768\times 448$	brightness $\pm 30\%$ , exposure $\pm 15\%$ , blur 2 px, noise $1.02\%$ , cutout $3\times 10\%$ , horizontal flip	2
Jaw structure	auto-orient; fit within $576\times 576$	brightness $\pm 15\%$ , exposure $\pm 10\%$ , blur 1.5 px	3
Periapical	auto-orient; fit within $672\times 672$ ; histogram equalisation	brightness $\pm 15\%$ , exposure $\pm 10\%$ , blur 1 px, rotate $\pm 10^{\circ}$ , horizontal flip	2
Dental caries	auto-orient; fit within $768\times 768$	brightness $\pm 15\%$ , exposure $\pm 10\%$ , rotate $\pm 10^{\circ}$	2

Appendix

Table 4: Preprocessing and augmentation per detector.

\times N

is the number of augmented copies generated per training image.

Model	Teeth	Patho	His	Jaw	Summ	Report	Overall
gemma4:31b (baseline)	30.3	26.1	43.7	52.6	27.9	32.3	34.7
gemma4:31b + AgenTeeth	48.6	49.2	55.8	69.7	36.5	48.9	51.8
qwen3.5 (baseline)	44.4	33.9	55.2	68.6	30.6	38.0	45.1
qwen3.5 + AgenTeeth	59.0	55.2	62.9	81.6	45.7	53.7	60.6
mistral-large-3 (baseline)	37.3	34.5	42.9	43.6	38.9	35.9	38.3
mistral-large-3 + AgenTeeth	53.1	57.9	62.1	83.4	46.7	55.3	59.0

Appendix

Table 7: Open-ended results under the GPT-4o judge, in percent. Tool access improves every backbone in every category without exception under this judge. Absolute scores are lower than under GPT-5-mini while the ordering of systems is preserved.

Reported in the MMOral-Bench study [ 8 ]
Model	Teeth	Patho	His	Jaw	Summ	Overall
Ovis2-34B	45.8	51.6	53.9	79.4	79.2	56.8
Qwen-QVQ-72B	48.7	49.1	59.3	74.5	51.0	56.6
HealthGPT-XL32	39.7	44.1	51.5	76.4	79.2	52.0
GPT-4o	39.7	41.0	46.7	55.8	56.3	45.4
Matched baselines (ours)

Appendix

Table 8: Closed-ended MMOral-Bench results, in percent. All four AgenTeeth configurations exceed the best previously reported closed-ended Overall (56.8), and our strongest reaches 74.1. The exceptions are Jaw and Summary, where Ovis2-34B remains marginally stronger; the Jaw result is consistent with the bone-loss detector precision discussed in the main paper.

Closed-ended
Model	Arm	$n$	In	Out	Total	Med.	Disp.	Synth.	Grand
Gemma4-31B	Baseline	491	3,227	158	3,385	3,328	2,118	1,266	1,661,882
Gemma4-31B	Tool-aided	491	5,863	139	6,003	4,581	3,856	2,146	2,947,317
Qwen3.5	Baseline	491	13,278	8,080	21,358	19,331	12,082	9,276	10,401,435
Qwen3.5	Tool-aided	491	16,629	4,765	21,394	19,128	9,582	11,812	10,504,621
Mistral-Large-3	Baseline	491	6,778	126	6,904	7,277	3,966	2,937	3,389,671

Appendix

Table 9: Token usage per question by model and arm. Dispatch and Synthesis are the mean total tokens of the two pipeline stages. Both arms make exactly two model calls per question.

Explore similar work

OralGPT-Plus: Learning to Use Visual Tools via Reinforcement Learning for Panoramic X-ray Analysis
Mar 6, 2026Yuxuan Fan, Jing Hao, Hong Chen +5Medical Image Analysis Medical VLMs
The Hong Kong University of Science and Technology (GZ) · School of Computer Science, Peking University · Faculty of Dentistry, The University of Hong Kong +3
OPGAgent: An Agent for Auditable Dental Panoramic X-ray Interpretation
Feb 28, 2026Zhaolin Yu, Litao Yang, Ben Babicka +7Medical Imaging Tool-Using Vision-Language Agents
AIM for Health Lab, Faculty of Information Technology, Monash University, Melbourne, Australia · Faculty of Information Technology, Monash University, Melbourne, Australia · Curae Health, Melbourne, Australia +4
Prompt-Based Caption Generation for Single-Tooth Dental Images Using Vision-Language Models
Mar 8, 2026Anastasiia Sukhanova, Aiden Taylor, Julian Myers +5Image Captioning Medical VLMs
Marshall University · West Virginia State University

Mar 6, 2026J/K move · Enter open · S save

OralGPT-Plus: Learning to Use Visual Tools via Reinforcement Learning for Panoramic X-ray Analysis

Yuxuan Fan, Jing Hao, Hong Chen, Jiahao Bao+4

The Hong Kong University of Science and Technology (GZ) · School of Computer Science, Peking University · Faculty of Dentistry, The University of Hong Kong+3

Medical Image Analysis Medical VLMs Tool Use in VLMs RL for VLMs

Open paper AskPDF arXiv

Panoramic dental radiographs require fine-grained spatial reasoning, bilateral symmetry understanding, and multi-step diagnostic verification, yet existing vision-language models operate under a static single-pass paradigm that limits their clinical reliability. In this paper, we introduce OralGPT-Plus, an agentic vision-language model designed to perform iterative and symmetry-aware diagnostic reasoning for panoramic dental radiograph analysis. To support this paradigm, we construct DentalProbe, a five-thousand-image dataset with expert-curated diagnostic trajectories that provide structured supervision for localized inspection and contralateral comparison. We further develop a Reinspection-driven reinforcement learning framework that encourages clinically meaningful re-examination and stabilizes long-horizon reasoning with rubric-based reward and conditioned diagnostic-driven reward. In parallel, we present MMOral-X, the first benchmark for holistic panoramic diagnosis, containing 300 open-ended questions and region-level annotations across multiple difficulty levels. OralGPT-Plus demonstrates consistent and reliable improvements over strong baselines on MMOral-X and established panoramic benchmarks, indicating the effectiveness of interactive and symmetry-informed reasoning. Our work highlights the value of agentic modeling for dental imaging and provides a foundation for future research in clinically aligned panoramic radiograph analysis.

Abstract

Explore similar work

OralGPT-Plus: Learning to Use Visual Tools via Reinforcement Learning for Panoramic X-ray Analysis

OPGAgent: An Agent for Auditable Dental Panoramic X-ray Interpretation

Prompt-Based Caption Generation for Single-Tooth Dental Images Using Vision-Language Models