Distilling Vision-Language Models for On-Device Fire Understanding

Mohammad Kazzazi; Zixuan Liu; Siavash Khajavi

cs.AISep 5, 2026

Distilling Vision-Language Models for On-Device Fire Understanding

Authors: Mohammad Kazzazi, Zixuan Liu, Siavash Khajavi

Organizations: EPFL, Lausanne, Switzerland · Detectium, Espoo, Finland

Abstract

Vision-language models (VLMs) offer a promising alternative to conventional fire detection systems by reasoning about the semantic context of a scene and thus reducing false alarms, yet their large model size makes deployment on embedded fire sensors impractical. In this paper, we study how domain-specialized VLMs can be compressed for fully on-device deployment without losing the safety-critical behavior required for fire detection. We develop a teacher-student knowledge distillation framework in which large VLMs fine-tuned for fire understanding can be distilled into lightweight students. Experiments across multiple VLM families and model scales show that compact students preserve most of their teachers' fire-understanding capability. We further deploy the distilled models on our commercial Detectium fire detection sensor and jointly evaluate reasoning accuracy, latency, and memory usage. The results show that compression and deployment affect not only accuracy but also model failure modes, with Qwen2.5-0.5B providing the strongest overall deployment trade-off. Our findings provide broader guidance for deploying domain-specialized VLMs in resource-constrained, safety-critical settings.

Figures & tables

Qwen2.5 family
		Fire-scene reasoning			Detection
Model	Vision encoder	Burning object	Environment	Severity	Fire/non-fire
Qwen2.5-7B (Teacher)	CLIP ViT-L/336	80.77	86.34	88.86	99.79
Qwen2.5-7B (Teacher)	MobileCLIP-S2	81.43	78.51	83.95	99.72
Qwen2.5-0.5B	CLIP ViT-L/336	76.92 $\pm$ 1.16	85.76 $\pm$ 1.48	88.28 $\pm$ 1.46	99.75 $\pm$ 0.04
Qwen2.5-1.5B	CLIP ViT-L/336	77.19 $\pm$ 1.63	85.41 $\pm$ 1.61	87.44 $\pm$ 1.30	99.56 $\pm$ 0.17

Table 1: Fire-understanding accuracy (%) across model families on the DetectiumFire evaluation set. The best-performing student within each family is shown in bold .

Qwen2.5 family
		Errors (counts)		Rates (%)
Model	Objective	False alarms	Missed fires	Recall	Specificity	Accuracy
Qwen2.5-0.5B	$\mathcal{L}_{\mathrm{KL}}$ only	671.3 $\pm$ 10.4	0 0.0 $\pm$ 0.0	100.00 $\pm$ 0.00	0 3.82 $\pm$ 1.49	53.76 $\pm$ 0.72
Qwen2.5-0.5B	$+\,\mathcal{L}_{\mathrm{CE}}$	00 2.0 $\pm$ 0 0.0	0 1.7 $\pm$ 0.6	0 99.78 $\pm$ 0.08	99.71 $\pm$ 0.00	99.75 $\pm$ 0.04
Qwen2.5-1.5B	$\mathcal{L}_{\mathrm{KL}}$ only	549.7 $\pm$ 28.0	0 0.0 $\pm$ 0.0	100.00 $\pm$ 0.00	21.25 $\pm$ 4.01	62.14 $\pm$ 1.93
Qwen2.5-1.5B	$+\,\mathcal{L}_{\mathrm{CE}}$	00 4.7 $\pm$ 0 2.1	0 1.7 $\pm$ 0.6	0 99.78 $\pm$ 0.08	99.33 $\pm$ 0.30	99.56 $\pm$ 0.17

Table 2: Comparison of safety-relevant detection errors under the two distillation objectives in Eq. 4 and Eq. 2 . False alarms denote non-fire images incorrectly classified as fire, while missed fires denote fire images incorrectly classified as non-fire. Recall is the proportion of fire images correctly identified, and specificity is the proportion of non-fire images correctly identified.

		Fire-scene reasoning						Detection
		Burning object		Environment		Severity		Fire/non-fire
Model	Vision encoder	Acc.	$\Delta$	Acc.	$\Delta$	Acc.	$\Delta$	Acc.	$\Delta$
Qwen2.5 family
Qwen2.5-0.5B	CLIP ViT-L/336	75.95 $\pm$ 0.85	$-0.97$	85.10 $\pm$ 0.43	$-0.66$	88.37 $\pm$ 2.48	$+0.09$	99.52 $\pm$ 0.12	$-0.23$
Qwen2.5-1.5B	CLIP ViT-L/336	75.73 $\pm$ 1.38	$-1.46$	84.13 $\pm$ 0.28	$-1.28$	87.31 $\pm$ 1.20	$-0.13$	99.54 $\pm$ 0.20	$-0.02$
Qwen2.5-3B	CLIP ViT-L/336	76.88 $\pm$ 0.41	$-1.41$	86.34 $\pm$ 1.04	$+0.75$	88.82 $\pm$ 0.55	$\phantom{+}0.00$	97.87 $\pm$ 0.62	$-0.64$

Table 3: Fire-understanding accuracy (%) of the exported student models measured on the commercial Detectium fire detection sensor.

\Delta

denotes the change relative to the corresponding student before export, as reported in Table 1 . The best-performing student within each model family is shown in bold .

Model	Vision	TTFT	E2E	RSS	Fits
		(ms)	(ms)	(MB)
Qwen2.5 family
Qwen2.5-0.5B	CLIP	3818	4034	1880	✓
Qwen2.5-1.5B	CLIP	6862	7419	4210	✓
Qwen2.5-3B	CLIP	11598	12519	7341	✓
Qwen2.5-3B	MobileCLIP	1757	18976	8778	$\times$

Table 4: On-device latency and memory usage of the exported student models. The final column indicates whether each model fits within the memory constraints of the Detectium sensor. Models marked

\times

exceed the available memory and are benchmarked with additional memory for reference only. The best overall value for each metric is shown in bold .

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Subset	NO.	Source
Description-style	12,000	LLaVA mix665k
Localization	2,409	LLaVA mix665k
Simple VQA	3,000	LLaVA mix665k
Fire VQA	13,077	DetectiumFire train
Bbox + non-fire	5,000	DetectiumFire train

Appendix

Table 5: Composition of the training data used in the task-specific fine-tuning stage.

Configuration	Initialization	Fine-Tuning
LLM	✗	✓ (LoRA)
VL Adaptor	✓	✓
ViT	✗	✗
LLM sequence length	2048	1536
Optimizer	AdamW	AdamW
Optimizer hyperparameter	$\beta_{1}=0.9,\ \beta_{2}=0.999$	$\beta_{1}=0.9,\ \beta_{2}=0.999$

Appendix

Table 6: Training hyperparameters for teacher initialization and task-specific fine-tuning.

Configuration	Initialization	Distillation
LLM	✗	✓ (LoRA)
VL Adaptor	✓	✓
ViT	✗	✗
LLM sequence length	2048	2048
Optimizer	AdamW	AdamW
Optimizer hyperparameter	$\beta_{1}=0.9,\ \beta_{2}=0.999$	$\beta_{1}=0.9,\ \beta_{2}=0.999$

Appendix

Table 7: Training hyperparameters for student initialization and distillation.

Qwen2.5 family
		Fire-scene reasoning			Detection
Model	Objective	Burning object	Environment	Severity	Fire/non-fire	$\Delta$
Qwen2.5-0.5B	$\mathcal{L}_{\mathrm{KL}}$ only	77.59 $\pm$ 0.48	86.16 $\pm$ 0.28	88.81 $\pm$ 0.31	53.76 $\pm$ 0.72
Qwen2.5-0.5B	$+\,\mathcal{L}_{\mathrm{CE}}$	76.92 $\pm$ 1.16	85.76 $\pm$ 1.48	88.28 $\pm$ 1.46	99.75 $\pm$ 0.04	+45.99
Qwen2.5-1.5B	$\mathcal{L}_{\mathrm{KL}}$ only	79.62 $\pm$ 0.15	87.23 $\pm$ 0.43	88.46 $\pm$ 0.27	62.14 $\pm$ 1.93
Qwen2.5-1.5B	$+\,\mathcal{L}_{\mathrm{CE}}$	77.19 $\pm$ 1.63	85.41 $\pm$ 1.61	87.44 $\pm$ 1.30	99.56 $\pm$ 0.17	+37.42

Appendix

Table 8: Effect of the distillation objective on fire-understanding accuracy (%) across model families on the DetectiumFire evaluation set.

\Delta

denotes the change in fire/non-fire detection accuracy. Removing

\mathcal{L}_{\mathrm{CE}}

leaves performance on the three fire-scene reasoning tasks largely unchanged, and in several cases slightly improves it, while reducing fire/non-fire detection accuracy by up to 45.99 percentage.

Teacher	Encoder	False alarms	Missed fires	Recall	Specificity	Accuracy
		Errors (counts)		Rates (%)
Qwen2.5-7B	CLIP	2	1	99.87	99.71	99.79
Qwen2.5-7B	MobileCLIP	2	2	99.73	99.71	99.72
Qwen3-8B	CLIP	1	1	99.87	99.86	99.86
Qwen3-8B	MobileCLIP	2	2	99.73	99.71	99.72
Llama-3.1-8B	CLIP	1	1	99.87	99.86	99.86

Appendix

Table 9: Safety-relevant detection errors for the teacher models.

Family	Prompt template
Qwen2.5	USER: <image>{prompt} ASSISTANT:
Qwen3	<\|im_start\|>system {system}<\|im_end\|> <\|im_start\|>user <image> {prompt}<\|im_end\|> <\|im_start\|>assistant
Llama-3.2	<\|begin_of_text\|><\|start_header_id\|>system<\|end_header_id\|> {system}<\|eot_id\|><\|start_header_id\|>user<\|end_header_id\|> <image> {prompt}<\|eot_id\|><\|start_header_id\|>assistant<\|end_header_id\|>
Gemma-2	<start_of_turn>user <image> {prompt}<end_of_turn> <start_of_turn>model

Appendix

Table 10: Prompt templates used for each student family at inference, matching the formats used during training. For Qwen3 and Llama-3.2, {system} denotes the system message specified in the text. Gemma-2 uses an empty system prompt.

		Errors (counts)				Rates (%)
		False alarms		Missed fires		Recall		Specificity
Model	Vision encoder	Count	$\Delta$	Count	$\Delta$	Rate	$\Delta$	Rate	$\Delta$
Qwen2.5 family
Qwen2.5-0.5B	CLIP ViT-L/336	0 5.7 $\pm$ 2.5	$+3.7$	0 1.3 $\pm$ 1.2	$-0.4$	99.83 $\pm$ 0.16	$+0.05$	99.18 $\pm$ 0.36	$-0.53$
Qwen2.5-1.5B	CLIP ViT-L/336	0 5.0 $\pm$ 1.7	$+0.3$	0 1.7 $\pm$ 1.2	$\phantom{+}0.0$	99.77 $\pm$ 0.16	$\phantom{+}0.00$	99.28 $\pm$ 0.24	$-0.04$
Qwen2.5-3B	CLIP ViT-L/336	27.7 $\pm$ 9.0	$+10.4$	0 3.3 $\pm$ 0.6	$-1.0$	99.56 $\pm$ 0.08	$+0.13$	96.03 $\pm$ 1.29	$-1.49$

Appendix

Table 11: Safety-relevant detection errors of the exported student models on the Detectium sensor.

\Delta

denotes the change relative to the corresponding student before export, as reported in Table 2 .

VLM Distillation

Vision-language model distillation transfers multimodal capabilities to smaller, faster models for efficient deployment.

See all 127 papers

Empirical Recipes for Efficient and Compact Vision-Language Models
Mar 17, 2026Jiabo Huang, Zhizhong Li, Sina Sajadmanesh +2Vision-Language Models Efficient VLM Inference
Sony AI.
Mechanistically Interpreting Compression in Vision-Language Models
Mar 26, 2026Veeraraju Elluru, Arth Singh, Roberto Aguero +3Cultural Safety Evaluation of VLMs LVLM Pruning
Indian Institute of Technology Jodhpur · AIM Intelligence · National Institute of Technology Agartala +3
VisionPsy-Nano: Improving Accuracy, Efficiency, and Reliability in On-Device Vision-Language Models
Sep 23, 2026Khurram Azeem Hashmi, Mohammadreza Zolfaghari, Changdae Park +6VLM Post-Training Vision-Language Models
Tether AI Research

Mar 17, 2026J/K move · Enter open · S save

Empirical Recipes for Efficient and Compact Vision-Language Models

Jiabo Huang, Zhizhong Li, Sina Sajadmanesh, Weiming Zhuang+1

Sony AI.

Vision-Language Models Efficient VLM Inference

Open paper AskPDF arXiv

Deploying vision-language models (VLMs) in resource-constrained settings demands low latency and high throughput, yet existing compact VLMs often fall short of the inference speedups their smaller parameter counts suggest. To explain this discrepancy, we conduct an empirical end-to-end efficiency analysis and systematically profile inference to identify the dominant bottlenecks. Based on these findings, we develop optimization recipes tailored to compact VLMs that substantially reduce latency while preserving accuracy. These techniques cut time to first token (TTFT) by 53% on InternVL3-2B and by 93% on SmolVLM-256M. Our recipes are broadly applicable across both VLM architectures and common serving frameworks, providing practical guidance for building efficient VLM systems. Beyond efficiency, we study how to extend compact VLMs with structured perception outputs and introduce the resulting model family, ArgusVLM. Across diverse benchmarks, ArgusVLM achieves strong performance while maintaining a compact and efficient design.

Abstract

VLM Distillation

Explore similar work

Empirical Recipes for Efficient and Compact Vision-Language Models

Mechanistically Interpreting Compression in Vision-Language Models

VisionPsy-Nano: Improving Accuracy, Efficiency, and Reliability in On-Device Vision-Language Models