Distilling Vision-Language Models for On-Device Fire Understanding

Mohammad Kazzazi; Zixuan Liu; Siavash Khajavi

cs.AISep 5, 2026

Distilling Vision-Language Models for On-Device Fire Understanding

Authors: Mohammad Kazzazi, Zixuan Liu, Siavash Khajavi

Organizations: EPFL, Lausanne, Switzerland · Detectium, Espoo, Finland

Abstract

Vision-language models (VLMs) offer a promising alternative to conventional fire detection systems by reasoning about the semantic context of a scene and thus reducing false alarms, yet their large model size makes deployment on embedded fire sensors impractical. In this paper, we study how domain-specialized VLMs can be compressed for fully on-device deployment without losing the safety-critical behavior required for fire detection. We develop a teacher-student knowledge distillation framework in which large VLMs fine-tuned for fire understanding can be distilled into lightweight students. Experiments across multiple VLM families and model scales show that compact students preserve most of their teachers' fire-understanding capability. We further deploy the distilled models on our commercial Detectium fire detection sensor and jointly evaluate reasoning accuracy, latency, and memory usage. The results show that compression and deployment affect not only accuracy but also model failure modes, with Qwen2.5-0.5B providing the strongest overall deployment trade-off. Our findings provide broader guidance for deploying domain-specialized VLMs in resource-constrained, safety-critical settings.

Figures & tables

Qwen2.5 family
		Fire-scene reasoning			Detection
Model	Vision encoder	Burning object	Environment	Severity	Fire/non-fire
Qwen2.5-7B (Teacher)	CLIP ViT-L/336	80.77	86.34	88.86	99.79
Qwen2.5-7B (Teacher)	MobileCLIP-S2	81.43	78.51	83.95	99.72
Qwen2.5-0.5B	CLIP ViT-L/336	76.92 $\pm$ 1.16	85.76 $\pm$ 1.48	88.28 $\pm$ 1.46	99.75 $\pm$ 0.04
Qwen2.5-1.5B	CLIP ViT-L/336	77.19 $\pm$ 1.63	85.41 $\pm$ 1.61	87.44 $\pm$ 1.30	99.56 $\pm$ 0.17

Table 1: Fire-understanding accuracy (%) across model families on the DetectiumFire evaluation set. The best-performing student within each family is shown in bold .

Qwen2.5 family
		Errors (counts)		Rates (%)
Model	Objective	False alarms	Missed fires	Recall	Specificity	Accuracy
Qwen2.5-0.5B	$\mathcal{L}_{\mathrm{KL}}$ only	671.3 $\pm$ 10.4	0 0.0 $\pm$ 0.0	100.00 $\pm$ 0.00	0 3.82 $\pm$ 1.49	53.76 $\pm$ 0.72
Qwen2.5-0.5B	$+\,\mathcal{L}_{\mathrm{CE}}$	00 2.0 $\pm$ 0 0.0	0 1.7 $\pm$ 0.6	0 99.78 $\pm$ 0.08	99.71 $\pm$ 0.00	99.75 $\pm$ 0.04
Qwen2.5-1.5B	$\mathcal{L}_{\mathrm{KL}}$ only	549.7 $\pm$ 28.0	0 0.0 $\pm$ 0.0	100.00 $\pm$ 0.00	21.25 $\pm$ 4.01	62.14 $\pm$ 1.93
Qwen2.5-1.5B	$+\,\mathcal{L}_{\mathrm{CE}}$	00 4.7 $\pm$ 0 2.1	0 1.7 $\pm$ 0.6	0 99.78 $\pm$ 0.08	99.33 $\pm$ 0.30	99.56 $\pm$ 0.17

Table 2: Comparison of safety-relevant detection errors under the two distillation objectives in Eq. 4 and Eq. 2 . False alarms denote non-fire images incorrectly classified as fire, while missed fires denote fire images incorrectly classified as non-fire. Recall is the proportion of fire images correctly identified, and specificity is the proportion of non-fire images correctly identified.

		Fire-scene reasoning						Detection
		Burning object		Environment		Severity		Fire/non-fire
Model	Vision encoder	Acc.	$\Delta$	Acc.	$\Delta$	Acc.	$\Delta$	Acc.	$\Delta$
Qwen2.5 family
Qwen2.5-0.5B	CLIP ViT-L/336	75.95 $\pm$ 0.85	$-0.97$	85.10 $\pm$ 0.43	$-0.66$	88.37 $\pm$ 2.48	$+0.09$	99.52 $\pm$ 0.12	$-0.23$
Qwen2.5-1.5B	CLIP ViT-L/336	75.73 $\pm$ 1.38	$-1.46$	84.13 $\pm$ 0.28	$-1.28$	87.31 $\pm$ 1.20	$-0.13$	99.54 $\pm$ 0.20	$-0.02$
Qwen2.5-3B	CLIP ViT-L/336	76.88 $\pm$ 0.41	$-1.41$	86.34 $\pm$ 1.04	$+0.75$	88.82 $\pm$ 0.55	$\phantom{+}0.00$	97.87 $\pm$ 0.62	$-0.64$

Table 3: Fire-understanding accuracy (%) of the exported student models measured on the commercial Detectium fire detection sensor.

\Delta

denotes the change relative to the corresponding student before export, as reported in Table 1 . The best-performing student within each model family is shown in bold .

Model	Vision	TTFT	E2E	RSS	Fits
		(ms)	(ms)	(MB)
Qwen2.5 family
Qwen2.5-0.5B	CLIP	3818	4034	1880	✓
Qwen2.5-1.5B	CLIP	6862	7419	4210	✓
Qwen2.5-3B	CLIP	11598	12519	7341	✓
Qwen2.5-3B	MobileCLIP	1757	18976	8778	$\times$

Table 4: On-device latency and memory usage of the exported student models. The final column indicates whether each model fits within the memory constraints of the Detectium sensor. Models marked

\times

exceed the available memory and are benchmarked with additional memory for reference only. The best overall value for each metric is shown in bold .

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Subset	NO.	Source
Description-style	12,000	LLaVA mix665k
Localization	2,409	LLaVA mix665k
Simple VQA	3,000	LLaVA mix665k
Fire VQA	13,077	DetectiumFire train
Bbox + non-fire	5,000	DetectiumFire train

Appendix

Table 5: Composition of the training data used in the task-specific fine-tuning stage.

Configuration	Initialization	Fine-Tuning
LLM	✗	✓ (LoRA)
VL Adaptor	✓	✓
ViT	✗	✗
LLM sequence length	2048	1536
Optimizer	AdamW	AdamW
Optimizer hyperparameter	$\beta_{1}=0.9,\ \beta_{2}=0.999$	$\beta_{1}=0.9,\ \beta_{2}=0.999$

Appendix

Table 6: Training hyperparameters for teacher initialization and task-specific fine-tuning.

Configuration	Initialization	Distillation
LLM	✗	✓ (LoRA)
VL Adaptor	✓	✓
ViT	✗	✗
LLM sequence length	2048	2048
Optimizer	AdamW	AdamW
Optimizer hyperparameter	$\beta_{1}=0.9,\ \beta_{2}=0.999$	$\beta_{1}=0.9,\ \beta_{2}=0.999$

Appendix

Table 7: Training hyperparameters for student initialization and distillation.

Qwen2.5 family
		Fire-scene reasoning			Detection
Model	Objective	Burning object	Environment	Severity	Fire/non-fire	$\Delta$
Qwen2.5-0.5B	$\mathcal{L}_{\mathrm{KL}}$ only	77.59 $\pm$ 0.48	86.16 $\pm$ 0.28	88.81 $\pm$ 0.31	53.76 $\pm$ 0.72
Qwen2.5-0.5B	$+\,\mathcal{L}_{\mathrm{CE}}$	76.92 $\pm$ 1.16	85.76 $\pm$ 1.48	88.28 $\pm$ 1.46	99.75 $\pm$ 0.04	+45.99
Qwen2.5-1.5B	$\mathcal{L}_{\mathrm{KL}}$ only	79.62 $\pm$ 0.15	87.23 $\pm$ 0.43	88.46 $\pm$ 0.27	62.14 $\pm$ 1.93
Qwen2.5-1.5B	$+\,\mathcal{L}_{\mathrm{CE}}$	77.19 $\pm$ 1.63	85.41 $\pm$ 1.61	87.44 $\pm$ 1.30	99.56 $\pm$ 0.17	+37.42

Appendix

Table 8: Effect of the distillation objective on fire-understanding accuracy (%) across model families on the DetectiumFire evaluation set.

\Delta

denotes the change in fire/non-fire detection accuracy. Removing

\mathcal{L}_{\mathrm{CE}}

leaves performance on the three fire-scene reasoning tasks largely unchanged, and in several cases slightly improves it, while reducing fire/non-fire detection accuracy by up to 45.99 percentage.

Teacher	Encoder	False alarms	Missed fires	Recall	Specificity	Accuracy
		Errors (counts)		Rates (%)
Qwen2.5-7B	CLIP	2	1	99.87	99.71	99.79
Qwen2.5-7B	MobileCLIP	2	2	99.73	99.71	99.72
Qwen3-8B	CLIP	1	1	99.87	99.86	99.86
Qwen3-8B	MobileCLIP	2	2	99.73	99.71	99.72
Llama-3.1-8B	CLIP	1	1	99.87	99.86	99.86

Appendix

Table 9: Safety-relevant detection errors for the teacher models.

Family	Prompt template
Qwen2.5	USER: <image>{prompt} ASSISTANT:
Qwen3	<\|im_start\|>system {system}<\|im_end\|> <\|im_start\|>user <image> {prompt}<\|im_end\|> <\|im_start\|>assistant
Llama-3.2	<\|begin_of_text\|><\|start_header_id\|>system<\|end_header_id\|> {system}<\|eot_id\|><\|start_header_id\|>user<\|end_header_id\|> <image> {prompt}<\|eot_id\|><\|start_header_id\|>assistant<\|end_header_id\|>
Gemma-2	<start_of_turn>user <image> {prompt}<end_of_turn> <start_of_turn>model

Appendix

Table 10: Prompt templates used for each student family at inference, matching the formats used during training. For Qwen3 and Llama-3.2, {system} denotes the system message specified in the text. Gemma-2 uses an empty system prompt.

		Errors (counts)				Rates (%)
		False alarms		Missed fires		Recall		Specificity
Model	Vision encoder	Count	$\Delta$	Count	$\Delta$	Rate	$\Delta$	Rate	$\Delta$
Qwen2.5 family
Qwen2.5-0.5B	CLIP ViT-L/336	0 5.7 $\pm$ 2.5	$+3.7$	0 1.3 $\pm$ 1.2	$-0.4$	99.83 $\pm$ 0.16	$+0.05$	99.18 $\pm$ 0.36	$-0.53$
Qwen2.5-1.5B	CLIP ViT-L/336	0 5.0 $\pm$ 1.7	$+0.3$	0 1.7 $\pm$ 1.2	$\phantom{+}0.0$	99.77 $\pm$ 0.16	$\phantom{+}0.00$	99.28 $\pm$ 0.24	$-0.04$
Qwen2.5-3B	CLIP ViT-L/336	27.7 $\pm$ 9.0	$+10.4$	0 3.3 $\pm$ 0.6	$-1.0$	99.56 $\pm$ 0.08	$+0.13$	96.03 $\pm$ 1.29	$-1.49$

Appendix

Table 11: Safety-relevant detection errors of the exported student models on the Detectium sensor.

\Delta

denotes the change relative to the corresponding student before export, as reported in Table 2 .

VLM Distillation

Vision-language model distillation transfers multimodal capabilities to smaller, faster models for efficient deployment.

See all 127 papers

Mar 17, 2026

Empirical Recipes for Efficient and Compact Vision-Language Models

Deploying vision-language models (VLMs) in resource-constrained settings demands low latency and high throughput, yet existing compact VLMs often fall short of the inference speedups their smaller parameter counts suggest. To explain this discrepancy, we conduct an empirical end-to-end efficiency analysis and systematically profile inference to identify the dominant bottlenecks. Based on these findings, we develop optimization recipes tailored to compact VLMs that substantially reduce latency while preserving accuracy. These techniques cut time to first token (TTFT) by 53% on InternVL3-2B and by 93% on SmolVLM-256M. Our recipes are broadly applicable across both VLM architectures and common serving frameworks, providing practical guidance for building efficient VLM systems. Beyond efficiency, we study how to extend compact VLMs with structured perception outputs and introduce the resulting model family, ArgusVLM. Across diverse benchmarks, ArgusVLM achieves strong performance while maintaining a compact and efficient design.

Jiabo Huang, Zhizhong Li, Sina Sajadmanesh +2

Sony AI.

Vision-Language Models Efficient VLM Inference

Ask PDF

Mar 26, 2026

Mechanistically Interpreting Compression in Vision-Language Models

Compressed vision-language models (VLMs) are widely used to reduce memory and compute costs, making them a suitable choice for real-world deployment. However, compressing these models raises concerns about whether internal computations and safety behaviors are preserved. In this work, we use causal circuit analysis and crosscoder-based feature comparisons to examine how pruning and quantization fundamentally change the internals across representative VLMs. We observe that pruning generally keeps circuit structure intact but rotates and attenuates internal features, while quantization modifies the circuits at a higher level yet leaves the surviving features better aligned. Leveraging this insight, we also introduce VLMSafe-420, a novel benchmark that pairs harmful inputs with matched benign counterfactuals across various safety categories. Our findings show that pruning causes a sharp drop in genuine refusal behavior, suggesting that the choice of compression has safety implications.

Veeraraju Elluru, Arth Singh, Roberto Aguero +3

Indian Institute of Technology Jodhpur · AIM Intelligence · National Institute of Technology Agartala +3

Cultural Safety Evaluation of VLMs LVLM Pruning VLM Quantization VLM Interpretability Neural Network Pruning VLM Evaluation

Ask PDF

Sep 23, 2026

VisionPsy-Nano: Improving Accuracy, Efficiency, and Reliability in On-Device Vision-Language Models

Sub-billion-parameter Vision-Language Models are increasingly viable for on-device deployment, yet compact model size alone does not guarantee usability. On a phone, such a model can still require more than two minutes to produce its first token. On-device usability depends on three axes: accuracy, efficiency, and behavioral reliability; standard benchmarks miss the third, with answers too short to expose doom loops and prompts too benign to probe adversarial safety. We introduce a diagnosis-driven post-training recipe in which a teacher VLM stress-tests the student, uncovers failure modes beyond human priors, and converts them into targeted supervision and preference alignment, supplementing generic data scaling with failure-driven optimization. Coupled with two visual-token policies, the recipe yields two accuracy-efficiency variants with improved behavioral reliability. \NanoFull attains a 62.3 normalized average over 17 benchmarks, the highest among openly released

\sim

0.5B models, +7.4 over its base at identical architecture and token budget, with doom-loop rates at or below the strongest baseline's. \FlashFull retains 61.4 while cutting warm time-to-first-token on a Pixel 9 from 138,s to 6.1,s (23

\times

). By jointly addressing all three axes, we move compact VLMs toward practical on-device usability.

Khurram Azeem Hashmi, Mohammadreza Zolfaghari, Changdae Park +6

Tether AI Research

VLM Post-Training Vision-Language Models Efficient VLM Inference VLM Robustness VLM Evaluation

Ask PDF

Abstract

VLM Distillation

Explore similar work

Empirical Recipes for Efficient and Compact Vision-Language Models

Mechanistically Interpreting Compression in Vision-Language Models

VisionPsy-Nano: Improving Accuracy, Efficiency, and Reliability in On-Device Vision-Language Models