Scout: Open-World Species Recognition on the Edge

Mohammad Mehdi Rastikerdar; Hui Guan; Deepak Ganesan

cs.CVSep 19, 2026

Scout: Open-World Species Recognition on the Edge

Authors: Mohammad Mehdi Rastikerdar, Hui Guan, Deepak Ganesan

Organizations: University of Massachusetts Amherst, Amherst, MA 01003, USA

Abstract

Large vision-language models (VLMs) enable recognition beyond a fixed class set, but their computational demands prevent them from running on many edge devices. Cloud offload makes this capability accessible, but sending every image consumes scarce bandwidth and communication energy. We ask how to bring the open-world recognition capability of VLMs to the edge while operating within tight compute, energy, and bandwidth budgets. Wildlife monitoring provides a natural setting for exploring this question because camera traps encounter species not known at deployment. We present Scout, an autonomous open-world recognition system that invokes a cloud VLM intermittently to teach new classes to a compact edge model. Given only the deployment location and empty site frames, Scout autonomously turns each species identified by the VLM into persistent, site-conditioned recognition capability in a resource-efficient edge model, without a predefined species list, human labeling, or manual tuning. Across 30 camera-trap deployments in three regions on an NVIDIA Jetson Orin Nano, the accuracy of Scout remains within 0.1-2.5% of a model given a predefined species list. On species outside its initial class set, Scout achieves 53.7-59.1% accuracy, compared with 56.5-65.1% for full cloud offload, while using 59-71% less deployment energy.

Figures & tables

Figure 1: Scout constructs site-conditioned training data and selects a resource-efficient model for the edge device. When the device encounters a species outside its current class set (1), a cloud VLM names the species and Scout automatically constructs new training data, retrains the classifier, and deploys the update (2). The on-device classifier recognizes the same species six hours later in the same deployment stream (3). Config lists the input image resolution and the model pruning ratio for the on-device classifier.

Figure 2: Scout system overview. At initialization, Scout identifies species likely to appear at the deployment location and constructs site-conditioned training data by combining animal foregrounds from iNaturalist with backgrounds from the deployment site. The LLM planner trains and evaluates candidate class contexts and model configurations before deploying a compact edge model. During deployment, the edge model handles familiar species locally and uploads only out-of-context (OoC) frames to the cloud VLM. A cloud response may correct an edge prediction, trigger a different class context, or identify a new species. For a new species, Scout constructs additional training data and updates the edge model so that future observations can be processed locally.

Method	Serengeti		Nkhotakota		New Hampshire
Method	Acc. (K./N.)	E. Gain	Acc. (K./N.)	E. Gain	Acc. (K./N.)	E. Gain
GPT-5	87.4 (89.0/65.1)	$1.00\times$	85.4 (87.8/63.8)	$1.00\times$	87.3 (88.0/56.5)	$1.00\times$
BioCLIP 2	72.7 (74.5/47.8)	$1.00\times$	64.4 (67.3/36.0)	$1.00\times$	84.7 (85.2/60.1)	$1.00\times$
SpeciesNet	93.7 (94.4/84.2)	$1.00\times$	69.2 (71.8/44.0)	$1.00\times$	91.6 (91.9/78.2)	$1.00\times$
On-Device	70.1 (75.1/0.0)	$2.68\times$	73.8 (81.4/0.0)	$1.48\times$	78.2 (80.3/0.0)	$1.18\times$
CACTUS	66.5 (71.4/0.0)	$2.54\times$	74.0 (81.5/0.0)	$2.34\times$	78.4 (80.1/0.0)	$3.04\times$

Table 1: End-to-end performance averaged over 10 camera locations from each dataset. Accuracy is reported overall and for known (K.) and novel (N.) species using the partition of Section 4.1 . Full-offload models do not use the initial class set; but for comparison, K./N. reports their accuracy on the corresponding image subsets. Scout-CS selects its own initial set, so only its overall accuracy is comparable. Energy gain is Full-Offload deployment energy divided by the method’s, so higher is better

Figure 3: Accuracy–energy trade-off. The dashed line in each plot shows the Pareto envelope formed by the baselines. Both Scout and Scout-CS are above the baseline envelope on all three datasets.

Selection Method	Serengeti		Nkhotakota
Selection Method	Acc. (%)	E. Gain	Acc. (%)	E. Gain
Palleon (B4) [ 13 ]	74.2	$2.77\times$	75.8	$1.70\times$
CACTUS (B0) [ 35 ]	81.8	$2.72\times$	82.8	$2.49\times$
Random Search	81.5	$2.97\times$	82.2	$2.75\times$
Scout LLM Planner (B0)	84.7	$\mathbf{3.11\times}$	85.0	$\mathbf{2.73\times}$

Table 2: Accuracy and energy gain for different context and model selection methods. Random search uses the same candidate-evaluation budget per event as the LLM planner. Energy gain is relative to Full-Offload in Table 1 .

Figure 4: Initial class-set size. Accuracy and deployment energy on Serengeti when the initial class-set size is 7 or 15 species in controlled evaluation. The dashed line shows the baseline Pareto envelope.

Figure 5: Bandwidth sweep. Accuracy and deployment energy on Serengeti for regional throughput scaled by

0.2

0.5

1

2

, and

5

. The dashed line shows the baseline Pareto envelope at each bandwidth.

Edge ML

Edge machine learning runs and adapts models on nearby devices to reduce latency, bandwidth use, and exposure of sensitive data.

See all 129 papers

Can Edge-Deployable Vision-Language Models Identify Species?
Sep 10, 2026William Zhou, Mayukha Siripuram, Xiao Yan +2VLM Evaluation VLM Robustness
Plano West Senior High School
Standing on Open Shoulders: Regional Fine-Tuning of Open-Weight Camera-Trap Models for UK Wildlife with Modest Data and Compute
Oct 23, 2026Paul FergusImage Classification Wildlife Monitoring
Liverpool John Moores University, Byrom Street, Liverpool L3 3AF, United Kingdom
EdgeLPR: On the Deep Neural Network trade-off between Precision and Performance in LiDAR Place Recognition
May 4, 2026Pierpaolo Serio, Hetian Wang, Zixiang Wei +4Mixed-Precision Quantization Efficient Neural Network Inference
Department of Information Engineering, University of Pisa, Pisa, Italy. · School of Engineering and Material Science, Queen Mary University of London, London, U.K. · Research and Development, Toyota Material Handling Manufacturing Italy, Bologna, Italy.

Sep 10, 2026J/K move · Enter open · S save

Can Edge-Deployable Vision-Language Models Identify Species?

William Zhou, Mayukha Siripuram, Xiao Yan, Ziqi Liu+1

Plano West Senior High School

VLM Evaluation VLM Robustness Vision-Language Models Image Classification

Open paper AskPDF arXiv

Camera traps often run in the field on edge hardware with limited or no connectivity, making small, locally-deployable vision-language models (VLMs) -- not frontier-scale ones -- the practically relevant class to evaluate for species identification. We test whether models in this deployment-relevant 2--8B range carry genuine taxonomic knowledge, evaluating four such VLMs (Qwen3-VL 2B/4B/8B, Gemma3 4B) against the domain-specific specialist BioCLIP (300M parameters) on a 96-species task, comparing clean iNaturalist photographs against camera-trap imagery from 6 LILA.science collections, on two independently-sampled evaluation sets. All models identify species far above chance, but every model -- general-purpose or specialist -- degrades sharply on field imagery (domain gaps of 9.6--26.6 percentage points, consistent across taxonomic levels and both evaluation sets), indicating the degradation reflects general image legibility rather than fine-grained discrimination failure. BioCLIP substantially outperforms every VLM tested (by 33.2--59.2 percentage points across an expanded 200-image sample for every model) despite its far smaller size, suggesting the gap reflects specialized training data rather than model scale; yet BioCLIP's own domain gap (18.0 points) is statistically indistinguishable from the best VLM's (22.3 points), suggesting the clean-to-field degradation itself is a property of the image-quality shift rather than a general-purpose-model weakness. Under open-set prompting, 5.9--9.6% of responses are syntactically valid but taxonomically nonexistent species names; the relative fabrication-rate ranking across models replicates exactly across both evaluation sets, a more robust finding than any single point estimate.

Abstract

Edge ML

Explore similar work

Can Edge-Deployable Vision-Language Models Identify Species?

Standing on Open Shoulders: Regional Fine-Tuning of Open-Weight Camera-Trap Models for UK Wildlife with Modest Data and Compute

EdgeLPR: On the Deep Neural Network trade-off between Precision and Performance in LiDAR Place Recognition