Evaluating Few-Shot Pill Recognition Under Visual Domain Shift

W. I. Chu; G. Tarroni; L. Li

cs.CVMar 11, 2026

Evaluating Few-Shot Pill Recognition Under Visual Domain Shift

Authors: W. I. Chu, G. Tarroni, L. Li

Organizations: Department of Engineering, City St George’s, University of London, London, United Kingdom · Department of Computer Science, City St George’s, University of London, London, United Kingdom

Abstract

Adverse drug events are a significant source of preventable harm, which has led to the development of automated pill recognition systems to enhance medication safety. Real-world deployment of these systems is hindered by visually complex conditions, including cluttered scenes, overlapping pills, reflections, and diverse acquisition environments. This study investigates few-shot pill recognition from a deployment-oriented perspective, prioritizing generalization under realistic cross-dataset domain shifts over architectural innovation. A two-stage object detection framework is employed, involving base training followed by few-shot fine-tuning. Models are adapted to novel pill classes using one, five, or ten labeled examples per class and are evaluated on a separate deployment dataset featuring multi-object, cluttered scenes. The evaluation focuses on classification-centric and error-based metrics to address heterogeneous annotation strategies. Findings indicate that semantic pill recognition adapts rapidly with few-shot supervision, with classification performance reaching saturation even with a single labeled example. However, stress testing under overlapping and occluded conditions demonstrates a marked decline in localization and recall, despite robust semantic classification. Models trained on visually realistic, multi-pill data consistently exhibit greater robustness in low-shot scenarios, underscoring the importance of training data realism and the diagnostic utility of few-shot fine-tuning for deployment readiness.

Figures & tables

Dataset	Images	Classes	Instances	Annotation
			per Image	Type
CURE	8,973	196	1	Full-image
				bounding
				box
MEDISEG	8,262	32	Multiple	Instance-level
				bounding

TABLE I: Characteristics of the base-training datasets used in this study. CURE provides large-scale, single-instance supervision under controlled conditions, while MEDISEG emphasizes realistic multi-object scenes with instance-level annotations.

Fig. 1: Example images from the MEDISEG and CURE datasets. The MEDISEG image (left) contains multiple pill instances within a container, while the CURE image (right) depicts a single pill captured under controlled conditions.

Fig. 2: Representative examples from the novel deployment dataset used for few-shot adaptation and evaluation. Images depict multiple pill instances with varying degrees of overlap, occlusion, and interference from pill containers and surrounding objects, reflecting realistic medication handling conditions.

Fig. 3: Evolution of classification-centric and loss-based metrics during few-shot fine-tuning for CURE- and MEDISEG-trained models under 1-shot, 5-shot, and 10-shot configurations. Shown are foreground classification accuracy, foreground false negative rate, classification loss, and total detection loss over 2,000 fine-tuning iterations.

	fg_cls_accuracy	false_negative	loss_cls	total_loss
CURE	$0.989$	$0.011$	$0.005$	$0.015$
1-shot	$\pm 0.004$	$\pm 0.004$	$\pm 0.001$	$\pm 0.003$
CURE	$0.980$	$0.020$	$0.014$	$0.039$
5-shot	$\pm 0.004$	$\pm 0.004$	$\pm 0.001$	$\pm 0.003$
CURE	$0.977$	$0.023$	$0.019$	$0.055$
10-shot	$\pm 0.004$	$\pm 0.004$	$\pm 0.002$	$\pm 0.005$

TABLE II: Metrics reported as mean

\pm

standard deviation over the final 500 training iterations. Bounding-box metrics are omitted due to annotation heterogeneity across training sources; results focus on classification-centric signals from the ROI head.

	fg_cls_	fg_cls_	false_	false_
	accuracy	accuracy	negative	negative
	(CURE)	(MEDISEG)	(CURE)	(MEDISEG)
1-shot	$0.989$	$0.994$	$0.011$	$0.006$
	$\pm 0.004$	$\pm 0.005$	$\pm 0.004$	$\pm 0.005$
5-shot	$0.980$	$0.991$	$0.020$	$0.009$
	$\pm 0.004$	$\pm 0.002$	$\pm 0.004$	$\pm 0.002$

TABLE III: Foreground classification accuracy and false negative rate are reported as mean ± standard deviation over the final 500 training iterations for 1-, 5-, and 10-shot fine-tuning. Metrics are grouped by shot count.

	fg_cls_	false_	loss_	loss_rpn_	total_
	accuracy	negative	cls	cls	loss
CURE
1-shot	0.131	0.816	0.351	0.863	1.326
CURE
5-shot	0.372	0.465	0.421	0.224	0.844
CURE

TABLE IV: Few-shot evaluation on an overlap-only test set. Metrics emphasise foreground recognition and error behaviour under realistic visual complexity. In the 1-shot regime, models base-trained on MEDISEG achieve up to a 210% relative improvement in foreground classification accuracy compared to CURE.

Fig. 4: Qualitative comparison of few-shot detection results for models base-trained on MEDISEG and CURE under 1-shot, 5-shot, and 10-shot fine-tuning. Each row shows a different test image, and each column corresponds to a specific base-training dataset and supervision level. Predicted bounding boxes illustrate detection and classification behavior under clutter, overlap, and occlusion.

Few-Shot Object Detection

Few-shot object detection adapts detectors to recognize and localize new object classes from only a handful of labeled examples.

See all 27 papers

A dataset of medication images with instance segmentation masks for preventing adverse drug events
Mar 11, 2026W. I. Chu, S. Hirani, G. Tarroni +1Object Detection Instance Segmentation
City St George’s, University of London, School of Science & Technology, London, EC1V 0HB, United Kingdom · City St George’s, University of London, School of Health & Medical Sciences, London, EC1V 0HB, United Kingdom · Imperial College London, Dept of Computer Science, London, SW7 2AZ, United Kingdom
Prompt-Driven Simulation with Feature Perturbation for Cross-Domain Few-Shot Object Detection
Aug 2, 2026Linhai Zhuo, Junxi Cai, Tianwen Qian +2Synthetic Data Augmentation Few-Shot Domain Adaptation
Fuzhou University · East China Normal University · Xiamen University +1
Enabling Validation for Robust Few-Shot Recognition
Jun 5, 2025Hanxin Wang, Tian Liu, Shu KongFew-Shot Learning Few-Shot Image Classification
University of Macau · Texas A&M University · Institute of Collaborative Innovation

Mar 11, 2026J/K move · Enter open · S save

A dataset of medication images with instance segmentation masks for preventing adverse drug events

W. I. Chu, S. Hirani, G. Tarroni, L. Li

City St George’s, University of London, School of Science & Technology, London, EC1V 0HB, United Kingdom · City St George’s, University of London, School of Health & Medical Sciences, London, EC1V 0HB, United Kingdom · Imperial College London, Dept of Computer Science, London, SW7 2AZ, United Kingdom

Object Detection Instance Segmentation

Open paper AskPDF arXiv

Medication errors and adverse drug events (ADEs) pose significant risks to patient safety, often arising from difficulties in reliably identifying pharmaceuticals in real-world settings. AI-based pill recognition models offer a promising solution, but the lack of comprehensive datasets hinders their development. Existing pill image datasets rarely capture real-world complexities such as overlapping pills, varied lighting, and occlusions. MEDISEG addresses this gap by providing instance segmentation annotations for 32 distinct pill types across 8262 images, encompassing diverse conditions from individual pill images to cluttered dosette boxes. We trained YOLOv8 and YOLOv9 on MEDISEG to demonstrate their usability, achieving mean average precision at IoU 0.5 of 99.5 percent on the 3-Pills subset and 80.1 percent on the 32-Pills subset. We further evaluate MEDISEG under a few-shot detection protocol, demonstrating that base training on MEDISEG significantly improves recognition of unseen pill classes in occluded multi-pill scenarios compared to existing datasets. These results highlight the dataset's ability not only to support robust supervised training but also to promote transferable representations under limited supervision, making it a valuable resource for developing and benchmarking AI-driven systems for medication safety.

Abstract

Few-Shot Object Detection

Explore similar work

A dataset of medication images with instance segmentation masks for preventing adverse drug events

Prompt-Driven Simulation with Feature Perturbation for Cross-Domain Few-Shot Object Detection

Enabling Validation for Robust Few-Shot Recognition