cs.CVNov 11, 2026

Where is My Mug? Training-Free Framework for Personalized Image Segmentation

Authors: Quang Qui-Vinh Nguyen, Hung Nguyen, Rang Nguyen, Khoi Nguyen

Organizations: Qualcomm AI Research‡ · National University of Singapore · VinFast, Vietnam

Abstract

Personalized Image Segmentation (PIS) aims to segment the specific object in a target image that matches a user-provided reference. Unlike category-based segmentation or few-shot segmentation, PIS must discriminate between multiple visually similar instances, a requirement central to deployed applications such as augmented reality, personalized content editing, robotics manipulation, inventory tracking, and fine-grained visual retrieval. We show that the strong numbers reported by existing PIS methods are largely an artifact of single-instance benchmarks: when evaluated on realistic scenes with several same-category look-alikes, prior point-prompt methods collapse, because the correct instance is frequently never surfaced as a prompt. We trace this to a recall bottleneck rather than a mask-quality one, and address it with a training-free, modular system. The pipeline first generates high-recall object proposals using an off-the-shelf class-agnostic detector, then matches each proposal to the reference using foundation-model features (e.g., BLIP-2), and finally refines the selected box with SAM. For robust matching we propose \textbf{top-KK Sinkhorn}, a patch-level distance that scores each candidate only on its most confidently matched patches; trimming the Sinkhorn dual potentials recovers the robustness of partial optimal transport at more than 100×100\times lower cost, keeping it inside the per-proposal candidate loop. To evaluate this regime, we introduce \datasetname, a realistic multi-instance benchmark. Across \datasetname, PerSeg, PerMIS, and DAVIS, our system surpasses PerSAM, P2SAM\mathrm{P}^2\mathrm{SAM}, and PDM by up to ∼ ⁣48\sim\!48 mIoU while matching their runtime and using fewer parameters, establishing a strong, deployable training-free baseline for real-world personalized segmentation.

Figures & tables

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. FoundYou: A Unified Model for Personalized Segmentation and Retrieval

    Aug 30, 2026Gabriele Trivigno, Marcos Alfaro, Claudia Cuttano +3Image SegmentationInteractive Image Segmentation

  2. SegRAG: Retrieval Augmented Spatial Prompting for Open Vocabulary Semantic Segmentation

    May 17, 2026Abderrahmene Boudiaf, Irfan Hussain, Sajid JavedImage SegmentationPromptable Image Segmentation

  3. LLMSeg: Training-Free Text-Prompted Image Segmentation with B-Splines

    Oct 21, 2026Abdullah HamdiImage SegmentationText-Guided Image Segmentation