cs.CVNov 14, 2026

Puzzle: One Pass Is Not Enough for Open-Vocabulary Remote Sensing Segmentation

Authors: Wensong Xu, Siqi Lu, Jibao Yuan, Yongbin Zheng

Organizations: 1National University of Defense Technology

Abstract

Open-vocabulary remote sensing image segmentation (OVRSIS) enables the segmentation of unseen classes without additional annotation or retraining. By analyzing the predictions of existing OVRSIS methods, we find that many misclassified patches coexist with visually similar, correctly classified patches of the same class within the same image. Unfortunately, most of these methods stop after a single inference rather than using their predictions to guide further inference. Motivated by this observation, we propose Puzzle, a plug-and-play iterative re-inference framework that combines prediction history with a visual-similarity prior to progressively refine patch-level attention. Puzzle employs a lightweight history-conditioned controller to regulate each patch's contribution as an information source and its reliance on visually similar patches, while an adaptive stopping rule determines when to stop. It keeps the segmentation model frozen and reuses the same controller across different models and datasets without retraining. Extensive experiments demonstrate consistent performance improvements across diverse OVRSIS models and datasets, highlighting the strong transferability of Puzzle. Moreover, Puzzle can further improve the performance of SOTA methods.

Figures & tables

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. ConInfer: Context-Aware Inference for Training-Free Open-Vocabulary Remote Sensing Segmentation

    Mar 31, 2026Wenyang Chen, Zhanxuan Hu, Yaping Zhang +2Open-Vocabulary Remote Sensing Image SegmentationImage Segmentation

  2. Perturbation-Regularized Open-Vocabulary Remote Sensing Segmentation with Unified Multi-Domain Evaluation

    Apr 17, 2026Bingyu Li, Tao Huo, Haocheng Dong +4Image SegmentationRemote Sensing Image Segmentation

  3. SatOV: Restoring Spatial Priors for Training-Free Open-Vocabulary Segmentation in Remote Sensing Imagery

    Sep 19, 2026Changhao Zhao, Linglin Zeng, Hai LiuOpen-Vocabulary Semantic SegmentationRemote Sensing Image Segmentation