Open-Vocabulary Semantic Segmentation

Follow this topic Ask about this topic

Open-vocabulary semantic segmentation aims to assign meaningful labels to each pixel, including categories not seen during training, making it useful when the set of objects or environments can change. Many methods reuse vision-language models such as CLIP, DINO, and SAM, combining text–image similarity with attention, mask proposals, feature alignment, or prompt and prototype calibration; others train with region-level text, pseudo-labels, or generated data. Systems are tested on general benchmarks such as Pascal, COCO, and ADE20K, as well as remote-sensing, urban-scene, 3D, and other specialized settings. A central challenge is balancing broad category recognition with precise boundaries and small-object localization, especially under domain shifts, while keeping inference efficient and evaluation protocols comparable.

Latest papers

87 papers8 papers in the last four weeks, up 14% on the four weeks before. 0.0% of all new papers.

OV-Stitcher: A Global Context-Aware Framework for Training-Free Open-Vocabulary Semantic Segmentation
Apr 9, 2026Seungjae Moon, Seunghyun Oh, Youngmin RoOpen-Vocabulary Semantic Segmentation
Machine Intelligence Laboratory, University of Seoul, Korea
Direct Segmentation without Logits Optimization for Training-Free Open-Vocabulary Semantic Segmentation
Apr 9, 2026Jiahao Li, Yang Lu, Yachao Zhang +3Open-Vocabulary Semantic Segmentation
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University. · Hanjiang National Laboratory. · East China Normal University.
SPAR: Single-Pass Any-Resolution ViT for Open-vocabulary Segmentation
Apr 2, 2026Naomi Kombol, Ivan Martinović, Siniša Šegvić +1Vision Transformer Efficient ViTs
Faculty of Electrical Engineering and Computing, University of Zagreb · VRG, Faculty of Electrical Engineering, Czech Technical University in Prague
Decouple and Rectify: Semantics-Preserving Structural Enhancement for Open-Vocabulary Remote Sensing Segmentation
Apr 2, 2026Jie Feng, Fengze Li, Junpeng Zhang +4Remote Sensing Image Segmentation Open-Vocabulary Semantic Segmentation
Xidian University China · Jimei University China · South China University of Technology China
ConInfer: Context-Aware Inference for Training-Free Open-Vocabulary Remote Sensing Segmentation
Mar 31, 2026Wenyang Chen, Zhanxuan Hu, Yaping Zhang +2Image Segmentation Remote Sensing Image Segmentation
Yunnan Normal University · Xi’an University of Posts & Telecommunications
GeoGuide: Hierarchical Geometric Guidance for Open-Vocabulary 3D Semantic Segmentation
Mar 27, 2026Xujing Tao, Chuxin Wang, Yubo Ai +8Open-Vocabulary 3D Segmentation 3D Semantic Segmentation
University of Science and Technology of China · National Key Laboratory of Deep Space Exploration, Deep Space Exploration Laboratory
Looking Beyond the Window: Global-Local Aligned CLIP for Training-free Open-Vocabulary Semantic Segmentation
Mar 24, 2026ByeongCheol Lee, Hyun Seok Seong, Sangeek Hyun +3Open-Vocabulary Semantic Segmentation Sliding-Window Attention
Sungkyunkwan University
PEARL: Geometry Aligns Semantics for Training-Free Open-Vocabulary Semantic Segmentation
Mar 23, 2026Gensheng Pei, Xiruo Jiang, Xinhao Cai +3Graph-Based Label Propagation Open-Vocabulary Semantic Segmentation
Department of Electrical and Computer Engineering, Sungkyunkwan University · School of Computing and Artificial Intelligence, Southwest Jiaotong University · School of Computer Science and Engineering, Nanjing University of Science and Technology
Mitigating Objectness Bias and Region-to-Text Misalignment for Open-Vocabulary Panoptic Segmentation
Mar 22, 2026Nikolay Kormushev, Josip Šarić, Matej KristanPanoptic Segmentation Vision-Language Alignment
University of Ljubljana Faculty of Comp. and Inf. Science · ETH Zurich Dept. of Computer Science · University of Zagreb Faculty of Elec. Eng. and Computing
MagicSeg: Open-World Segmentation Pretraining via Counterfactural Diffusion-Based Auto-Generation
Mar 20, 2026Kaixin Cai, Pengzhen Ren, Jianhua Han +4Semantic Segmentation Open-Vocabulary Semantic Segmentation
Shenzhen Campus of Sun Yat-sen University, Shenzhen, China · PengCheng Laboratory · Huawei Noah’s Ark Lab
dinov3.seg: Open-Vocabulary Semantic Segmentation with DINOv3
Mar 19, 2026Saikat Dutta, Biplab Banerjee, Hamid RezatofighiOpen-Vocabulary Semantic Segmentation Vision-Language Models
IITB-Monash Research Academy · IIT Bombay · Monash University
MM-OVSeg:Multimodal Optical-SAR Fusion for Open-Vocabulary Segmentation in Remote Sensing
Mar 18, 2026Yimin Wei, Aoran Xiao, Hongruixuan Chen +2Remote Sensing Image Segmentation Open-Vocabulary Semantic Segmentation
The University of Tokyo · RIKEN AIP
PCA-Seg: Revisiting Cost Aggregation for Open-Vocabulary Semantic and Part Segmentation
Mar 18, 2026Jianjian Yin, Tao Chen, Yi Chen +4Open-Vocabulary Semantic Segmentation Vision-Language Alignment
Nanjing University of Science and Technology · State Key Laboratory of Intelligent Manufacturing of Advanced Construction Machinery · Nanjing Normal University +2
DCP-CLIP:A Coarse-to-Fine Framework for Open-Vocabulary Semantic Segmentation with Dual Interaction
Mar 14, 2026Jing Wang, Huimin Shi, Quan Zhou +3Open-Vocabulary Semantic Segmentation Text-Guided Image Segmentation
National Engineering Research Center of Communications and Networking, Nanjing University of Posts & Telecommunications, Nanjing 210003, P. R. China. · Institute for Advanced Ocean Research (Nantong), Southeast University, Nantong 226334, P. R. China. · Department of Internet of Things, Nanjing University of Posts & Telecommunications, Nanjing 210003, P. R. China +1
Making Training-Free Diffusion Segmentors Scale with the Generative Power
Mar 6, 2026Benyuan Meng, Qianqian Xu, Zitai Wang +3Text-Guided Image Segmentation Open-Vocabulary Semantic Segmentation
Institute of Information Engineering, CAS · School of Cyber Security, University of Chinese Academy of Sciences · State Key Laboratory of AI Safety, Institute of Computing Technology, CAS +5
JOPP-3D: Joint Open Vocabulary Semantic Segmentation on Point Clouds and Panoramas
Mar 6, 2026Sandeep Inuganti, Hideaki Kanayama, Kanta Shimizu +43D Point Cloud Segmentation Open-Vocabulary 3D Segmentation
German Research Center for Artificial Intelligence, DFKI, Germany · RPTU Kaiserslautern, Germany · Ricoh Company, Ltd. Japan +1
Benchmarking Semantic Segmentation Models via Appearance and Geometry Attribute Editing
Mar 2, 2026Zijin Yin, Bing Li, Kongming Liang +4Semantic Segmentation Open-Vocabulary Semantic Segmentation
School of Artificial Intelligence, Beijing University of Posts and Telecommunications, and Beijing Key Laboratory of Multimodal Data Intelligent Perception and Governance, Beijing 100876, China. · Visual Computing Center, King Abdullah University of Science and Technology, Jeddah, Saudi Arabia. · China Telecom Artificial Intelligence Technology Co. Ltd, Beijing 100034, China.
Open-Vocabulary vs Supervised Learning Methods for Post-Disaster Visual Scene Understanding
Mar 1, 2026Anna Michailidou, Georgios Angelidis, Vasileios Argyriou +2Object Detection Open-Vocabulary Object Detection
Department of Informatics and Telematics, Harokopio University of Athens, Athens, Greece · Department of Networks and Digital Media, Kingston University, London, United Kingdom · Department of Electrical and Computer Engineering, University of Western Macedonia, Kozani, Greece +1
Open-Vocabulary Semantic Segmentation in Remote Sensing via Hierarchical Attention Masking and Model Composition
Feb 27, 2026Mohammadreza Heidarianbaei, Mareike Dorozynski, Hubert Kanyamahanga +2Open-Vocabulary Semantic Segmentation Remote Sensing Image Segmentation
Institute of Photogrammetry and GeoInformation Leibniz University Hannover Germany
Retrieve and Segment: Are a Few Examples Enough to Bridge the Supervision Gap in Open-Vocabulary Segmentation?
Feb 26, 2026Tilemachos Aravanis, Vladan Stojnić, Bill Psomas +2Open-Vocabulary Semantic Segmentation Test-Time Adaptation of VLMs
VRG, FEE, Czech Technical University in Prague · University of Crete · Archimedes, Athena RC +1
Open-Vocabulary Domain Generalization in Urban-Scene Segmentation
Feb 21, 2026Dong Zhao, Qi Zang, Nan Pu +3Domain Generalization Open-Vocabulary Semantic Segmentation
Department of Information Engineering and Computer Science, University of Trento, Italy · School of Computer Science and Information Engineering, Hefei University of Technology, China
Enabling Training-Free Text-Based Remote Sensing Segmentation
Feb 19, 2026Jose Sosa, Danila Rukhovich, Anis Kacem +1Text-Guided Image Segmentation Remote Sensing Image Segmentation
SnT, University of Luxembourg
Exploring the Underwater World Segmentation without Extra Training
Nov 11, 2025Bingyu Li, Tao Huo, Da Zhang +3Image Segmentation Open-Vocabulary Semantic Segmentation
Institute of Artificial Intelligence (TeleAI), China Telecom, China · University of Science and Technology of China, China · Northwestern Polytechnical University, China
Generalized Decoupled Learning for Enhancing Open-Vocabulary Dense Perception
Aug 15, 2025Junjie Wang, Keyu Chen, Yulin Li +4Open-Vocabulary Object Detection Open-Vocabulary Semantic Segmentation
Harbin Institute of Technology (Shenzhen) · University of Hong Kong
SynSeg: Feature Synergy for Multi-Category Contrastive Learning in End-to-End Open-Vocabulary Semantic Segmentation
Aug 8, 2025Weichen Zhang, Kebin Liu, Fan Dang +3Contrastive Learning Weakly Supervised Image Segmentation
Tsinghua University, 100084, Beijing, China · Beijing Jiaotong University, 100044, Beijing, China
LlamaSeg: Image Segmentation via Autoregressive Mask Generation
May 26, 2025Jiru Deng, Tengjin Weng, Tianyu Yang +3Image Segmentation Text-Guided Image Segmentation
Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) · Tsinghua University · Shenzhen University +2
Synergistic Vision-Language Reinforcement Enables Scalable On-Demand Analysis across Diverse Clinical Tasks
May 6, 2025Haonan Wang, Jiaji Mao, Lehan Wang +11Image Segmentation Medical Imaging Foundation Models
Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology, Hong Kong SAR · Department of Radiology, Guangdong Provincial Key Laboratory of Malignant Tumor Epigenetics and Gene Regulation, Sun Yat-Sen Memorial Hospital, Sun Yat-Sen University, Guangzhou 510120, China · Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong SAR

Apr 9, 2026J/K move · Enter open · S save

OV-Stitcher: A Global Context-Aware Framework for Training-Free Open-Vocabulary Semantic Segmentation

Seungjae Moon, Seunghyun Oh, Youngmin Ro

Machine Intelligence Laboratory, University of Seoul, Korea

Open-Vocabulary Semantic Segmentation

Open paper AskPDF arXiv

Training-free open-vocabulary semantic segmentation(TF-OVSS) has recently attracted attention for its ability to perform dense prediction by leveraging the pretrained knowledge of large vision and vision-language models, without requiring additional training. However, due to the limited input resolution of these pretrained encoders, existing TF-OVSS methods commonly adopt a sliding-window strategy that processes cropped sub-images independently. While effective for managing high-resolution inputs, this approach prevents global attention over the full image, leading to fragmented feature representations and limited contextual reasoning. We propose OV-Stitcher, a training-free framework that addresses this limitation by stitching fragmented sub-image features directly within the final encoder block. By reconstructing attention representations from fragmented sub-image features, OV-Stitcher enables global attention within the final encoder block, producing coherent context aggregation and spatially consistent, semantically aligned segmentation maps. Extensive evaluations across eight benchmarks demonstrate that OV-Stitcher establishes a scalable and effective solution for open-vocabulary segmentation, achieving a notable improvement in mean Intersection over Union(mIoU) from 48.7 to 50.7 compared with prior training-free baselines.