cs.CVNov 9, 2026

Text Alone Predicts Which Classes a Segmentation Readout Keeps

Authors: David Yan

Organizations: Southeast University

Abstract

An open-vocabulary segmentation readout silently loses a class deployed beyond its training categories. It still emits masks with high confidence, but the class never wins a single pixel. The training loss falls anyway, and mIoU can even rise while classes die. We show that which classes collapse is written in the class-name list alone. Before any image is seen, we rank each class's collapse risk from its text similarity to the training vocabulary alone. The ranking holds at AUROC 0.69-0.96 across nine model families, twelve backbones and eight pretraining recipes. Two further families, measured only after the predictions were written down, land in the same band. What the list writes, it can rewrite. With identical images and a class's counterpart moved in or out of the training list, retraining flips that class's gain over the frozen baseline by 21 IoU points; swapping in unrelated names changes nothing. We find a vocabulary-distance mechanism behind the collapse. Collapsed classes lose every vote to near-name neighbors, but in the frozen features they survive. Turning down the readout's amplitude, or stopping training early, brings them back, up to +16 points, without retraining; trained with its amplitude held down, the readout never collapses.

Figures & tables

Explore similar work

CardsList
  1. Calibrate Once, Grow the Vocabulary: Candidate-Independent Scores for Conformal Risk Control in Open-Vocabulary Segmentation

    Nov 16, 2026Md Tanveer Hossain Munim, Bijoy Ahmed Saiem, Muhammad Abdullah AdnanConformal Risk ControlOpen-Vocabulary Semantic Segmentation

  2. SegRAG: Retrieval Augmented Spatial Prompting for Open Vocabulary Semantic Segmentation

    May 17, 2026Abderrahmene Boudiaf, Irfan Hussain, Sajid JavedImage SegmentationPromptable Image Segmentation