Semantic Mismapping in Generalized Category Discovery: Evaluation and Mitigation

Zitai Wang; Qianqian Xu; Cong Hua; Xi Chen; Zhiyong Yang; Xiaochun Cao; Qingming Huang

cs.CVOct 29, 2026

Semantic Mismapping in Generalized Category Discovery: Evaluation and Mitigation

Authors: Zitai Wang, Qianqian Xu, Cong Hua, Xi Chen, Zhiyong Yang, Xiaochun Cao, Qingming Huang

Organizations: State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China · Beijing Academy of Artificial Intelligence (BAAI), Beijing 100083, China · School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing 101408, China · School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing 100049, China ·

Abstract

Generalized Category Discovery (GCD) aims to perform old-category recognition and novel-category discovery in the same unlabeled set. Early methods address both objectives through clustering and evaluate the exchangeable predicted indices with global Hungarian (GH) matching. Recent methods adopt trainable parametric heads, using supervision for old-category recognition and discovery-oriented objectives for new categories. In this way, the output space becomes partially exchangeable: old indices are semantically anchored, while new indices remain exchangeable. However, these methods still apply global matching, which can reassign anchored old indices to other categories and hide some prediction errors. Empirically, we are surprised to observe that this reassignment can induce an overestimation of new-category accuracy by up to 25.63 points. We term this phenomenon semantic mismapping and further derive a permutation-cycle condition that characterizes when semantic mismapping can increase the global matching score. To fix this issue, we propose Semantics-Preserving Evaluation (SPE), which preserves the supervised semantics of old indices and restricts Hungarian matching to the new region. Furthermore, we propose Mismapping-Mitigating Optimization (MMO) to alleviate the prediction errors that can give rise to semantic mismapping, combining a region-balance loss (RBL) for old/new routing with a novel-diversity loss (NDL) that encourages orthogonality among new prototypes. Theoretically, we show that minimizing the two losses can tighten an upper bound on spurious accuracy gains from semantic mismapping, with controllable residual terms. Empirical results under both GH and SPE demonstrate the effectiveness of our proposed method.

Figures & tables

Fig. 1: (a) When evaluating parametric GCD methods, global matching could reassign an old output to a new category ( e.g. , $o_{1}\to n_{1}$ ), making a false-old prediction appear correct. (b) Our SPE protocol preserves the semantics of old outputs and performs local matching within the new region. (c) Our mismapping-mitigating optimization (MMO) combines region-balance loss (RBL), which matches old/new prediction mass to estimated sample proportions, with novel-diversity loss (NDL), which encourages orthogonality among new prototypes.

Fig. 2: Comparison of GH and SPE using the same predictions. (a) The raw confusion matrix has predicted outputs as rows and ground-truth categories as columns, with old and new categories denoted by

o

and

n

. (b) GH reassigns the anchored old output

\hat{o}_{1}

n_{1}

and the new output

\hat{n}_{1}

o_{1}

, counting 46 false-old predictions and 44 old-to-new errors as correct. (c) SPE preserves the semantics of old outputs and applies local matching within the new region. In this way, predictions in either cross-region block remain errors, including the 46 and 44 samples hidden by GH.

Protocol	Matching rule	Each output assigned only once?	Within-old semantics preserved?	Cross-region errors detected?
Legacy NCD	Repeated local matching	✗	✗	✗
Global Hungarian (GH)	Single global matching	✓	✗	✗
SPE	Single local matching	✓	✓	✓

TABLE I: Comparison of existing GCD evaluation protocols. The first property indicates whether a predicted output can be repeatedly assigned to different categories. The latter two properties correspond to the prevention of Type (I) and Type (II) semantic mismapping, respectively.

Panel of Table 4 — TABLE III: Comparison between GH and SPE across benchmark families for seven method–backbone configurations. Gray shading highlights configurations whose ranks differ between the two protocols. The score gaps $\Delta_{r},r\in\{\mathrm{new},\mathrm{all}\}$ and rank changes reveal GH-induced performance overestimation, as the answer to RQ1 .

Dataset (Backbone)	$\Delta_{\mathrm{all}}$	$N_{u}$	Total cycle margin $\sum_{\gamma\in\Gamma(\mathcal{A}_{\mathrm{GH}},\mathcal{A}_{0})}m_{\gamma}$	Old $\rightarrow$ wrong-old mismapping (Type I)	Old $\rightarrow$ new mismapping (Type II)
CIFAR-10 (DINO)	0.00 (Zero)	37,500	$0$	0	0
CUB (CLIP)	0.22 (Low)	4,496	$-10$	1	3
Stanford Cars (CLIP)	1.43 (Medium)	6,144	$-88$	0	6
Herbarium 19 (CLIP)	12.13 (High)	25,356	$-3{,}076$	4	119

TABLE IV: Empirical validation of the cycle-margin analysis in Thm. 1 using SimGCD. The summed cycle margins

\sum_{\gamma\in\Gamma(\mathcal{A}_{\mathrm{GH}},\mathcal{A}_{0})}m_{\gamma}

exactly recover the observed accuracy gaps

\Delta_{\mathrm{all}}

through Eq.( 27 ). The last two columns report the numbers of old outputs involved in Type (I) within-old and Type (II) cross-boundary semantic mismapping, respectively, as defined in Rem. 1 .

Dataset	Method	Backbone	GH				SPE
Dataset	Method	Backbone	$\texttt{All}_{\mathrm{GH}}$	$\texttt{Old}_{\mathrm{GH}}$	$\texttt{New}_{\mathrm{GH}}$	$\texttt{H}_{\mathrm{GH}}$	$\texttt{All}_{\mathrm{SPE}}$	$\texttt{Old}_{\mathrm{SPE}}$	$\texttt{New}_{\mathrm{SPE}}$	$\texttt{H}_{\mathrm{SPE}}$
	LegoGCD	DINO	62.19	71.18	57.69	63.73	61.50	73.05	55.72	63.22
	+ Ours	DINO	63.88	74.85	58.39 (+0.70)	65.60 (+1.87)	63.63	74.72	58.09 (+2.37)	65.36 (+2.14)
	SimGCD-BIA	DINO	61.61	60.84	62.00	61.41	60.90	60.91	60.89	60.90
	+ Ours	DINO	61.81	62.17	61.63 (-0.37)	61.90 (+0.49)	61.17	62.91	60.29 (-0.60)	61.57 (+0.67)
	ProtoGCD	DINO	60.97	64.64	59.13	61.76	60.32	64.78	58.09	61.25

TABLE V: Comprehensive comparison on four fine-grained benchmarks under GH and SPE. Adding MMO ( i.e. , +Ours ) helps improve new-category recognition and old–new balance under both GH and SPE, with particularly large gains on Herbarium 19. Moreover, MMO yields larger average gains under SPE than under GH, indicating better alignment with our semantics-preserving evaluation, as the answer to RQ2 .

Panel of Figure 7 — Fig. 3: Old–new accuracy trade-offs for the CLIP-based methods in Tab. V . The top and bottom rows report GH and SPE, respectively; columns show CUB, FGVC-Aircraft, Herbarium 19, and Stanford Cars. TAR denotes GET-TAR, and +Ours denotes the corresponding method with MMO.

Panel of Figure 8 — Fig. 4: Sensitivity to the labeling fraction on the imbalanced Herbarium 19 benchmark with SimGCD–CLIP. From left to right, the panels show All, Old, New, and harmonic-mean ( $H$ ) accuracy under GH and SPE. Baseline denotes the unmodified result, the middle points use manually specified $\widehat{\rho}$ , and Auto uses the TIcE estimate. Complete numerical results are provided in App. Tab. XIII .

Panel of Figure 9 — Fig. 5: Joint sensitivity of SimGCD–DINO on CUB to the balancing coefficient $\lambda_{\mathrm{RB}}$ and the NDL coefficient $\lambda_{\mathrm{ND}}$ . The top and bottom rows report GH and SPE, respectively; the columns from left to right show All, Old, and New accuracy.

Method	Backbone	$\texttt{All}_{\mathrm{GH}}$	$\texttt{Old}_{\mathrm{GH}}$	$\texttt{New}_{\mathrm{GH}}$	$\texttt{All}_{\mathrm{SPE}}$	$\texttt{Old}_{\mathrm{SPE}}$	$\texttt{New}_{\mathrm{SPE}}$
SimGCD	DINO	57.04	77.14	46.94	54.81	85.15	39.58
+ Ours	DINO	58.69	77.29	49.34	56.88	85.28	42.60
SimGCD	CLIP	57.99	77.06	48.41	55.98	85.26	41.27
+ Ours	CLIP	59.03	77.63	49.68	57.35	85.10	43.40

TABLE VI: Large-scale comparison on ImageNet-1K. Adding MMO improves both new-category and overall accuracy under GH and SPE with DINO and CLIP backbones, demonstrating its effectiveness on the large-scale benchmark, as the answer to RQ2 .

Fig. 6: t-SNE visualization of ten selected new categories from Herbarium 19 for (a) GET, (b) GET +Ours , (c) GET-TAR, and (d) GET-TAR +Ours . Colored circles denote samples, and stars denote the corresponding class prototypes. Red dashed ellipses highlight representative local structures, while arrows connect the baseline and +Ours views for visual comparison. The complete visualizations and the definitions of the high-dimensional intra- and inter-class statistics are provided in App. C-G .

Dataset	Method	Estimator	$C_{\mathrm{new}}$	$\widehat{C}_{\mathrm{new}}$	$\texttt{New}_{\mathrm{GH}}$	$\texttt{New}_{\mathrm{SPE}}$	$\texttt{Old}_{\mathrm{SPE}}$	$\texttt{All}_{\mathrm{SPE}}$	$\Delta_{\mathrm{new}}$	$\Delta_{\mathrm{all}}$
	SimGCD	DINO	100	131	60.19	60.56	63.11	61.41	-0.37	0.27
	+ Ours	DINO	100	131	60.46 (+0.27)	60.99 (+0.43)	66.64	62.88	-0.53	0.15
	SimGCD	CLIP	100	112	67.60	67.23	76.32	70.26	0.37	0.13
CUB	+ Ours	CLIP	100	112	68.57 (+0.97)	67.27 (+0.04)	74.92	69.82	1.30	0.71
	SimGCD	DINO	98	132	44.53	42.60	72.91	52.47	1.93	0.41
	+ Ours	DINO	98	132	46.68 (+2.15)	45.84 (+3.24)	68.97	53.37	0.84	0.28

TABLE VII: Evaluation of SPE with an estimated new-category number following Vaze et al. [ 44 ] . SPE remains applicable when the estimated category number differs from the ground truth and reveals GH–SPE score differences on CUB and Stanford Cars, as the answer to RQ3 .

Dataset	Cali. Acc.	Old			New			All
Dataset	Cali. Acc.	$\texttt{Old}_{\mathrm{GH}}$	$\texttt{Old}_{\mathrm{SPE}}$	$\Delta_{\mathrm{old}}$	$\texttt{New}_{\mathrm{GH}}$	$\texttt{New}_{\mathrm{SPE}}$	$\Delta_{\mathrm{new}}$	$\texttt{All}_{\mathrm{GH}}$	$\texttt{All}_{\mathrm{SPE}}$	$\Delta_{\mathrm{all}}$
CIFAR-100	94.77	81.04	82.15	-1.11	68.20	65.45	2.75	76.76	76.58	0.18
CUB	99.93	67.71	67.58	0.13	74.04	73.74	0.30	71.93	71.69	0.24
Stanford Cars	99.75	75.21	77.26	-2.05	50.88	48.73	2.15	58.81	58.02	0.78
FGVC-Aircraft	99.70	65.01	65.97	-0.96	57.00	56.34	0.66	59.67	59.55	0.12
Herbarium 19	82.93	45.08	45.22	-0.14	30.10	28.17	1.93	35.34	34.13	1.21

TABLE VIII: Evaluation of the non-parametric SPE extension using SelEx [ 38 ] equipped with Bures-Isotropy Alignment (BIA) [ 42 ] . Labeled-set calibration enables SPE to evaluate non-parametric predictions and reveals GH–SPE score differences across the five benchmarks, as the answer to RQ3 .

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 21 — Fig. 7: Old–new trade-offs under GH for CLIP-based methods. The axes show old-category (horizontal) and new-category (vertical) accuracy (%). TAR denotes GET-TAR.

Panel of Figure 22 — Fig. 8: Old–new trade-offs under SPE for CLIP-based methods. The axes show old-category (horizontal) and new-category (vertical) accuracy (%). TAR denotes GET-TAR.

Panel of Figure 23 — Fig. 9: New-category versus overall accuracy under GH for CLIP-based methods. The axes show new-category (horizontal) and overall (vertical) accuracy (%). TAR denotes GET-TAR.

Panel of Figure 24 — Fig. 10: New-category versus overall accuracy under SPE for CLIP-based methods. The axes show new-category (horizontal) and overall (vertical) accuracy (%). TAR denotes GET-TAR.

Dataset	Setting	RBL	NDL	GH				SPE
Dataset	Setting	RBL	NDL	$\texttt{All}_{\mathrm{GH}}$	$\texttt{Old}_{\mathrm{GH}}$	$\texttt{New}_{\mathrm{GH}}$	$\texttt{H}_{\mathrm{GH}}$	$\texttt{All}_{\mathrm{SPE}}$	$\texttt{Old}_{\mathrm{SPE}}$	$\texttt{New}_{\mathrm{SPE}}$	$\texttt{H}_{\mathrm{SPE}}$
CUB	Baseline	–	–	68.53	75.38	65.10	69.86	68.31	76.12	64.40	69.77
	+RBL	✓	–	70.55	76.38	67.63	71.74	70.40	75.85	67.67	71.53
	+RBL+NDL	✓	✓	72.69	76.72	70.67	73.57	72.26	75.78	70.50	73.04
FGVC-Aircraft	Baseline	–	–	55.59	59.54	53.61	56.42	55.47	59.54	53.43	56.32
	+RBL	✓	–	57.79	63.99	54.69	58.98	57.57	65.07	53.82	58.91

Appendix

TABLE XIV: Component ablation of MMO with SimGCD–CLIP on three fine-grained benchmarks. Orange New/

H

cells indicate gains from the baseline. Best/second-best scores are bold/underlined.

Panel of Figure 26 — Fig. 11: Comparative t-SNE visualization on Herbarium 19 for SimGCD (left) and SimGCD +Ours (right). Colors identify the same ten selected categories, circles denote samples, and stars denote class prototypes.

Generalized Category Discovery

Generalized category discovery learns known classes from labeled examples while grouping unlabeled data into known and previously unseen categories.

See all 27 papers

CoGe-GCD: Reframing Generalized Category Discovery with Compositional Generalization
Sep 9, 2026Luyao Tang, Jiewei Zheng, Kunze Huang +3Geometric Representation Learning Classification
The University of Hong Kong, Hong Kong SAR, China · Xiamen University, Xiamen, China · Shenzhen University, Shenzhen, China.
Relational Retrieval: Leveraging Known-Novel Interactions for Generalized Category Discovery
May 10, 2026Yulin Xu, Chunqi Guo, Yuanzhen Shuai +1Semi-Supervised Learning Generalized Category Discovery
University College London London, UK · Juniata College Huntingdon, USA
Multi-Modal Representation Learning via Semi-Supervised Rate Reduction for Generalized Category Discovery
Feb 23, 2026Wei He, Xianghan Meng, Zhiyuan Huang +3Generalized Category Discovery Cross-Modal Representation Learning
School of Artificial Intelligence, Beijing University of Posts and Telecommunications · Intellifusion Inc., Shenzhen, P.R. China

Sep 9, 2026J/K move · Enter open · S save

CoGe-GCD: Reframing Generalized Category Discovery with Compositional Generalization

Luyao Tang, Jiewei Zheng, Kunze Huang, Chaoqi Chen+2

The University of Hong Kong, Hong Kong SAR, China · Xiamen University, Xiamen, China · Shenzhen University, Shenzhen, China.

Geometric Representation Learning Classification Compositional Generalization Generalized Category Discovery

Open paper AskPDF arXiv

Generalized Category Discovery (GCD) assigns unlabeled instances, mixed with labeled data, to known or novel categories, requiring human-like compositional reasoning: reusing primitives learned from known classes and deciding when new combinations imply new categories. Existing GCD methods operate on unstructured token features and struggle to extrapolate to novel compositions. We propose CoGe-GCD, which rethinks GCD through compositional generalization with two coupled stages. (i) Compositional Perception structures patch tokens by mapping them to a small vocabulary of primitives and refining token embeddings via competitive token-primitive assignment and information passing, yielding coherent groups for discovery. (ii) Generalizing Induction exploits the induced geometric structure and applies a structure-preserving calibration over spatial relations, maintaining probabilistic semantics while improving extrapolation to unseen primitive combinations. CoGe-GCD is implemented as an inductive-bias module between backbone and projection head, without modifying heads or losses, and can be plugged into diverse GCD frameworks. On standard benchmarks, it consistently improves all-class accuracy, unknown-class number estimation, and geometric quality, with marginal computational overhead. Code is available at https://github.com/lytang63/CoGe-GCD.

Abstract

Generalized Category Discovery

Explore similar work

CoGe-GCD: Reframing Generalized Category Discovery with Compositional Generalization

Relational Retrieval: Leveraging Known-Novel Interactions for Generalized Category Discovery

Multi-Modal Representation Learning via Semi-Supervised Rate Reduction for Generalized Category Discovery