PRISM: Primitive-Integrated Separable Modulation with Priority Visual Memory Queue for Compositional Zero-Shot Learning
Authors: Jianhua Wang, Zhihang Zheng, Yang Liu, Jiale Du
Organizations: School of Electronic Engineering, Xidian University, Shaanxi 710071, China · School of Computer Science and Technology, Xidian University, Shaanxi 710071, China · School of Telecommunications Engineering, Xidian University, Shaanxi 710071, China
Compositional Zero-Shot Learning (CZSL) recognizes unseen attribute--object pairs by recombining primitives from seen compositions. Existing methods stop sharing evidence at deployment: fixed prototypes cannot exploit unlabeled test streams, while composition-wise refinement cannot propagate corrections across pairs sharing a primitive. We introduce PRISM, an online CZSL framework that extends primitive sharing to test-time adaptation. Its Hierarchical Knowledge Accumulation Module (HiKAM) factorizes prototype migration into composition-, attribute-, and object-level residual banks. A Priority Visual Memory Queue (PVMQ), initialized by Prototype Queue Initialization (PQI), grounds text updates with bounded visual evidence. The same adaptive core supports exhaustive Full PRISM and recall-calibrated Adaptive PRISM-Lite, which selects candidate budgets from held-out ranks. Comprehensive closed- and open-world experiments on UT-Zappos, MIT-States, and C-GQA demonstrate the state of the art, with Full PRISM achieving the best open-world AUC and harmonic mean on all three benchmarks and the best closed-world results on UT-Zappos and C-GQA. Fine-grained long-tail and non-stationary primitive-block evaluations reveal gains on under-supported concepts and stable adaptation under prolonged skew. Adaptive PRISM-Lite further delivers substantial efficiency gains in the full open-world space with only a small accuracy cost. PRISM thus combines primitive-shared online adaptation, multimodal grounding, and candidate-aware inference.
Figures & tables
Figure 1: Test-time adaptation under label-space shift, showing primitive-shared textual updates and Priority Visual Memory Queue retrieval from an unlabeled test stream.
Figure 2: Overall architecture of PRISM at test time, showing primitive-shared textual and visual updates to composition prototypes.
Methods
UT-Zappos
MIT-States
C-GQA
AUC
HM
Seen
Unseen
AUC
HM
Seen
Unseen
AUC
HM
Seen
Unseen
DFSP ( Lu et al. 2023 ) CVPR’23
36.0
47.2
66.7
71.7
20.6
37.3
46.9
52.0
10.5
27.1
38.2
32.0
Troika ( Huang et al. 2024 ) CVPR’24
41.7
54.6
66.8
73.8
22.1
39.3
49.0
53.0
12.4
29.4
41.0
35.7
PLID ( Bao et al. 2024 ) ECCV’24
38.7
52.4
67.3
68.8
22.1
39.0
49.7
52.4
11.0
27.9
38.8
33.0
IMAX ( Jiang et al. 2025 ) TPAMI’25
40.6
54.2
69.3
70.7
21.9
39.1
48.7
53.8
12.8
29.8
39.7
35.8
ClusPro ( Qu et al. 2025 ) ICLR’25
46.6
58.5
70.7
76.0
23.8
40.7
52.1
54.0
14.9
32.8
44.3
37.8
Table 1: Closed-world task-level comparison of complete CZSL systems on UT-Zappos, MIT-States, and C-GQA. The best and second-best AUC/HM values are bold and underlined, respectively.
Variant
HiKAM
Prim.
PVMQ
PQI
UT-Zappos
MIT-States
C-GQA
AUC
HM
Seen
Unseen
AUC
HM
Seen
Unseen
AUC
HM
Seen
Unseen
w/o TTA
43.6
55.5
68.7
74.3
22.2
39.3
49.1
52.9
15.2
33.1
43.9
39.6
Text TTA only
✓
38.4
52.6
66.8
66.4
22.2
39.2
49.1
53.0
15.2
33.1
44.7
38.7
w/o PVMQ
✓
✓
44.5
57.3
71.3
71.2
22.2
39.1
49.0
53.0
15.6
33.4
45.3
38.9
w/o PQI
✓
✓
✓
48.9
60.5
75.4
72.6
22.3
39.0
49.3
53.3
15.6
33.4
45.2
39.2
Full PRISM
✓
✓
✓
✓
53.7
64.8
79.0
73.8
23.3
40.2
50.7
53.7
16.7
35.4
45.4
41.1
Table 3: Closed-world ablation study on UT-Zappos, MIT-States, and C-GQA.
Setting
Mode
Candidates
Recall (%)
Img/s
Mean (ms)
AUC
HM
Closed
Full PRISM
7,767
100.00
2.35
423.86
16.69
35.41
Fixed Lite (100)
100
90.92
15.32
64.48
15.67
34.16
Fixed Lite (200)
200
95.21
14.83
66.63
15.65
34.04
Adaptive PRISM-Lite
148/152.3
93.55
13.11
76.28
15.93
34.41
Open
Full PRISM
278,362
100.00
1.84
543.89
5.45
17.58
Fixed Lite (100)
100
71.81
10.63
94.09
5.10
16.42
Table 4: C-GQA closed- and open-world profiling; Adaptive candidate counts are calibrated/realized.
Analysis
Group
#Comp.
#Images
Closed-world
Open-world
No TTA
PRISM
Δ
No TTA
PRISM
Δ
c
All
36
2914
24.3
34.6
+10.3
24.1
33.1
+8.9
Tail
11
50
13.6
32.1
+18.5
13.6
29.7
+16.1
Body
14
546
19.2
20.9
+1.8
18.7
19.9
+1.2
Head
11
2318
41.6
54.5
+12.9
41.5
53.2
+11.7
a
Tail
8
152
26.1
37.1
+11.0
26.1
37.3
+11.2
Table 5: Long-tail analysis on UT-Zappos under closed- and open-world protocols.
Figure 3: Closed-world cumulative top-1 accuracy with increasing test sample size on three datasets.
Figure 4: Closed-world sensitivity analysis of α .
Figure 5: Closed-world sensitivity analysis of Priority Visual Memory Queue capacity K .
Zero-shot learning lets models recognize classes or solve tasks absent from task-specific training examples by using semantic descriptions or pretrained knowledge.
Compositional Zero-Shot Learning (CZSL) aims to recognize novel attribute-object compositions based on the knowledge learned from seen ones. Existing methods suffer from performance degradation caused by the distribution shift of label space at test time, which stems from the inclusion of unseen compositions recombined from attributes and objects. To overcome the challenge, we propose a novel approach that accumulates comprehensive knowledge in both textual and visual modalities from unsupervised data to update multimodal prototypes at test time. Building on this, we further design an adaptive update weight to control the degree of prototype adjustment, enabling the model to flexibly adapt to distribution shift during testing. Moreover, a dynamic priority queue is introduced that stores high-confidence images to acquire visual prototypes from historical images for inference. Since the model tends to favor compositions already stored in the queue during testing, we warm-start the queue by initializing it with training images for visual prototypes of seen compositions and generating unseen visual prototypes using the mapping learned between seen and unseen textual prototypes. Considering the semantic consistency of multimodal knowledge, we align textual and visual prototypes by multimodal collaborative representation learning. To provide a more reliable evaluation for CZSL, we introduce a new benchmark dataset, C-Fashion, and refine the widely used but noisy MIT-States dataset. Extensive experiments indicate that our approach achieves state-of-the-art performance on four benchmark datasets under both closed-world and open-world settings. The source code and datasets are available at https://github.com/xud-yan/WARM-CAT .
Xudong Yan, Songhe Feng, Jiaxin Wang +2
School of Computer Science and Technology, Beijing Jiaotong University, Beijing 100044, China
Compositional Zero-Shot Learning (CZSL) aims to combine known attributes and objects as primitives for recognizing previously unseen attribute-object pairs. Prior works either predict attributes and objects independently, missing their strong contextual dependency, or use unidirectional conditional modeling (e.g., object-guided attribute prediction), which is prone to error propagation. We propose PRPC, a Progressive Reasoning framework with Primitive Correction, which explicitly models the bidirectional dependency between attributes and objects via step-wise inference. PRPC performs mutual correction of primitives to suppress prediction errors in earlier steps. Specifically, we formulate CZSL as structured, Q&A-style Chain-of-Thought reasoning process and constrain the MLLM to follow predefined semantic steps to generate intermediate decisions. To further enhance the reliability and logical consistency of intermediate reasoning, we introduce reinforcement learning post-training with a GRPO-based objective, providing step-level rewards aligned with the progressive inference procedure. Extensive experiments on three CZSL benchmarks demonstrate that PRPC achieves state-of-the-art performance, validating the effectiveness of progressive reasoning and bidirectional correction for robust compositional generalization.
Ziyi Chen, Haoyan Shi, Sunhan Xu +1
School of Computer Science and Technology, Beijing Jiaotong University · Key Laboratory of Big Data & Artificial Intelligence in Transportation (Ministry of Education)
Compositional Zero Shot Learning aims to recognize unseen compositions by recombining learned primitives. Recent methods rely on vision language models and attempt to explicitly model contextual variations of primitives through multiple representations. However, such approaches are limited by fixed variant capacity and competition between abstract and concrete semantics. In this work, we present a new perspective that views primitive variations as the context-driven activation of concrete visual cues rather than independent entities. Based on it, we propose CLEAR, a CLoze-style rEAsoning-based Re-ranking framework inspired by human perceptual processes. CLEAR extracts conditional variants from the primitive candidate set in a coarse-to-fine manner, performs cloze-style reasoning to infer high-level semantics, and re-ranks predictions to correct biases toward salient concrete primitives. Extensive experiments demonstrate that CLEAR consistently improves the Base Model and outperforms state-of-the-art methods on the challenging C-GQA and MIT-States datasets. Code is available at https://github.com/buptLwz/CLEAR.
Weize Li, Zhicheng Zhao, Fei Su
Beijing University of Posts and Telecommunications · Beijing Key Laboratory of Network System and Network Culture · Key Laboratory of Interactive Technology and Experience System