PRISM: Primitive-Integrated Separable Modulation with Priority Visual Memory Queue for Compositional Zero-Shot Learning
Authors: Jianhua Wang, Zhihang Zheng, Yang Liu, Jiale Du
Organizations: School of Electronic Engineering, Xidian University, Shaanxi 710071, China · School of Computer Science and Technology, Xidian University, Shaanxi 710071, China · School of Telecommunications Engineering, Xidian University, Shaanxi 710071, China
Compositional Zero-Shot Learning (CZSL) recognizes unseen attribute--object pairs by recombining primitives from seen compositions. Existing methods stop sharing evidence at deployment: fixed prototypes cannot exploit unlabeled test streams, while composition-wise refinement cannot propagate corrections across pairs sharing a primitive. We introduce PRISM, an online CZSL framework that extends primitive sharing to test-time adaptation. Its Hierarchical Knowledge Accumulation Module (HiKAM) factorizes prototype migration into composition-, attribute-, and object-level residual banks. A Priority Visual Memory Queue (PVMQ), initialized by Prototype Queue Initialization (PQI), grounds text updates with bounded visual evidence. The same adaptive core supports exhaustive Full PRISM and recall-calibrated Adaptive PRISM-Lite, which selects candidate budgets from held-out ranks. Comprehensive closed- and open-world experiments on UT-Zappos, MIT-States, and C-GQA demonstrate the state of the art, with Full PRISM achieving the best open-world AUC and harmonic mean on all three benchmarks and the best closed-world results on UT-Zappos and C-GQA. Fine-grained long-tail and non-stationary primitive-block evaluations reveal gains on under-supported concepts and stable adaptation under prolonged skew. Adaptive PRISM-Lite further delivers substantial efficiency gains in the full open-world space with only a small accuracy cost. PRISM thus combines primitive-shared online adaptation, multimodal grounding, and candidate-aware inference.
Figures & tables
Figure 1: Test-time adaptation under label-space shift, showing primitive-shared textual updates and Priority Visual Memory Queue retrieval from an unlabeled test stream.
Figure 2: Overall architecture of PRISM at test time, showing primitive-shared textual and visual updates to composition prototypes.
Methods
UT-Zappos
MIT-States
C-GQA
AUC
HM
Seen
Unseen
AUC
HM
Seen
Unseen
AUC
HM
Seen
Unseen
DFSP ( Lu et al. 2023 ) CVPR’23
36.0
47.2
66.7
71.7
20.6
37.3
46.9
52.0
10.5
27.1
38.2
32.0
Troika ( Huang et al. 2024 ) CVPR’24
41.7
54.6
66.8
73.8
22.1
39.3
49.0
53.0
12.4
29.4
41.0
35.7
PLID ( Bao et al. 2024 ) ECCV’24
38.7
52.4
67.3
68.8
22.1
39.0
49.7
52.4
11.0
27.9
38.8
33.0
IMAX ( Jiang et al. 2025 ) TPAMI’25
40.6
54.2
69.3
70.7
21.9
39.1
48.7
53.8
12.8
29.8
39.7
35.8
ClusPro ( Qu et al. 2025 ) ICLR’25
46.6
58.5
70.7
76.0
23.8
40.7
52.1
54.0
14.9
32.8
44.3
37.8
Table 1: Closed-world task-level comparison of complete CZSL systems on UT-Zappos, MIT-States, and C-GQA. The best and second-best AUC/HM values are bold and underlined, respectively.
Variant
HiKAM
Prim.
PVMQ
PQI
UT-Zappos
MIT-States
C-GQA
AUC
HM
Seen
Unseen
AUC
HM
Seen
Unseen
AUC
HM
Seen
Unseen
w/o TTA
43.6
55.5
68.7
74.3
22.2
39.3
49.1
52.9
15.2
33.1
43.9
39.6
Text TTA only
✓
38.4
52.6
66.8
66.4
22.2
39.2
49.1
53.0
15.2
33.1
44.7
38.7
w/o PVMQ
✓
✓
44.5
57.3
71.3
71.2
22.2
39.1
49.0
53.0
15.6
33.4
45.3
38.9
w/o PQI
✓
✓
✓
48.9
60.5
75.4
72.6
22.3
39.0
49.3
53.3
15.6
33.4
45.2
39.2
Full PRISM
✓
✓
✓
✓
53.7
64.8
79.0
73.8
23.3
40.2
50.7
53.7
16.7
35.4
45.4
41.1
Table 3: Closed-world ablation study on UT-Zappos, MIT-States, and C-GQA.
Setting
Mode
Candidates
Recall (%)
Img/s
Mean (ms)
AUC
HM
Closed
Full PRISM
7,767
100.00
2.35
423.86
16.69
35.41
Fixed Lite (100)
100
90.92
15.32
64.48
15.67
34.16
Fixed Lite (200)
200
95.21
14.83
66.63
15.65
34.04
Adaptive PRISM-Lite
148/152.3
93.55
13.11
76.28
15.93
34.41
Open
Full PRISM
278,362
100.00
1.84
543.89
5.45
17.58
Fixed Lite (100)
100
71.81
10.63
94.09
5.10
16.42
Table 4: C-GQA closed- and open-world profiling; Adaptive candidate counts are calibrated/realized.
Analysis
Group
#Comp.
#Images
Closed-world
Open-world
No TTA
PRISM
Δ
No TTA
PRISM
Δ
c
All
36
2914
24.3
34.6
+10.3
24.1
33.1
+8.9
Tail
11
50
13.6
32.1
+18.5
13.6
29.7
+16.1
Body
14
546
19.2
20.9
+1.8
18.7
19.9
+1.2
Head
11
2318
41.6
54.5
+12.9
41.5
53.2
+11.7
a
Tail
8
152
26.1
37.1
+11.0
26.1
37.3
+11.2
Table 5: Long-tail analysis on UT-Zappos under closed- and open-world protocols.
Figure 3: Closed-world cumulative top-1 accuracy with increasing test sample size on three datasets.
Figure 4: Closed-world sensitivity analysis of α .
Figure 5: Closed-world sensitivity analysis of Priority Visual Memory Queue capacity K .
Zero-shot learning lets models recognize classes or solve tasks absent from task-specific training examples by using semantic descriptions or pretrained knowledge.
School of Computer Science and Technology, Beijing Jiaotong University · Key Laboratory of Big Data & Artificial Intelligence in Transportation (Ministry of Education)
Beijing University of Posts and Telecommunications · Beijing Key Laboratory of Network System and Network Culture · Key Laboratory of Interactive Technology and Experience System