Incentivizing Generative Zero-Shot Learning via Outcome-Reward Reinforcement Learning with Visual Cues

Wenjin Hou; Xiaoxiao Sun; Hehe Fan

cs.CVMar 22, 2026

Incentivizing Generative Zero-Shot Learning via Outcome-Reward Reinforcement Learning with Visual Cues

Authors: Wenjin Hou, Xiaoxiao Sun, Hehe Fan

Organizations: CCAI, Zhejiang University · Stanford University · State Key Laboratory of CAD&CG, Zhejiang University

Abstract

Recent advances in zero-shot learning (ZSL) have demonstrated the potential of generative models. Typically, generative ZSL synthesizes visual features conditioned on semantic prototypes to model the data distribution of unseen classes, followed by training a classifier on the synthesized data. However, the synthesized features often remain task-agnostic, leading to degraded performance. Moreover, inferring a faithful distribution from semantic prototypes alone is insufficient for classes that are semantically similar but visually distinct. To address these and advance ZSL, we propose RLVC, an outcome-reward reinforcement learning RL framework with visual cues for generative ZSL. At its core, RL empowers the generative model to self-evolve, implicitly enhancing its generation capability. In particular, RLVC updates the generative model using an outcome-based reward, encouraging the synthesis of task-relevant features. Furthermore, we introduce class-wise visual cues that (i) align synthesized features with visual prototypes and (ii) stabilize the RL training updates. For the training process, we present a novel cold-start strategy. Comprehensive experiments and analyses on three prevalent ZSL benchmarks demonstrate that RLVC achieves state-of-the-art results with a 4.7% gain.

Figures & tables

Figure 1: Motivating illustration. (a) Existing generative ZSL methods train with adversarial losses conditioned only on semantic prototypes. This often leads to task-agnostic synthesized features and inter-class overlap. (b) Our RLVC incentivizes the generative model updating via RL reward and visual cues, enabling synthesized features that remain task-relevant and faithfully represent the data distribution.

Figure 2: Model architecture and training of RLVC. The top panel shows how we train the reward model with a visual encoder to produce fine-tuned visual features and reward signals. The bottom panel depicts how we update the policy model $G_{\theta}$ ( i.e . , generator) via outcome-reward reinforcement learning ( blue arrows ) and visual cues ( green arrows ), enabling synthesized features that remain task-relevant and faithfully represent the data distribution. $\mathbf{x}_{0}$ and $\tilde{\mathbf{x}}_{0}$ denote the real and synthesized features of seen classes, respectively.

Figure 3: The training trends of our RLVC on CUB, including raw reward, EMA-adjusted advantage and ZSL accuracy.

Method		Backbone	CUB				SUN				AWA2
Method		Backbone	Acc	U	S	H	Acc	U	S	H	Acc	U	S	H
Embedding	CLIP ⋆ [ICML’21] [ 61 ]	ViT	–	55.2	54.8	55.0	–	–	–	–	–	–	–	–
	TransZero++ [TPAMI’22] [ 8 ]	ResNet	78.3	67.5	73.6	70.4	67.6	48.6	37.8	42.5	72.6	64.6	82.7	72.5
	DUET [AAAI’23] [ 19 ]	ViT	72.3	62.9	72.8	67.5	64.4	45.7	45.8	45.8	69.9	63.7	84.7	72.7
	ICIS [ICCV’23] [ 20 ]	ResNet	60.6	45.8	73.7	56.5	51.8	45.2	25.6	32.7	64.6	35.6	93.3	51.6
	HAS [ACM MM’23] [ 18 ]	ResNet	76.5	69.6	74.1	71.8	63.2	42.8	38.9	40.8	71.4	63.1	87.3	73.3

Table 1: Compared our RLVC with the SOTA methods in CZSL and GZSL settings on CUB, SUN and AWA2 benchmarks. The symbol “

\star

” indicates the semantic prototypes from the class name. The symbol “–” denotes that no results are provided in the original papers. The bold and underlined markings indicate the best and second-best results, respectively.

Method	Semantic prototype	CUB				SUN				AWA2
Method	Semantic prototype	Acc	U	S	H	Acc	U	S	H	Acc	U	S	H
Vanilla model	word embedding	61.8	52.4	66.3	58.5	72.0	55.4	46.4	50.5	73.1	62.8	82.3	71.2
RLVC	word embedding	62.8 (+1.0)	51.5	70.9	59.7 (+1.2)	72.1 (+0.1)	55.0	50.3	52.5 (+2.0)	74.5 (+1.4)	61.3	91.7	73.5 (+2.3)
Vanilla model	attribute vector	88.6	71.0	79.8	75.1	75.8	58.3	52.3	55.1	75.7	70.1	76.0	72.8
RLVC	attribute vector	90.1 (+1.5)	80.9	81.4	81.2 (+6.1)	77.7 (+1.9)	59.6	55.6	57.6 (+2.6)	84.0 (+8.3)	78.4	82.4	80.4 (+7.6)

Table 2: Effectiveness validation of RLVC across different semantic prototypes, including word embeddings of class names and expert-annotated attribute vectors. We mark the best results in bold and the accuracy gains (%) in parentheses .

Configuration	CUB		SUN		AWA2
Configuration	Acc	H	Acc	H	Acc	H
RLVC w/o RL & visual cues	88.6	75.1	75.8	55.1	75.7	72.8
RLVC w/o RL ( i.e . , Eq. ( 11 ))	89.2	80.1	76.1	55.6	79.4	73.9
RLVC w/o visual cues ( i.e . , Eq. ( 13 ))	88.9	79.2	77.0	56.9	74.9	76.6
RLVC w/o fine-tuning visual encoder	89.2	77.5	76.0	56.4	81.1	76.3
RLVC w/o advantage ( i.e . , Eq. ( 9 ))	89.6	79.7	76.0	56.0	82.4	78.2

Table 3: Results of RLVC variants on CUB, SUN and AWA2 datasets. We ablate specific components to assess their effectiveness. The bold marking indicates the best results.

Loss	CUB		SUN		AWA2
Loss	Acc	H	Acc	H	Acc	H
$\rm KL$	88.9	80.1	77.2	58.2	76.4	76.4
$\ell_{\rm{1}}$	89.2	80.9	77.3	57.8	75.9	77.6
$\mathcal{L}_{\rm{PD}}$	90.1	81.2	77.7	57.6	84.0	80.4

Table 4: Comparison results for different prototype-distillation losses combined with RLVC on CUB, SUN and AWA2 datasets. The bold marking indicates the best results.

Figure 4: Qualitative t-SNE visualization of RLVC on CUB: (a) without RL and visual cues, (b) without visual cues, and (c) full RLVC. We use real features of seen classes and synthetic features of unseen classes. Zoom in for details.

Panel of Figure 5 — Figure 5: Effect of hyperparameters on CUB, including the epoch of RL cold-start, the coefficient of visual loss, and the number of synthetic unseen samples.

Zero-Shot Learning

Zero-shot learning lets models recognize classes or solve tasks absent from task-specific training examples by using semantic descriptions or pretrained knowledge.

See all 184 papers

Learning Adaptive and Visually Aligned Conditions for Generative Zero-Shot Learning
Mar 6, 2026Haojie Pu, Zhuoming Li, Yongbiao Gao +3Zero-Shot Learning Vision-Language Alignment
School of Computer Science and Engineering, Southeast University, Nanjing, China
Dynamic Visual-semantic Alignment for Zero-shot Learning with Ambiguous Labels
Apr 20, 2026Jiangnan Li, Linqing Huang, Xiaowen Yan +3Zero-Shot Learning Noisy-Label Learning
College of Computer Science and Technology, Qingdao University · School of Computer Science, Shanghai JiaoTong University · Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center, Qilu University of Technology (Shandong Academy of Sciences)
Mutually Causal Semantic Distillation Network for Zero-Shot Learning
Mar 18, 2026Shiming Chen, Shuhuang Chen, Guo-Sen Xie +1Zero-Shot Learning Cross-Modal Representation Learning
Huazhong University of Science and Technology, Wuhan, China · National Anti-Counterfeit Engineering Research Center, Wuhan, Chna · Nanjing University of Science and Technology, Nanjing, China

Mar 6, 2026J/K move · Enter open · S save

Learning Adaptive and Visually Aligned Conditions for Generative Zero-Shot Learning

Haojie Pu, Zhuoming Li, Yongbiao Gao, Hui Liu+2

School of Computer Science and Engineering, Southeast University, Nanjing, China

Zero-Shot Learning Vision-Language Alignment Conditional Generative Modeling

Open paper AskPDF arXiv

Generative zero-shot learning (ZSL) synthesizes visual features for unseen classes by learning a semantic-conditioned generator from seen classes. Since semantic conditions determine the knowledge transfer from seen to unseen classes, their effectiveness is critical for generating plausible features. However, existing class-level semantic prototypes lack semantic diversity to capture intra-class variations, while exhibiting inconsistent inter-class structures with the visual space due to the semantic-visual gap. To address these issues, we propose Adaptive Attribute Distribution and Visual Structure Alignment (AAVS), a unified semantic condition learning framework for generative ZSL. Specifically, the Adaptive Attribute Distribution (AAD) learns dimension-adaptive attribute distributions with discriminability-guided variation calibration to capture transferable semantic diversity, while Visual Structure Alignment (VSA) aligns the diverse attributes with refined visual prototypes to learn visual inter-class structures. By jointly modeling semantic diversity and visual structural consistency, AAVS provides more effective generation conditions for unseen classes to generate visual features. Extensive quantitative and qualitative evaluations on three benchmarks demonstrate that AAVS outperforms existing generative ZSL methods and validate the effectiveness of the learned generation conditions.

Abstract

Zero-Shot Learning

Explore similar work

Learning Adaptive and Visually Aligned Conditions for Generative Zero-Shot Learning

Dynamic Visual-semantic Alignment for Zero-shot Learning with Ambiguous Labels

Mutually Causal Semantic Distillation Network for Zero-Shot Learning