Noise-aware few-shot learning through bi-directional multi-view prompt alignment

Lu Niu; Cheng Xue

cs.CVMar 12, 2026

Noise-aware few-shot learning through bi-directional multi-view prompt alignment

Authors: Lu Niu, Cheng Xue

Organizations: Southeast University · AIIA, Ministry of Education, China

Abstract

Vision-language models offer strong few-shot capability through prompt tuning but remain vulnerable to noisy labels, which can corrupt prompts and degrade cross-modal alignment. Existing approaches struggle because they often lack the ability to model fine-grained semantic cues and to adaptively separate clean from noisy signals. To address these challenges, we propose NA-MVP, a framework for Noise-Aware few-shot learning through bi-directional Multi-View Prompt alignment. NA-MVP is built upon a key conceptual shift: robust prompt learning requires moving from global matching to region-aware alignment that explicitly distinguishes clean cues from noisy ones. To realize this, NA-MVP employs (1) multi-view prompts combined with unbalanced optimal transport to achieve fine-grained patch-to-prompt correspondence while suppressing unreliable regions; (2) a bi-directional prompt design that captures complementary clean-oriented and noise-aware cues, enabling the model to focus on stable semantics; and (3) an alignment-guided selective refinement strategy that uses optimal transport to correct only mislabeled samples while retaining reliable data. Experiments on synthetic and real-world noisy benchmarks demonstrate that NA-MVP consistently outperforms state-of-the-art baselines, confirming its effectiveness in enabling robust few-shot learning under noisy supervision.

Figures & tables

Figure 1: Limitations of existing prompt learning approaches under noisy labels. Single-view reliance : Limited prompts miss diverse visual patterns. Explicit negatives : Fixed negatives impose rigid supervision. Fixed threshold : Coarse denoising lets noise propagate.

Figure 2: Overview of the NA-MVP framework. Our framework consists of two key modules: (1) Noise-aware alignment (blue arrows): Multiple clean and noise-aware prompts per class are encoded and aligned with local image patches via UOT to generate clean/noisy probabilities. (2) Selective label refinement (green arrows): An adaptive threshold $\phi$ derived from these probabilities identifies mislabeled samples, which are refined via classical OT by aligning global image features with clean text features. The two modules work together to iteratively update the training set while optimizing the prompts, producing a denoised dataset for robust prediction under noisy supervision.

Dataset	Method	Noise rate: Sym						Noise rate: Asym
		0.125	0.25	0.375	0.5	0.625	0.75	0.25	0.5
Caltech101	CoOp [ 68 ]	86.43	81.03	76.73	70.90	61.33	46.90	75.23	49.43
	GCE [ 55 ]	92.00	90.90	90.80	89.30	86.70	79.03	91.20	85.80
	JoAPR [ 20 ]	88.32	87.85	87.00	87.03	84.55	80.15	82.79	69.02
	NLPrompt [ 38 ]	91.73	91.13	90.77	89.93	88.30	86.70	91.17	89.27
	NA-MVP	92.07	92.10	91.60	91.30	90.07	89.37	91.47	89.53

Table 1: Comparison of methods under symmetric and asymmetric noise on five datasets. (%)

Method	4-shot	8-shot	16-shot	32-shot
NLPrompt [ 38 ]	70.57	73.93	76.46	76.87
NA-MVP	76.10	76.27	76.90	77.03

Table 2: Test accuracy (%) on Food101N.

	Method/Noise Ratio	25%	50%	75%	Avg
(a)	CoOp	59.83	50.73	33.67	48.08
(b)	(a)+ Negative label	59.53	52.53	34.40	48.82
(c)	(a)+ Bi-directional	60.13	53.73	35.03	49.63
(d)	(c)+ Multi-view	62.73	55.13	37.63	51.83
(e)	(d)+UOT	62.50	57.70	42.33	54.18
(f)	(d)+OT	62.30	56.80	41.00	53.37

Table 3: Ablation studies on DTD. (%)

Noise rate	12.5%	25%	37.5%	50%	62.5%	75%
DEFT [ 54 ]	88.83	88.23	88.10	86.73	84.10	75.87
NA-MVP	88.50	88.40	88.23	88.13	86.93	86.23

Table 4: The accuracy comparison on OxfordPets Dataset.

Explore similar work

Seeing is Believing: Robust Vision-Guided Cross-Modal Prompt Learning under Label Noise
Apr 10, 2026Zibin Geng, Xuefeng Jiang, Jia Li +6VLM Robustness Visual Prompt Learning
Institute of Computing Technology, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Beijing, China · PCALab, VCIP, College of Computer Science, Nankai University, Tianjin, China +1
TecoPrompt: Temporal-Conservative Prompt Learning for Vision-Language Models
Sep 15, 2026Zeyi Shao, Haowen Hua, Jiaxin Zhang +3Prompt Learning Noisy-Label Learning
Soochow University, Suzhou, China · NVIDIA, Shanghai, China · Heriot-Watt University Malaysia, Putrajaya, Malaysia +1
Intrinsic Gradient Suppression for Label-Noise Prompt Tuning in Vision-Language Models
May 1, 2026Jiayu Li, Jiaxin Qi, Sheng Zhou +2VLM Robustness VLM Adaptation

Apr 10, 2026J/K move · Enter open · S save

Seeing is Believing: Robust Vision-Guided Cross-Modal Prompt Learning under Label Noise

Zibin Geng, Xuefeng Jiang, Jia Li, Zheng Li+5

Institute of Computing Technology, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Beijing, China · PCALab, VCIP, College of Computer Science, Nankai University, Tianjin, China+1

VLM Robustness Visual Prompt Learning VLM Adaptation Noisy-Label Learning Cross-Modal Learning

Open paper AskPDF arXiv

Prompt learning is a parameter-efficient approach for vision-language models, yet its robustness under label noise is less investigated. Visual content contains richer and more reliable semantic information, which remains more robust under label noise. However, the prompt itself is highly susceptible to label noise. Motivated by this intuition, we propose VisPrompt, a lightweight and robust vision-guided prompt learning framework for noisy-label settings. Specifically, we exploit a cross-modal attention mechanism to reversely inject visual semantics into prompt representations. This enables the prompt tokens to selectively aggregate visual information relevant to the current sample, thereby improving robustness by anchoring prompt learning to stable instance-level visual evidence and reducing the influence of noisy supervision. To address the instability caused by using the same way of injecting visual information for all samples, despite differences in the quality of their visual cues, we further introduce a lightweight conditional modulation mechanism to adaptively control the strength of visual information injection, which strikes a more robust balance between text-side semantic priors and image-side instance evidence. The proposed framework effectively suppresses the noise-induced disturbances, reduce instability in prompt updates, and alleviate memorization of mislabeled samples. VisPrompt significantly improves robustness while keeping the pretrained VLM backbone frozen and introducing only a small amount of additional trainable parameters. Extensive experiments under synthetic and real-world label noise demonstrate that VisPrompt generally outperforms existing baselines on seven benchmark datasets and achieves stronger robustness. Our code is publicly available at https://github.com/gezbww/Vis_Prompt.

Abstract

Explore similar work

Seeing is Believing: Robust Vision-Guided Cross-Modal Prompt Learning under Label Noise

TecoPrompt: Temporal-Conservative Prompt Learning for Vision-Language Models

Intrinsic Gradient Suppression for Label-Noise Prompt Tuning in Vision-Language Models