cs.CVNov 2, 2026

Behavioral Affinity Transduction for Zero-Shot Vision-Language Classification

Authors: Junxi Huang, Wenshuo Chen, Xiucheng Wang, Meitong Shen, Shaofeng Liang, Yuehan Kou, Yutao Yue, Nan Cheng

Organizations: School of Telecommunications Engineering, Xidian University, Xi’an, Shaanxi 710071, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China

Abstract

An image that is hard to classify on its own may have a helpful neighbor in the same unlabeled batch. Zero-shot vision-language models (VLMs) score images independently and leave this evidence unused. We study small unlabeled query batches with no labeled examples. Visual similarity can reflect pose or background, so it may connect images from different classes. Each image's scores over the class vocabulary offer another way to choose neighbors. We introduce Behavioral Affinity Transduction (BAT), which builds a sparse graph from these class-response profiles. The graph supplies neighbor evidence during prediction. For EM-Dirichlet, it also screens candidate classes and initializes assignments. BAT plugs into direct and iterative readouts of a frozen VLM. Across diverse datasets, BAT improves the mean accuracy of every tested readout. In the main RN50 comparison, soft and hard EM-Dirichlet gain 4.224.22 and 2.722.72 percentage points.

Figures & tables

Explore similar work

CardsList
  1. Neutral-Reference Prompting for Vision-Language Models

    May 15, 2026Senmao Tian, Xiang Wei, Shunli ZhangVision-Language ModelsVLM Adaptation

  2. Batch-Conditioned Semantic Anchors for Robust Transductive Adaptation of Vision--Language Models

    Nov 4, 2026Mohammed Rahman Sherif Khan Mohammad, Ardhendu Behera, Sandip Pradhan +2VLM AdaptationTest-Time Adaptation of VLMs

  3. Von Mises-Fisher Mixture Model with Dynamic Shrinkage for Realistic Test-Time Transduction

    Jul 17, 2026Jiazhen Huang, Zhiming Liu, Changhu Wang +3Class-Imbalanced LearningTest-Time Adaptation