Class Embeddings

Class embeddings represent objects or concepts as vectors in a high-dimensional space, aiming to capture semantic relationships and facilitate tasks like image classification and retrieval. Current research focuses on improving the accuracy and efficiency of class embeddings, particularly in low-data regimes (few-shot learning) and across diverse modalities (vision-language models), often employing techniques like contrastive learning, prompt engineering, and multimodal fusion within transformer and other neural network architectures. These advancements are significantly impacting fields like computer vision, natural language processing, and financial technology by enabling more robust and efficient classification and information retrieval systems.

Papers

November 12, 2024

CJST: CTC Compressor based Joint Speech and Text Training for Decoder-Only ASR
Wei Zhou, Junteng Jia, Leda Sari, Jay Mahadeokar, Ozlem Kalinli
Text to Text CTC Based Audio Encoder Class Embeddings Decoder Only Model Joint Speech

September 21, 2024

A Feature Generator for Few-Shot Learning
Heethanjan Kanagalingam, Thenukan Pathmanathan, Navaneethan Ketheeswaran, Mokeeshan Vathanakumar, Mohamed Afham, Ranga Rodrigo
LeArning Abstract Shot Learning Feature Generation Class Embeddings

June 19, 2024

Semantic Enhanced Few-shot Object Detection
Zheng Wang, Yingjie Gao, Qingjie Liu, Yunhong Wang
Shot Object Detection Semantic Embeddings Semantic Loss Class Representation Class Embeddings

June 17, 2024

BaFTA: Backprop-Free Test-Time Adaptation For Zero-Shot Vision-Language Models
Xuefeng Hu, Ke Zhang, Min Sun, Albert Chen, Cheng-Hao Kuo, Ram Nevatia
Vision Language Model Test Time Adaptation Visual Embeddings Actor Loss Class Embeddings

May 28, 2024

FinEmbedDiff: A Cost-Effective Approach of Classifying Financial Documents with Vector Sampling using Multi-modal Embedding Models
Anjanava Biswas, Wrick Talukdar
Full Model Financial Text Class Embeddings Cost Effective Approach Vector Similarity Search Multimodal Design Document

May 22, 2024

Embedding Generalized Semantic Knowledge into Few-Shot Remote Sensing Segmentation
Yuyu Jia, Wei Huang, Junyu Gao, Qi Wang, Qiang Li
Image Segmentation Remote Sensing Shot Segmentation Class Embeddings Image Speak Generalized Knowledge

May 6, 2024

Class-relevant Patch Embedding Selection for Few-Shot Image Classification
Weihao Jiang, Haoyang Cui, Kun He
Image Classification View Selection Pre Trained Vision Transformer Class Relevant Patch Patch Embeddings Class Embeddings

December 7, 2023

Bootstrapping Autonomous Driving Radars with Self-Supervised Learning
Yiduo Hao, Sohrab Madani, Junfeng Guan, Mohammed Alloulah, Saurabh Gupta, Haitham Hassanieh
Self Supervised Learning Self Supervised Learning Framework Class Embeddings Multiple Radar

November 20, 2023

LLMs as Visual Explainers: Advancing Image Classification with Evolving Visual Descriptions
Songhao Han, Le Zhuo, Yue Liao, Si Liu
Large Language Model Vision Language Model Medical LLM Image Classification Visual Explanation Visual Description Class Embeddings

August 24, 2023

Boosting Semantic Segmentation from the Perspective of Explicit Class Embeddings
Yuhe Liu, Chuanjian Liu, Kai Han, Quan Tang, Zengchang Qin
Semantic Segmentation Visual Perspective Segmentation Mask Class Embeddings Agnostic Mask

July 23, 2023

Geometry-Aware Adaptation for Pretrained Models
Nicholas Roberts, Xintong Li, Dyah Adila, Sonia Cromp, Tzu-Heng Huang, Jitian Zhao, Frederic Sala
Zero Shot Zero Shot Model Class Embeddings Optimal Training

June 12, 2023

Active Globally Explainable Learning for Medical Images via Class Association Embedding and Cyclic Adversarial Generation
Ruitao Xie, Jingbang Chen, Limai Jiang, Rui Xiao, Yi Pan, Yunpeng Cai
Generative Adversarial Network Medical Image Explainable Artificial Intelligence Class Specific Feature Class Embeddings Active Learning With Contrastive Explanation

April 12, 2023

NoisyTwins: Class-Consistent and Diverse Image Generation through StyleGANs
Harsh Rangwani, Lavish Bansal, Kartik Sharma, Tejan Karmali, Varun Jampani, R. Venkatesh Babu
Latent Space Image Generation StyleGAN Latent ImageNet LT Class Embeddings

December 29, 2022

Cross Version Defect Prediction with Class Dependency Embeddings
Moti Cohen, Lior Rokach, Rami Puzis
Code Metric Class Embeddings Defect Prediction

October 10, 2022

FS-DETR: Few-Shot DEtection TRansformer with prompting and without re-training
Adrian Bulat, Ricardo Guerrero, Brais Martinez, Georgios Tzimiropoulos
Detection Transformer Visual Prompting Shot Detection Lw Detr Class Embeddings DETR Model

September 26, 2022

Information-Theoretic Hashing for Zero-Shot Cross-Modal Retrieval
Yufeng Shi, Shujian Yu, Duanquan Xu, Xinge You
Cross Modal Retrieval Semantic Encoding Class Embeddings Hamming Space

August 10, 2022

Imbalance Trouble: Revisiting Neural-Collapse Geometry
Christos Thrampoulidis, Ganesh R. Kini, Vala Vakilian, Tina Behnia
Cross Entropy Loss Neural Collapse Group Imbalance Class Embeddings

July 18, 2022

Angular Gap: Reducing the Uncertainty of Image Difficulty through Model Calibration
Bohua Peng, Mobarakol Islam, Mei Tu
High Uncertainty Anticipation Filling Gap Model Calibration Image Complexity Class Embeddings Point to Plane Distance Hyperspherical Learning

December 21, 2021

Contrastive Object Detection Using Knowledge Graph Embeddings
Christopher Lang, Alexander Braun, Abhinav Valada
Object Detection Object Recognition Knowledge Graph Embeddings Transformer Based Object Open World Object Class Embeddings