cs.CVNov 16, 2026

Learning a Task-Adaptive Low-Dimensional Semantic Space for Improved Visual Classification

Authors: Nikalal Kaldera, Wanqing Li, Chris Bunn, Philip O. Ogunbona

Organizations: School of Computing and Information Technology University of Wollongong Wollongong, NSW 2500, Australia

Abstract

Semantic visual classification commonly aligns visual embeddings with semantic spaces induced by pretrained text encoders. However, such spaces are typically high-dimensional, weakly aligned with visual feature geometry, and insufficiently discriminative for downstream classification tasks. We propose TaSS, a framework for learning a task-adaptive low-dimensional semantic space that preserves semantic structure while explicitly optimizing class separability. TaSS is constructed through a parameterized geometry transformation of pretrained text embeddings that enlarges inter-class margins via distance reshaping and improves intra-class compactness through prototype-preserving semantic constraints. The resulting semantic space maintains meaningful semantic relationships while producing a discriminative geometry tailored for classification. Frozen visual encoder features are subsequently aligned to TaSS using a lightweight MLP, enabling improved discriminative learning without fine-tuning pretrained backbones. We further provide theoretical analysis showing that TaSS preserves semantic consistency while enforcing enhanced inter-class separation and controlled intra-class variance. Extensive experiments on skeleton-based human action recognition, video classification, and image classification benchmarks demonstrate consistent improvements over existing semantic alignment approaches, with gains exceeding 3 percentage points on average and up to 8.5 points on challenging datasets.

Figures & tables

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. SPACE: Semantic Projection and Alignment of CLIP Embeddings for Domain Adaptation

    Sep 19, 2026João Renato Ribeiro Manesco, Danilo Samuel Jodas, Douglas Rodrigues +2Domain AdaptationUnsupervised Domain Adaptation

  2. MTV: Revisiting Multi-Task Visual Representation Learning

    Jan 20, 2026Shangzhe Di, Zhonghua Zhai, Weidi XieVisual Representation LearningSelf-Supervised Visual Representation Learning

  3. Learning to Perceive "Where": Spatial Pretext Tasks for Robust Self-Supervised Learning

    May 11, 2026Yang Shen, Yusen Cai, Weronika Hryniewska-Guzik +2Visual Spatial ReasoningSelf-Supervised Visual Representation Learning