Task-Guided Multi-Annotation Triplet Learning for Remote Sensing Representations

Meilun Zhou; Alina Zare

cs.CVApr 4, 2026

Task-Guided Multi-Annotation Triplet Learning for Remote Sensing Representations

Authors: Meilun Zhou, Alina Zare

Organizations: Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL, USA, 32611 · Artificial Intelligence and Informatics Research Institute, University of Florida, Gainesville, FL, USA, 32611

Abstract

Prior multi-task triplet loss methods relied on static weights to balance supervision between various types of annotation. However, static weighting requires tuning and does not account for how tasks interact when shaping a shared representation. To address this, the proposed task-guided multi-annotation triplet loss removes this dependency by selecting triplets through a mutual-information criteria that identifies triplets most informative across tasks. This strategy modifies which samples influence the representation rather than adjusting loss magnitudes. Experiments on an aerial wildlife dataset compare the proposed task-guided selection against several triplet loss setups for shaping a representation in an effective multi-task manner. The results show improved classification and regression performance and demonstrate that task-aware triplet selection produces a more effective shared representation for downstream tasks.

Figures & tables

Fig. 1 : Two-stage architecture. Stage one extracts frozen Vision Transformer–based model embeddings and applies different representation learning losses to shape the latent space. Stage two trains linear task heads on each latent space to measure the impact on downstream performance.

Category	Cow	Deer	Horse
Sample Count	116	51	73
Mean	0.180	0.115	0.374
Std. Dev.	0.096	0.039	0.194
Min	0.028	0.046	0.1105
Max	0.630	0.214	0.744

TABLE I : Sample-wise mutual information for AWIR dataset. Values summarize how strongly the area and symmetric squareness attributes align with species-level semantic categories.

Representation Learning

Representation learning develops data transformations that expose useful structure for prediction, transfer, generation, and control.

See all 773 papers

Beyond Task-Driven Features for Object Detection
Apr 4, 2026Meilun Zhou, Alina ZareObject Detection Representation Learning
Department of Electrical and Computer Engineering University of Florida Gainesville, FL, USA, 32611 · Artificial Intelligence and Informatics Research Institute University of Florida Gainesville, FL, USA, 32611
CORE-MTL: Rethinking Gradient Balancing via Causal Orthogonal Representations
Jun 1, 2026Chengfeng Wu, Tao Zou, Yanru Wu +1Disentangled Representation Learning Causal Representation Learning
Shenzhen Key Laboratory of Ubiquitous Data Enabling, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · State Key Laboratory of Complex & Critical Software Environment, Beihang University, Beijing, China
UniTriGen: Unified Triplet Generation of Aligned Visible-Infrared-Label for Few-Shot RGB-T Semantic Segmentation
May 14, 2026Ping Zhou, Haoyu Wang, Mengmeng Zheng +4Cross-Modal Alignment Multimodal Generation
School of Computer Science, Northwestern Polytechnical University · School of Computer Science & Technology, Xi’an University of Posts & Telecommunications · MMLab, The Chinese University of Hong Kong

Apr 4, 2026J/K move · Enter open · S save

Beyond Task-Driven Features for Object Detection

Meilun Zhou, Alina Zare

Department of Electrical and Computer Engineering University of Florida Gainesville, FL, USA, 32611 · Artificial Intelligence and Informatics Research Institute University of Florida Gainesville, FL, USA, 32611

Object Detection Representation Learning

Open paper AskPDF arXiv

Task-driven features learned by modern object detectors optimize end task loss yet often capture shortcut correlations that fail to reflect underlying annotation structure. Such representations limit transfer, interpretability, and robustness when task definitions change or supervision becomes sparse. This paper introduces an annotation-guided feature augmentation framework that injects embeddings into an object detection backbone. The method constructs dense spatial feature grids from annotation-guided latent spaces and fuses them with feature pyramid representations to influence region proposal and detection heads. Experiments across wildlife and remote sensing datasets evaluate classification, localization, and data efficiency under multiple supervision regimes. Results show consistent improvements in object focus, reduced background sensitivity, and stronger generalization to unseen or weakly supervised tasks. The findings demonstrate that aligning features with annotation geometry yields more meaningful representations than purely task optimized features.

Abstract

Representation Learning

Explore similar work

Beyond Task-Driven Features for Object Detection

CORE-MTL: Rethinking Gradient Balancing via Causal Orthogonal Representations

UniTriGen: Unified Triplet Generation of Aligned Visible-Infrared-Label for Few-Shot RGB-T Semantic Segmentation