cs.CVFeb 28, 2026

From Visual to Multimodal: Systematic Ablation of Encoders and Fusion Strategies in Animal Identification

Authors: Vasiliy Kudryavtsev, Kirill Borodin, German Berezin, Kirill Bubenchikov, Grach Mkrtchian, Alexander Ryzhkov

Organizations: Faculty of IT, Technical University of Communication and Informatics, Moscow, Russia, 111024 · AI lab, Avito, Moscow, Russia, 125196

Abstract

Automated animal identification is a practical task for reuniting lost pets with their owners, yet current systems often struggle due to limited dataset scale and reliance on unimodal visual cues. This study introduces a multimodal verification framework that enhances visual features with semantic identity priors derived from synthetic textual descriptions. We constructed a massive training corpus of 1.9 million photographs covering 695,091~unique animals to support this investigation. Through systematic ablation studies, we identified SigLIP2-Giant and E5-Small-v2 as the optimal vision and text backbones. We further evaluated fusion strategies ranging from simple concatenation to adaptive gating to determine the best method for integrating these modalities. Our proposed approach utilizes a gated fusion mechanism and achieved a Top-1 accuracy of 84.28% and an Equal Error Rate of 0.0422 on a comprehensive test protocol. These results represent an 11% improvement over leading unimodal baselines and demonstrate that integrating synthesized semantic descriptions significantly refines decision boundaries in large-scale pet re-identification.\\ Conference update. This work was accepted to the FGVC13 Workshop at CVPR 2026. We thank Vadim Ezhov and Nikolai Shcheglov for their help with the poster presentation and demonstration at the conference.

Figures & tables

Explore similar work

CardsList
  1. DS@GT ARC at AnimalCLEF 2026: Species-Aware Graph Construction for Multi-Species Animal Re-Identification

    Jul 17, 2026Evan Sinclair Smith, Anthony Miyaguchi, Snigdha Palamari +1Animal Re-IdentificationImage Matching

  2. Advancing Wildlife Conservation through Multimodal Animal Re-Identification with Environmental Metadata

    Sep 28, 2026Yuzhuo Li, Di Zhao, Tingrui Qiao +3Animal Re-IdentificationWildlife Monitoring

  3. Visual-Prompt Guided Wildlife Instance-Level Recognition

    Aug 18, 2026Mufhumudzi Muthivhi, Jiahao Huo, Terence van Zyl +1Animal Re-IdentificationObject Detection