cs.CVAug 26, 2026

Asymmetric Cross-Modal Fine-Grained Visual Categorization: ACF-Net and the BirdPro Benchmark

Authors: Bohan Deng, Shuo Ye, Zitong Yu

Organizations: Great Bay University

Abstract

Audio-visual cross-modal Fine-Grained Visual Categorization (FGVC) aims to identify fine-grained categories by jointly leveraging visual and auditory information. However, FGVC under asymmetric cross-modal scenarios has received limited attention, where paired video and audio are not strictly synchronized and may not even correspond to the same individual or moment. Such weak and ambiguous cross-modal correspondence poses substantial challenges to effective representation learning and modality alignment. To address these issues, we propose ACF-Net, a novel optical flow-guided framework for asymmetric audio-visual fine-grained learning. ACF-Net consists of two key modules: Optical Flow-Guided Motion (OFGM) and Asymmetric CrossModal Adaptive Fusion (ACAF). OFGM captures motion-sensitive visual cues and suppresses irrelevant background interference, thereby enhancing discriminative dynamic representations in videos. ACAF estimates modality reliability under weakly matched audio-video pairs and performs uncertainty-aware adaptive fusion to improve category-level recognition robustness. To support research on asymmetric cross-modal FGVC, we further construct BirdPro, a new bird-oriented audio-visual benchmark, since existing datasets often lack large-scale category-level audio-video associations under non-strict temporal and instance correspondence. BirdPro contains 1,919 audio recordings and 11,965 videos covering 194 bird species. Extensive experiments show that ACF-Net achieves the best results compared with representative baseline methods, outperforming the strongest baselines by 2.97% and 1.92% in the fused and mismatched settings, respectively.

Figures & tables

Explore similar work

CardsList
  1. AVSCap: Orchestrating Audio-Visual Synergy for Omni-modal Video Captioning

    Jul 14, 2026Yanghai Wang, Jiahao Wang, Jiafu Tang +9Audio-Visual UnderstandingAudio-Visual Alignment

  2. Acoustically Grounded Cost Learning for Open-Vocabulary Audio-Visual Semantic Segmentation

    Aug 29, 2026Tianrui Hui, Shaofei Huang, Qisong Han +6Audio-Visual UnderstandingOpen-Vocabulary Semantic Segmentation

  3. AGF-Net: A Robust Multimodal Emotion Recognition Model for Asynchronous Audio-Visual Signals Based on Predictive Dynamic Fusion with Negative Learning

    Oct 22, 2026Chuan Han Li, Yun Chien ChengMultimodal Emotion RecognitionMultimodal Fusion