cs.SDOct 12, 2026

Behavior-Conditioned Animal Vocalization Generation: Event-Aware Modeling and Multidimensional Evaluation

Authors: Xinlong Deng, Yong Tian, Jian Guan, Qiuqiang Kong, Jie Jiang, Yin Cao

Organizations: College of Artificial Intelligence, China University of Petroleum, Beijing · College of Computer Science and Technology, Harbin Engineering University, Harbin, China · Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong, China · Institute of Acoustics, Chinese Academy of Sciences

Abstract

Behavior-conditioned animal vocalization generation aims to synthesize vocalizations from natural-language descriptions of animal behaviors, social interactions, and contexts. This task remains largely unexplored, with limited behavioral text supervision and substantial variation in vocalization duration. We construct the Behavior-Vocalization Dataset (BeVo) by integrating public bioacoustic data from six species with natural-language behavioral conditions derived from species-specific ethological and bioacoustic literature, totaling 31,191 events. We then adapt a rectified-flow-based generator with Event-Aware variable-length modeling, which encodes vocalizations at their actual durations and uses masks to prevent attention to padded latent positions and exclude these positions from the training loss. We evaluate vocalization discriminability, acoustic distribution fidelity, and language-aligned representation fidelity. Experiments show that task-specific adaptation improves all evaluated metrics compared with its zero-shot baseline, while Event-Aware modeling further increases vocalization-anchor macro accuracy from 59.86% to 61.99% and reduces Frechet Audio Distance from 1.40 to 1.22. Conditioning analyses further show that behavioral conditions improve target-anchor discriminability, while swapping behavior shifts generations toward the corresponding vocalization anchors. Demos are available on our project page.

Figures & tables

Explore similar work

CardsList
  1. Designed Vocalizations Dataset: Sound-Designed Human and Animal Voices for Non-human Voice Conversion

    Jul 23, 2026Seolhee Lee, Minsu Kang, Yangsun Lee +3Voice Conversion

  2. AC-Foley: Reference-Audio-Guided Video-to-Audio Synthesis with Acoustic Transfer

    Mar 16, 2026Pengjun Fang, Yingqing He, Yazhou Xing +3Audio-Video GenerationAudio Generation

  3. Speech LLMs That Breathe: Modeling Non-Verbal Vocalizations in Dialogue

    Oct 19, 2026Heyang Liu, Lixin Zhang, Jiayi Huang +6Nonverbal Vocalization GenerationSpeech Language Models