Audio-Visual Navigation

Latest papers

11 papers3 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Two-Stage Multi-modal Learning for Dynamic Audio-Visual Navigation: From Physical Alignment to Scene Aggregation
Oct 8, 2026Jia Li, Yinfeng YuAudio-Visual Navigation Audio-Visual Alignment
School of Computer Science and Technology, Xinjiang University, Urumqi 830017, China · Joint Research Laboratory for Embodied Intelligence, Xinjiang University · Joint International Research Laboratory of Silk Road Multilingual Cognitive Computing, Xinjiang University
RAO-Nav: Probing Omni-Language Models for Zero-shot Semantic Audio-Visual Navigation
Sep 26, 2026Qilang Ye, Meng Liu, Yu ZhouAudio-Visual Navigation Embodied Navigation
VCIP & TMCC & DISSec, College of Computer Science & College of Cryptology and Cyber Science, Nankai University · Zhongguancun Academy · Shandong University
OmniEcho: Audio-Visual Spatial Understanding for Omni-Modal Embodied Agents
Sep 20, 2026Ruixun Liu, Yuxuan Wang, Jiacheng Xie +10Audio-Visual Understanding Spatial Reasoning Benchmarks
School of Intelligence Science and Technology, Peking University · State Key Laboratory of General Artificial Intelligence, Peking University · Alibaba Token Hub, Alibaba Group +1
A Hybrid Mamba for Audio-Visual Navigation
Jul 14, 2026Yi Wang, Yinfeng YuMamba Embodied Navigation
School of Computer Science and Technology, Xinjiang University, Urumqi 830017, China. · Joint Research Laboratory for Embodied Intelligence, Xinjiang University. · Joint International Research Laboratory of Silk Road Multilingual Cognitive Computing, Xinjiang University.
LH-AVLN: A Benchmark for Long-Horizon Audio-Visual-Language Navigation
Jul 4, 2026Rufeng Chen, Yue Chang, Zili Shao +4Embodied Navigation Audio-Visual Navigation
The Hong Kong University of Science and Technology (Guangzhou) · Jilin University
AMAVA: Adaptive Motion-Aware Video-to-Audio Framework for Visually-Impaired Assistance
Apr 26, 2026Benjamin Klein, Kazi Ruslan Rahman, Sanchita GhoseText-to-Audio Generation Visually Impaired Navigation Assistance
Department of Computer Science, San Francisco State University, San Francisco, CA 94132, USA · Department of Mathematics, San Francisco State University, San Francisco, CA 94132, USA · Department of Computer Engineering, San Francisco State University, San Francisco, CA 94132, USA
Generalizable Audio-Visual Navigation via Binaural Difference Attention and Action Transition Prediction
Apr 6, 2026Jia Li, Yinfeng YuAudio-Visual Navigation Embodied Navigation
Joint Research Laboratory for Embodied Intelligence, Xinjiang University · Joint International Research Laboratory of Silk Road Multilingual Cognitive Computing, Xinjiang University · School of Computer Science and Technology, Xinjiang University, Urumqi 830017, China
Reliability-Aware Geometric Fusion for Robust Audio-Visual Navigation
Apr 2, 2026Teng Liu, Yinfeng YuAudio-Visual Navigation Embodied Navigation
Joint Research Laboratory for Embodied Intelligence, Xinjiang University · Joint International Research Laboratory of Silk Road Multilingual Cognitive Computing, Xinjiang University · School of Computer Science and Technology, Xinjiang University, Urumqi 830017, China
Spatial-Aware Conditioned Fusion for Audio-Visual Navigation
Apr 2, 2026Shaohang Wu, Yinfeng YuAudio-Visual Navigation Embodied Navigation
Joint Research Laboratory for Embodied Intelligence, Xinjiang University · Joint International Research Laboratory of Silk Road Multilingual Cognitive Computing, Xinjiang University · School of Computer Science and Technology, Xinjiang University, Urumqi 830017, China
Audio Spatially-Guided Fusion for Audio-Visual Navigation
Apr 2, 2026Xinyu Zhou, Yinfeng YuAudio-Visual Navigation Embodied Navigation
Joint Research Laboratory for Embodied Intelligence, Xinjiang University · Joint International Research Laboratory of Silk Road Multilingual Cognitive Computing, Xinjiang University · School of Computer Science and Technology, Xinjiang University, Urumqi 830017, China
Semantic Audio-Visual Navigation in Continuous Environments
Mar 20, 2026Yichen Zeng, Hebaixu Wang, Meng Liu +4Audio-Visual Navigation Multimodal Transformer
Wuhan University · Zhongguancun Academy · Shandong Jianzhu University +4

Oct 8, 2026J/K move · Enter open · S save

Two-Stage Multi-modal Learning for Dynamic Audio-Visual Navigation: From Physical Alignment to Scene Aggregation

Jia Li, Yinfeng Yu

School of Computer Science and Technology, Xinjiang University, Urumqi 830017, China · Joint Research Laboratory for Embodied Intelligence, Xinjiang University · Joint International Research Laboratory of Silk Road Multilingual Cognitive Computing, Xinjiang University

Audio-Visual Navigation Audio-Visual Alignment Multimodal Sensor Fusion

Open paper AskPDF arXiv

Embodied Audio-Visual Navigation (AVN) requires an agent to locate sound targets by jointly leveraging visual and auditory cues. While prior work has achieved notable success with static sound sources, real-world scenarios often involve moving targets, giving rise to the more challenging problem of Dynamic Audio-Visual Navigation (D-AVN). Existing approaches largely rely on semantic auditory features, which limits generalization to unheard sound categories and hinders fine-grained spatial reasoning. To address these challenges, we propose a novel Two-Stage Multi-modal Learning framework for D-AVN. In the Physical Alignment stage, the proposed Intensity-Calibrated Geometry Encoder (ICGE) performs physics-informed alignment of sound intensity with visual geometry to capture category-agnostic spatial correspondence. In the Scene Aggregation stage, a Transformer-based Cross-Modal Scene Aggregator (CMSA) dynamically integrates multi-modal features to construct a robust state representation for policy learning. Extensive experiments on Replica and Matterport3D demonstrate that our method achieves state-of-the-art success rates, while significantly improving generalization and navigation efficiency in complex environments and under unheard sound scenarios.