ADAPT: Audio-Depth Action Planning with Transition-Aware Rewards for Lights-Out Industrial Navigation

Wenjun Huang; Shenghao Fu; Yian Jin; Yang Ni; Yifan Li; Lichi Li; Xingjian Diao; Pengfei Zhang

cs.AIOct 11, 2026

ADAPT: Audio-Depth Action Planning with Transition-Aware Rewards for Lights-Out Industrial Navigation

Authors: Wenjun Huang, Shenghao Fu, Yian Jin, Yang Ni, Yifan Li, Lichi Li, Xingjian Diao, Pengfei Zhang,

Organizations: University of California, Irvine · University of Pennsylvania · Carnegie Mellon University · Purdue University Northwest ·

Abstract

Vision-language-action models for embodied navigation typically rely on RGB observations, limiting their applicability in lights-out industrial environments where visible illumination may be unavailable. We study illumination-independent navigation using spatial audio for target localization and depth for obstacle-aware geometry. To support this setting, we introduce DarkNav-Bench, a benchmark with 24 industrial scenes and four progressively challenging acoustic conditions. We further propose ADAPT, an audio-depth action-planning framework equipped with spatial-audio perception and optimized for closed-loop navigation through reinforcement learning (RL). Our key RL contribution, transition-aware reward shaping (TARS), defines a pose-aware shortest-action potential over a position-heading graph. Unlike conventional geodesic-progress rewards, TARS accounts for both feasible movements and in-place rotations, providing action-consistent feedback for policy optimization. Experiments show that ADAPT consistently outperforms training-based navigation baselines and zero-shot multimodal models on both seen and unseen scenes. On unseen environments, it achieves 90.6% success under Clean conditions and retains 60.0% success under the most challenging Localized Noise setting. These results demonstrate the potential of spatial audio and depth as complementary modalities for reliable RGB-free navigation in lights-out industrial environments.

Figures & tables

Figure 1: Motivation of ADAPT. Existing RGB-based VLA agents depend on visible illumination and therefore require continuous lighting in lights-out factories. ADAPT instead combines spatial audio for target direction with depth sensing for local obstacle geometry, enabling reliable and energy-efficient navigation without visible light.

Figure 2: Overview of ADAPT . The model is trained in three stages: spatial-audio adaptation, multimodal SFT for audio-depth action prediction, and PPO-based closed-loop optimization with TARS. At inference time, the final policy directly maps spatial audio and depth observations to discrete navigation actions.

1. Oracle-assisted methods
Method	Clean				Prop				DN				LN
Method	SR $\uparrow$	SPL $\uparrow$	SNA $\uparrow$	FDG $\downarrow$	SR $\uparrow$	SPL $\uparrow$	SNA $\uparrow$	FDG $\downarrow$	SR $\uparrow$	SPL $\uparrow$	SNA $\uparrow$	FDG $\downarrow$	SR $\uparrow$	SPL $\uparrow$	SNA $\uparrow$	FDG $\downarrow$
1.1 Zero-shot (strict stopping)
Qwen 2.5-Omni Qwen (2025)	19.00	13.60	7.70	15.19	16.00	13.00	6.70	14.96	22.00	17.29	9.35	14.11	19.37	13.91	8.10	15.64
Qwen 3-Omni Xu et al. (2025)	72.00	71.68	59.48	6.00	73.00	72.76	60.87	5.18	70.00	69.83	58.02	5.50	70.20	69.78	59.31	6.53
2. Non-oracle methods

Table 1: Navigation performance in seen scenes.

1. Oracle-assisted methods (zero-shot)
Method	Clean				Prop				DN				LN
Method	SR $\uparrow$	SPL $\uparrow$	SNA $\uparrow$	FDG $\downarrow$	SR $\uparrow$	SPL $\uparrow$	SNA $\uparrow$	FDG $\downarrow$	SR $\uparrow$	SPL $\uparrow$	SNA $\uparrow$	FDG $\downarrow$	SR $\uparrow$	SPL $\uparrow$	SNA $\uparrow$	FDG $\downarrow$
1.1 Zero-shot (strict stopping)
Qwen 2.5-Omni Qwen (2025)	14.29	11.40	6.12	16.63	12.71	10.06	5.55	16.79	15.10	12.20	6.41	16.25	17.10	14.34	7.88	16.30
Qwen 3-Omni Xu et al. (2025)	62.14	61.79	52.12	6.36	62.38	61.93	52.87	6.40	63.33	62.92	53.27	6.16	62.62	62.24	52.70	6.31
2. Non-oracle methods

Table 2: Navigation performance in unseen scenes.

Figure 3: Qualitative comparison of navigation trajectories. Each panel reports the number of executed actions, FDG, and traveled path length. More examples are included in the appendix (see Fig. 6 ).

(a) SFT-only ablation

Component LN SAA Action Hist. * SR

\uparrow

SPL

\uparrow

SNA

\uparrow

FDG

\downarrow

\times

\times

52.07 42.86 30.84 12.59

\times

\checkmark

58.73 47.74 33.13 10.00

\checkmark

\times

53.27 46.29 31.89 12.15

\checkmark

\checkmark

62.93 52.58 35.42 9.00

(b) RL ablation

Component LN SAA TARS ** Action Hist. SR

\uparrow

SPL

\uparrow

SNA

\uparrow

FDG

\downarrow

\times

\times

\times

53.93 46.28 33.26 12.14

\times

\checkmark

\times

57.67 48.43 35.13 11.66

\times

\times

\checkmark

65.93 53.77 37.85 8.59

\times

\checkmark

\checkmark

66.87 55.50 39.58 7.65

\checkmark

\times

\times

56.60 50.05 35.04 12.00

\checkmark

\checkmark

\times

62.87 52.75 38.98 10.52

\checkmark

\times

\checkmark

66.43 56.14 38.38 7.67

\checkmark

\checkmark

\checkmark

67.40 56.51 38.22 7.54

Table 3: Ablation study under the LN setting. (a) SFT-only ablation. (b) RL ablation.

Metric	Clean	Prop	DN @ $20$ dB	DN @ $10$ dB	DN @ $0$ dB	DN @ $-10$ dB
MAE $\downarrow$	$35.45^{\circ}$	$\mathbf{25.03^{\circ}}$	$\underline{28.45^{\circ}}$	$32.02^{\circ}$	$34.06^{\circ}$	$34.17^{\circ}$
Median error $\downarrow$	$13.23^{\circ}$	$\underline{10.93^{\circ}}$	$\mathbf{10.74^{\circ}}$	$11.47^{\circ}$	$14.93^{\circ}$	$17.89^{\circ}$

Table 4: DoA estimation under different difficulties.

SNR (dB)	SR $\uparrow$	SPL $\uparrow$	SNA $\uparrow$	FDG $\downarrow$
10	67.10	59.30	39.00	7.89
0	61.90	51.00	32.60	9.18
-5	52.60	41.50	27.40	12.42
-10	31.20	23.70	14.40	18.39

Table 5: Navigation performance under different SNR under the DN setting.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Material	Abs.-L	Abs.-M	Abs.-H	Scat.	Trans.-L	Trans.-M	Trans.-H
Concrete	0.05	0.07	0.08	0.05	0.015	0.002	0.001
Metal	0.20	0.07	0.06	0.05	0.200	0.025	0.010
Wood	0.11	0.07	0.06	0.05	0.070	0.014	0.005
Plastic	0.03	0.04	0.05	0.05	0.080	0.030	0.015
Rubber	0.05	0.10	0.15	0.10	0.020	0.010	0.005

Appendix

Table 6: Acoustic material parameters used for geometry-dependent sound propagation. Abs.-L/M/H denote low-, mid-, and high-frequency absorption coefficients; Scat. denotes the scattering coefficient; and Trans.-L/M/H denote low-, mid-, and high-frequency transmission coefficients.

Source	SPL snd @1 m unweighted	SPL snd @1 m A-weighted	10–90% Energy Band	Energy in 500–900 Hz	Temporal Structure
Target	95.1	93.6	560–840 Hz	96.0%	50 ms sweep, 20 Hz
Conveyor	72.0	65.6	178–477 Hz	5.8%	steady hum
Forklift	68.0	58.4	110–265 Hz	0.15%	steady hum
Reach truck	66.0	60.6	183–443 Hz	5.6%	steady hum
AGV	62.0	60.7	551–1480 Hz	74.7%	steady hum

Appendix

Table 7: Acoustic characteristics of the target and localized equipment-noise sources. SPL snd values are nominal levels at 1 m under our waveform calibration. The 10–90% energy band denotes the frequency range containing the central 80% of spectral energy, and the overlap column reports the fraction of total energy within the target chirp band (500–900 Hz).

Figure 4: Comparison of geodesic progress and TARS. (a). In-place rotations leave geodesic distance unchanged, providing no feedback on heading changes. (b). TARS uses a position–heading action graph to measure the minimum remaining action count (

N(s)

). (c). With

\gamma_{\Phi}=1

, TARS rewards turns that reduce

N(s)

and penalizes those that increase it, distinguishing helpful from counterproductive rotations at the same position.

Input / Information	Zero-shot w/o Oracle Info.	Zero-shot w/ Oracle Info.
Current FOA audio	✓	✓
Current depth image	✓	✓
Recent action history	✓	–
Previous sensory observations	–	–
Ground-truth source bearing	–	✓
Ground-truth source distance	–	✓

Appendix

Table 8: Input information provided to the zero-shot multimodal baselines. Oracle bearing and distance are used only in the oracle-assisted diagnostic and are never available to ADAPT at inference time.

Figure 5: DoA estimation accuracy across difficulty levels. Accuracy is the percentage of predictions with angular error within

10^{\circ}

20^{\circ}

, or

45^{\circ}

of target direction.

Panel of Figure 6 — Figure 6: More qualitative comparison of navigation trajectories.

Robot Navigation

Robot navigation studies how robots perceive, localize, plan, and move safely toward goals in static or changing environments.

See all 554 papers

LH-AVLN: A Benchmark for Long-Horizon Audio-Visual-Language Navigation
Jul 4, 2026Rufeng Chen, Yue Chang, Zili Shao +4Embodied Navigation Audio-Visual Navigation
The Hong Kong University of Science and Technology (Guangzhou) · Jilin University
BCNav: Bearing-Conditioned Depth Policies for Sound Source Navigation
Sep 29, 2026Yaozhong Kang, Jiang Wang, Takeshi Ashizawa +2Robot Policy Learning Robot Navigation
Department of Systems and Control Engineering, Institute of Science Tokyo, Japan
Generalizable Audio-Visual Navigation via Binaural Difference Attention and Action Transition Prediction
Apr 6, 2026Jia Li, Yinfeng YuAudio-Visual Navigation Embodied Navigation
Joint Research Laboratory for Embodied Intelligence, Xinjiang University · Joint International Research Laboratory of Silk Road Multilingual Cognitive Computing, Xinjiang University · School of Computer Science and Technology, Xinjiang University, Urumqi 830017, China

Jul 4, 2026J/K move · Enter open · S save

LH-AVLN: A Benchmark for Long-Horizon Audio-Visual-Language Navigation

Rufeng Chen, Yue Chang, Zili Shao, Zhaofan Zhang+3

The Hong Kong University of Science and Technology (Guangzhou) · Jilin University

Embodied Navigation Audio-Visual Navigation Long-Horizon Agent Evaluation

Open paper AskPDF arXiv

Embodied navigation is moving toward long-horizon missions, yet existing long-horizon benchmarks are largely acoustically silent, and audio-visual navigation tasks typically focus on a single goal. We introduce LH-AVLN, a benchmark for Long-Horizon Audio-Visual-Language Navigation that combines multi-goal mission execution, heterogeneous goal specifications, and persistent spatialized acoustic cues. In LH-AVLN, an agent receives a global mission of two to four goals specified by category, language description, or reference image, and navigates with RGB-D observations, pose, and binaural audio in indoor 3D environments. The benchmark supports both ordered and unordered missions, where alternating goal-associated sounds can guide non-line-of-sight search but may also become distractors as mission progress changes. We further develop PAG-Nav, a training-free reference agent that maintains a temporal uniform semantic map and performs progressive goal-state planning, using sound for search while reserving completion for visual-semantic verification. Experiments show that existing vision-language, memory-based, and audio-visual agents struggle to complete full LH-AVLN missions, and that PAG-Nav provides a stronger diagnostic baseline while leaving substantial room for future progress.

Abstract

Robot Navigation

Explore similar work

LH-AVLN: A Benchmark for Long-Horizon Audio-Visual-Language Navigation

BCNav: Bearing-Conditioned Depth Policies for Sound Source Navigation

Generalizable Audio-Visual Navigation via Binaural Difference Attention and Action Transition Prediction