Query, Align, and Distill: Navigation-Aware Cross-Modal Interaction for Efficient Vision-and-Language Navigation

Zhihao Chen; Yiyuan Ge; Ziyang Wang; Pu Cao; Lu Yang

Fig. 1: Accuracy–efficiency trade-off on R2R (test unseen). We plot success rate (SR) against model computation (GFLOPs). Our compact student achieves near-teacher performance with substantially lower computation. Bubble size represents GFLOPs, with the 5G–300G reference bubbles indicating the scale.

Fig. 2: Overview of Query, Align, and Distill. Following the global-planning and local-acting paradigm in Vision-and-Language Navigation, history-based global candidates support long-range planning, while current-view local candidates support fine-grained action selection. The Navigable Query Generator extracts action-sufficient global and local queries, which the Instruction–Query Aligner grounds with the instruction for policy prediction. The shallower student is trained by Navigation-aware Token-adaptive Distillation and policy distribution distillation during fine-tuning. Blue and orange dashed lines denote global and local query distillation, respectively. Flame and snowflake icons mark trainable and frozen modules, respectively.

	Validation Seen				Validation Unseen				Test Unseen
Method	SR $\uparrow$	SPL $\uparrow$	OSR $\uparrow$	NE $\downarrow$	SR $\uparrow$	SPL $\uparrow$	OSR $\uparrow$	NE $\downarrow$	SR $\uparrow$	SPL $\uparrow$	OSR $\uparrow$	NE $\downarrow$	Param (M)↓	GFLOPs↓
HOP+ ‡ [ 46 ]	78	73	-	2.33	67	61	-	3.49	66	60	-	3.71	164.62	50.77
KERM ‡ [ 47 ]	80	74	-	2.19	72	61	-	3.22	70	59	-	3.61	222.04	169.46
GeoVLN ‡ [ 26 ]	79	76	-	2.22	68	63	-	3.35	65	61	-	3.95	182.64	296.65
DSRG ‡ [ 48 ]	81	76	88	2.23	73	62	81	3.00	72	61	78	3.33	188.87	102.85
BEVBert ‡ [ 49 ]	81	74	88	2.17	75	64	84	2.81	73	62	81	3.13	181.08	294.32

TABLE I: Comparison with state-of-the-art methods on R2R. Pink, yellow, and orange indicate the best, second-best, and third-best results, respectively, and “–” denotes unavailable results.

\ddagger

marks Params/GFLOPs measured from released code under our protocol, whereas

\dagger

marks values reported in the original papers under the same protocol.

	Validation Seen				Validation Unseen				Test Unseen
Method	SR↑	SPL↑	RGS↑	RGSPL↑	SR↑	SPL↑	RGS↑	RGSPL↑	SR↑	SPL↑	RGS↑	RGSPL↑
BEVBert [ 49 ]	73.72	65.32	57.70	51.73	51.78	36.37	34.71	24.44	52.81	36.41	32.06	22.09
GOAT [ 36 ]	78.64	71.40	63.74	57.85	53.37	36.70	38.43	26.09	57.72	40.53	38.32	26.70
EAM [ 53 ]	72.59	64.96	57.83	51.80	48.00	35.65	32.72	24.22	-	-	-	-
WPCL [ 56 ]	79.71	71.39	-	-	54.03	37.41	-	-	-	-	-	-
CONSOLE [ 57 ]	74.14	65.15	60.08	52.69	50.07	34.40	34.05	23.33	55.13	37.13	33.18	22.25

TABLE II: Comparison with state-of-the-art methods on REVERIE. Pink, yellow, and orange indicate the best, second-best, and third-best results, respectively, and “–” denotes unavailable results.

	Validation Unseen				Test Unseen
Method	SR↑	SPL↑	OSR↑	RGSPL↑	SR↑	SPL↑	OSR↑	RGSPL↑
DUET [ 18 ]	36.28	22.58	50.91	3.75	33.44	21.42	43.00	4.17
GridMM [ 19 ]	37.46	24.81	53.39	3.91	36.27	21.25	48.02	4.15
GOAT [ 36 ]	40.35	28.05	54.69	5.85	40.50	25.18	50.63	6.10
WPCL [ 56 ]	42.92	26.57	52.21	-	-	-	-	-
NavQ [ 58 ]	39.09	26.65	58.79	5.51	38.59	24.50	48.92	4.48

TABLE III: Comparison with state-of-the-art methods on SOON. Pink, yellow, and orange indicate the best, second-best, and third-best results, respectively, and “–” denotes unavailable results.

	Validation Seen				Validation Unseen
Method	SR $\uparrow$	SPL $\uparrow$	nDTW $\uparrow$	sDTW $\uparrow$	SR $\uparrow$	SPL $\uparrow$	nDTW $\uparrow$	sDTW $\uparrow$
HOP+ [ 46 ]	53.6	47.9	59.0	43.0	45.7	38.4	52.0	36.0
ADAPT [ 60 ]	50.3	44.6	56.3	40.6	46.9	40.2	54.1	37.7
CLEAR-C [ 61 ]	-	-	-	-	46.0	40.1	57.2	38.7
MAR M-MP [ 62 ]	-	-	-	-	50.2	-	60.3	43.9
PETL [ 63 ]	60.5	56.8	65.7	51.7	57.9	54.2	64.9	49.7

TABLE IV: Comparison with state-of-the-art methods on RxR-English. Pink, yellow, and orange indicate the best, second-best, and third-best results, respectively, and “–” denotes unavailable results.

	Test Unseen (R2R)				Validation Unseen (RxR-English)
Method	SR↑	SPL↑	OSR↑	NE↓	SR↑	SPL↑	nDTW↑	sDTW↑
Ours-Teacher	80.43 $\pm$ 0.04	71.98 $\pm$ 0.02	86.14 $\pm$ 0.04	2.40 $\pm$ 0.03	77.1 $\pm$ 0.01	69.8 $\pm$ 0.2	73.4 $\pm$ 0.3	63.0 $\pm$ 0.2
Ours-Student	79.10 $\pm$ 0.05	69.53 $\pm$ 0.03	83.21 $\pm$ 0.05	2.57 $\pm$ 0.01	73.1 $\pm$ 0.02	67.2 $\pm$ 0.1	69.8 $\pm$ 0.2	61.2 $\pm$ 0.1

TABLE V: Teacher–student performance on R2R test-unseen and RxR-English validation-unseen. Values are reported as mean

\pm

standard deviation over five runs with seeds

\{0,1,2,3,4\}

.

	Test Unseen (REVERIE)				Test Unseen (SOON)
Method	SR↑	SPL↑	RGS↑	RGSPL↑	SR↑	SPL↑	OSR↑	RGSPL↑
Ours-Teacher	60.18 $\pm$ 0.04	44.20 $\pm$ 0.02	41.90 $\pm$ 0.02	29.62 $\pm$ 0.02	43.43 $\pm$ 0.01	28.04 $\pm$ 0.01	53.45 $\pm$ 0.04	7.32 $\pm$ 0.02
Ours-Student	59.01 $\pm$ 0.03	43.12 $\pm$ 0.01	40.32 $\pm$ 0.01	28.24 $\pm$ 0.03	41.04 $\pm$ 0.01	27.13 $\pm$ 0.02	51.36 $\pm$ 0.02	6.98 $\pm$ 0.03

TABLE VI: Teacher–student performance on the goal-oriented REVERIE and SOON test-unseen splits. Values are reported as mean

\pm

standard deviation over five runs with seeds

\{0,1,2,3,4\}

.

	R2R (Test Unseen)				SOON (Test Unseen)
Interaction design	SR $\uparrow$	SPL $\uparrow$	OSR $\uparrow$	NE $\downarrow$	SR $\uparrow$	SPL $\uparrow$	OSR $\uparrow$	RGSPL $\uparrow$
DUET-style dense interaction	77.91	68.12	84.31	2.68	40.02	25.47	50.03	6.73
Shared generic queries	79.21	69.21	84.99	2.54	40.54	25.68	50.74	6.95
QAD (Full)	80.43	71.98	86.16	2.41	43.42	28.03	53.44	7.31

TABLE VII: Controlled comparison of teacher-side cross-modal interaction designs on R2R and SOON test-unseen. All non-interaction components and training settings are held fixed.

Fig. 3: Jetson runtime and query-number ablations on R2R. (a) Step throughput (steps/s) measured on Jetson Nano for representative VLN agents, including HOP+, BEVBert, GOAT, and our QAD teacher/student. (b) R2R test-unseen SR and SPL versus the number of navigable queries per branch

K_{b}

, where

K_{L}=K_{G}=K_{b}

Fig. 4: Sensitivity to the distillation hyper-parameter

\alpha

. We sweep the

\alpha

that balances query-level distillation

\mathcal{L}_{\mathrm{NTD}}

and policy KL distillation

\mathcal{L}_{\pi}

during fine-tuning :

\mathcal{L}_{D}=\alpha\,\mathcal{L}_{\mathrm{NTD}}+(1-\alpha)\,\mathcal{L}_{\pi}

. Results are reported on R2R ( test-unseen : SR/SPL) and REVERIE ( test-unseen : SR/SPL). A moderate

\alpha

yields the best trade-off, while overly small/large

\alpha

over-emphasizes policy- or query-level supervision, respectively.

		Test Unseen
Option	Model	SR↑	SPL↑	OSR↑	NE↓
1	Teacher (full)	80.43	71.98	86.16	2.41
2	Teacher w/o NQG	78.69	68.67	84.91	2.61
3	Teacher w/o IQA	77.13	67.92	83.11	2.74
4	Student (full)	79.12	69.54	83.21	2.57
5	Student w/o NTD	70.23	61.44	72.14	3.71

TABLE VIII: Component and distillation ablations on R2R test-unseen.

		Test Unseen
Option	Model	SR↑	SPL↑	OSR↑	RGSPL↑
1	Teacher (full)	43.42	28.03	53.44	7.31
2	Teacher w/o NQG	40.11	25.51	50.16	6.96
3	Teacher w/o IQA	39.54	24.87	49.45	6.54
4	Student (full)	41.03	27.11	51.34	6.97
5	Student w/o NTD	37.54	25.04	48.26	5.54

TABLE IX: Component and distillation ablations on SOON test-unseen.

Fig. 5: Qualitative trajectory comparison on R2R. The top-down graphs show navigation trajectories, the right panels show final viewpoints, and the corresponding instruction is provided below each episode. Yellow, green, and pink trajectories denote our student, the ground truth, and GOAT [ 36 ] , respectively. Green boxes with check marks indicate correct grounding by our student, whereas red boxes indicate incorrect grounding by GOAT. Our student follows the instructions more faithfully, avoids detours, and reaches the correct destinations.

					Test Unseen
Option	N(text)	N(pano)	N(NQG)	N(IQA)	SR↑	SPL↑	Param (M)↓	GFLOPs↓
1 (Teacher)	6	2	4	3	80.43	71.98	150.78	45.56
2	6	2	2	1	80.01	71.21	129.62	38.87
3	3	1	4	3	79.81	70.12	52.81	15.57
4	3	1	2	1	79.32	69.94	31.65	8.06
5 (Student)	1	1	1	1	79.12	69.54	9.56	1.92

TABLE X: Architecture-scaling ablation on R2R test-unseen. We vary the depths of the text encoder, panoramic encoder, Navigable Query Generator, and Instruction–Query Aligner. Performance and efficiency are evaluated using SR, SPL, parameter count, and GFLOPs.

		Test Unseen
Option	Model	SR↑	SPL↑	OSR↑	NE↓
1	Global Only	75.41	65.13	80.81	3.05
2	Local Only	74.22	64.34	80.23	3.11
3	Both (Full)	79.12	69.54	83.21	2.57

TABLE XI: Distillation-target ablation on R2R test-unseen, comparing global-only, local-only, and joint global/local navigable-query distillation.

		Test Unseen
Option	Model	SR↑	SPL↑	OSR↑	RGSPL↑
1	Global Only	37.94	23.37	47.64	4.94
2	Local Only	38.61	24.51	49.01	5.11
3	Both (Full)	41.03	27.11	51.34	6.97

TABLE XII: Distillation-target ablation on SOON test-unseen, comparing global-only, local-only, and joint global/local navigable-query distillation.

Fig. 6: Step-wise visualization of the navigation process. Each row shows the panoramic observation at one navigation step, and the input instruction is provided at the bottom with key cues highlighted in red. In the left example, the green check and red cross denote correct and incorrect turning decisions, respectively. In the right example, the green and red boxes indicate correctly and incorrectly grounded visual evidence, respectively. Our student follows the highlighted cues more accurately and reaches the destination with a shorter trajectory than GOAT [ 36 ] .

Fig. 7: Teacher–student failure cases. The teacher succeeds while the compact student fails. The left case shows a destination-room grounding error: the student fails to associate “pool room / pool table” with the correct visual evidence. The right case shows a premature STOP error: the student stops once exercise equipment becomes visible, while the teacher moves to the correct viewpoint specified by “next to the exercise equipment.”

Fig. 8: PCA visualization of teacher–student query alignment. Global and local navigable-query slots (

K{=}32,D{=}768

) are projected into a shared 2D PCA space across three training stages. Blue circles and orange crosses denote teacher and student slots, respectively, and each line connects a same-index pair. Shorter links indicate stronger slot-wise alignment.

Fig. 9: Cosine similarity matrices for teacher–student query slots. Each matrix

\mathbf{S}\in\mathbb{R}^{K\times K}

(

K{=}32

) measures

\mathbf{S}_{ij}=\cos(\mathbf{q}^{T}_{i},\mathbf{q}^{S}_{j})

. All panels share the same color scale. Stronger diagonal responses indicate better same-index alignment, whereas off-diagonal responses suggest slot permutation or mixing. Distillation progressively sharpens the diagonal structure for both global and local queries.

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Query, Align, and Distill: Navigation-Aware Cross-Modal Interaction for Efficient Vision-and-Language Navigation

Abstract

Figures & tables

Vision-Language Navigation

What do VLM-Based Vision-Language Navigation Models Rely on: Interpreting and Steering Policy Behavior

NavJev: Efficient Vision-Language Navigation via Action-Centric Visual Compression and Discriminative Action-Semantic Memory

FutureNav: Unified World-Action Modeling for Vision-and-Language Navigation

Abstract

Vision-Language Navigation

Explore similar work

What do VLM-Based Vision-Language Navigation Models Rely on: Interpreting and Steering Policy Behavior

NavJev: Efficient Vision-Language Navigation via Action-Centric Visual Compression and Discriminative Action-Semantic Memory

FutureNav: Unified World-Action Modeling for Vision-and-Language Navigation