CoFL: Continuous Flow Fields for Language-Conditioned Navigation

Haokun Liu; Zhaoqi Ma; Yicheng Chen; Masaki Kitagawa; Wentao Zhang; Zicen Xiong; Jinjie Li; Moju Zhao

cs.ROMar 3, 2026

CoFL: Continuous Flow Fields for Language-Conditioned Navigation

Authors: Haokun Liu, Zhaoqi Ma, Yicheng Chen, Masaki Kitagawa, Wentao Zhang, Zicen Xiong, Jinjie Li, Moju Zhao

Abstract

Existing language-conditioned navigation systems typically rely on modular pipelines or trajectory generators, but the latter use each scene--instruction annotation mainly to supervise one start-conditioned rollout. To address these limitations, we present CoFL, an end-to-end policy that maps a bird's-eye view (BEV) observation and a language instruction to a continuous flow field for navigation. CoFL reformulates navigation as workspace-conditioned field learning rather than start-conditioned trajectory prediction: it learns local motion vectors at arbitrary BEV locations, turning each scene--instruction annotation into dense spatial control supervision. Trajectories are generated from any start by numerical integration of the predicted field, enabling simple real-time rollout and closed-loop recovery. To enable large-scale training and evaluation, we build a dataset of over 500k BEV image--instruction pairs, each procedurally annotated with a flow field and a trajectory derived from semantic maps built on Matterport3D and ScanNet. Evaluating on strictly unseen scenes, CoFL significantly outperforms modular Vision-Language Model (VLM)-based planners and trajectory generation policies in both navigation precision and safety, while maintaining real-time inference. Finally, we deploy CoFL zero-shot in real-world experiments with BEV observations across multiple layouts, maintaining feasible closed-loop control and a high success rate.

Figures & tables

Symbol	Domain	Description
Problem Setup
$\mathcal{E}$	$[0,1]^{2}$	Normalized BEV-aligned workspace.
$\mathcal{X}_{\mathrm{free}},\ \mathcal{X}_{\mathrm{obs}}$	$\subseteq\mathcal{E}$	Free-space and obstacle subsets.
$I$	$\mathbb{R}^{H\times W\times 3}$	RGB BEV input (top-down/high-oblique).
$\ell$	text	Language instruction.
$\mathbf{x}_{0},\ \tau$	$\mathcal{X}_{\mathrm{free}},\ [0,1]\!\to\!\mathcal{E}$	Start state and trajectory.

TABLE I : Notation used in this paper.

Fig. 2 : Overview of the CoFL’s network architecture. Given an RGB BEV observation $I$ and a language instruction $\ell$ , a SigLIP 2-based [ 34 , 35 ] vision–language encoder produces language-conditioned context tokens over the BEV image. The decoder then queries this context with 2D normalized spatial coordinates $\mathbf{X}$ and outputs the corresponding velocities $\mathbf{V}$ , forming a continuous flow field $\mathbf{v}(\mathbf{x}\mid I,\ell)$ via bilinear interpolation of discrete velocity predictions.

Fig. 3 : Overview of the trajectory inference. The predicted flow field over the workspace guides agents from different starts toward the same goal while smoothly avoiding obstacles.

Fig. 4 : Overview of the proposed visual observation generation pipeline. Images are captured from a multi-view camera array. (a)-(b) Top-down view RGB and semantic segmentation maps. (c)-(d) Oblique view sample.

Dataset	# Scenes	# Regions	# Views	# Samples
Matterport3D [ 7 ]	90	2,056	9	245,604
ScanNet [ 8 ]	1,513	1,513	9	342,038

TABLE II : Statistics of the Implemented Datasets.

Fig. 5 : Overview of the procedural annotation pipeline. We combine cost-weighted geodesic distance and obstacle repulsion to derive the field and trajectory annotations.

Fig. 6 : Overview of the training/validation split. We employ a strict scene-wise split to assess generalization.

Fig. 7 : Examples of trajectories on validation set. (a) Matterport3D: “Please navigate to the sofa in the upper right” ; (b) ScanNet: “Please go to the toilet” . From left to right: ground truth, pure VLM/VLM+Planner, DP-family and CoFL. More examples are provided in Appendix D .

Method	Head params (M)	Section A $\ast$				Section B $\dagger$				Section C $\ddagger$				Field quality		Lat. (ms)
Method	Head params (M)	FGE $\downarrow$	CR $\downarrow$	Curv	PLR	FGE $\downarrow$	CR $\downarrow$	Curv	PLR	FGE $\downarrow$	CR $\downarrow$	Curv	PLR	AE $\downarrow$	ME $\downarrow$	Lat. (ms)
	Matterport3D (A: 18 regions, 2635 samples; B: 20 regions, 1879 samples; C: 22 regions, 3421 samples)
Pure VLM	API	0.22	0.90	0.03	0.84	0.25	0.91	0.03	1.13	0.22	0.81	0.03	1.07	–	–	–
VLM+Planner	API	0.24	0.84	0.06	1.05	0.25	0.90	0.06	1.35	0.22	0.80	0.06	1.08	–	–	–
DP-C-DDPM	74.02	0.17	0.86	0.06	0.86	0.19	0.79	0.05	0.94	0.16	0.72	0.05	0.88	–	–	17.62
DP-T-DDPM	81.21	0.17	0.86	0.08	0.84	0.18	0.80	0.08	0.96	0.16	0.68	0.08	0.87	–	–	19.28

TABLE III : Navigation performance and field quality. We report trajectory metrics (detailed in § V-A2 ) aggregated over the validation set for all baselines and CoFL. Field metrics are reported for CoFL only.

Panel of Figure 10 — Fig. 8 : Latency–performance trade-offs of real-time models (DP-family and CoFL). DP-family: {1, 2, 3, 5 , 7, 10, 15, 20} denoising steps (horizon 100); CoFL: {50, 75, 100 , 125, 150, 175, 200} grid size (100-step Euler rollout). Underlines mark the operating points reported in Table III .

$L$	Encoder param (M)	FGE $\downarrow$	CR $\downarrow$	AE $\downarrow$	ME $\downarrow$	Lat. (ms)
2	395.28	0.14	0.22	18.73 ∘	0.06	15.08
4	414.18	0.13	0.22	18.16 ∘	0.06	15.81
6	433.08	0.13	0.24	18.76 ∘	0.06	16.21

TABLE IV : Encoder Fusion Depth

$\tilde{L}$	Head param (M)	FGE $\downarrow$	CR $\downarrow$	AE $\downarrow$	ME $\downarrow$	Lat. (ms)
1	8.27	0.14	0.23	18.55 ∘	0.06	12.41
2	15.36	0.13	0.22	18.16 ∘	0.06	15.81
3	22.45	0.14	0.22	18.01 ∘	0.06	20.69

TABLE V : Decoder Query Depth

$d$	Overall param (M)	FGE $\downarrow$	CR $\downarrow$	AE $\downarrow$	ME $\downarrow$	Lat. (ms)
512	399.63	0.13	0.26	18.67 ∘	0.07	12.29
768	429.54	0.13	0.22	18.16 ∘	0.06	15.81
1024	471.25	0.13	0.22	17.90 ∘	0.06	22.55

TABLE VI : Hidden Dimension

Variant	Head param (M)	FGE $\downarrow$	CR $\downarrow$	AE $\downarrow$	ME $\downarrow$	Lat. (ms)
W/O	15.36	0.13	0.22	17.89 ∘	0.07	15.93
With	20.09	0.14	0.23	18.30 ∘	0.07	49.34

TABLE VII : Self-Attention in CoFL Decoder

Variant	FGE $\downarrow$	CR $\downarrow$	Curv	PLR
Stabilized rescaling ( $\frac{\mathbf{v}}{(1-t_{k})+\beta\,t_{k}^{\alpha}}$ )	0.13	0.22	0.11	0.87
No stabilizer ( $\frac{\mathbf{v}}{1-t_{k}}$ )	0.75	1.00	0.18	4.57
Unit-speed rollout ( $\mathbf{v}/\\|\mathbf{v}\\|$ )	0.12	0.57	1.68	3.70

TABLE VIII : Ablation of rollout parameterization.

Panel of Figure 16 — Fig. 9 : Example of query distributions (in yellow) for the supervision coverage ablation. Workspace supervises the full BEV domain, Tube supervises a corridor around the reference path, and On-trajectory supervises only the demonstrated trajectory.

Variant	FGE $\downarrow$	CR $\downarrow$	Curv	PLR	AE $\downarrow$	ME $\downarrow$
Full workspace	0.13	0.22	0.11	0.87	18.73 ∘	0.06
Tube	0.12	0.61	1.66	3.70	24.14 ∘	0.08
On-trajectory	0.34	0.91	0.52	2.48	71.71 ∘	0.46
DP-T-DDPM	0.16	0.78	0.08	0.91	–	–

TABLE IX : Ablation of supervision coverage. All variants use the same model and query budget, differing only in where training queries are sampled.

Fig. 10 : Example trajectories from real-world experiments. (a)–(d) E1 : from left to right, instructions are “ move to the right side of the chair ”, “ move to the left side of the table ”, “ move to the back side of the box ”, and “ move to the front side of the right chair ”. (e)–(h) E2 : three static layouts and one reactive adversary, all with the same instruction “ move to the chair ”. Trajectories are replanned at each control step from the latest observation and visualized in the figure.

Setting	Case	On-Target $\uparrow$	CR $\downarrow$	PN (m)	CLR (m)	TTS (s)	PL (m)
E1: W/O Obstacles	C1: Target = chair	1.00	–	–	–	14.13	2.83
	C2: Target = table	0.80	–	–	–	13.37	2.79
	C3: Target = box	0.60	–	–	–	33.88	4.79
	C4: Target = chair pair (language disambiguation)	1.00	–	–	–	18.33	4.04
	Avg	0.85	–	–	–	18.67	3.52
E2: With Obstacles	C5: Target = chair , Static layout A	1.00	0.20	0.12	0.04	22.69	3.78

TABLE X : Real-world target navigation under two settings. Each setting contains four task cases (C1–C8; 5 trials per case). PN denotes penetration depth in collided trials, while CLR denotes the minimum obstacle clearance in non-collision trials.

Panel of Figure 20 — Fig. 11 : Four collision trials drawn from E2 as examples. Trajectories are colored by elapsed time from start to end. The static obstacle is shown with diagonal hatching, and the moving adversary is shown with a dashed orange outline. The deepest-penetration frame is highlighted by the red robot outline, with the penetration depth annotated.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Encoder	Trajectory denoiser	Sampler
DP-C-DDPM	Vision–Language Encoder same as CoFL	Temporal U-Net (1D Conv; FiLM)	DDPM (stochastic)
DP-T-DDPM	Vision–Language Encoder same as CoFL	Transformer (token-conditioning)	DDPM (stochastic)
DP-C-Flow/ODE	Vision–Language Encoder same as CoFL	Temporal U-Net (1D Conv; FiLM)	ODE (deterministic)

Appendix

TABLE XI : DP-family baselines used in this work . All variants reuse the same SigLIP-based vision–language encoder and differ in the denoiser architecture and sampler.

Fig. 12 : More examples of trajectories on validation set. From left to right: ground truth, pure VLM/VLM+Planner, DP-family and CoFL. (a-b) ScanNet: “please walk to the door” and “can you approach the bathtub” ; (c-d) ScanNet: “go to ahead of the table” and “can you approach right side of the table” ; (e-f) Matterport3D: “can you proceed to back of the chair in the lower right” and “please travel to the chair in the lower left” .

Panel of Figure 23 — Fig. 13 : Effect of pseudo receding-horizon segmentation $n$ on DP (DP-T-DDPM, 5 denoising steps, fixed seed 42). The red horizontal line indicates CoFL with a fixed $100\times 100$ query grid.

Vision-Language Navigation

Vision-Language Navigation studies how embodied agents use natural-language instructions and visual observations to plan and execute routes.

See all 310 papers

CoFL-S: Spatially Queryable Sector Flow Fields for Local Language-Conditioned Navigation
Jul 2, 2026Haokun Liu, Zhaoqi Ma, Yicheng Chen +5Robotic Control Robot Navigation
Dragon Lab, Department of Mechanical Engineering, The University of Tokyo
FlowDec: Temporal Conditional Flow Decorruptor for Robust Continuous Vision-Language Navigation
Jun 21, 2026Yufei Zhang, Changhao ChenVLM Robustness Vision-Language Navigation
The Hong Kong University of Science and Technology (Guangzhou), China
LiveVLN: Breaking the Stop-and-Go Loop in Vision-Language Navigation
Apr 21, 2026Xiangchen Wang, Weiye Zhu, Teng Wang +5Vision-Language Navigation Action Chunking
Southern University of Science and Technology · Harbin Institute of Technology, Shenzhen · Spatialtemporal AI

Jul 2, 2026J/K move · Enter open · S save

CoFL-S: Spatially Queryable Sector Flow Fields for Local Language-Conditioned Navigation

Haokun Liu, Zhaoqi Ma, Yicheng Chen, Wentao Zhang+4

Dragon Lab, Department of Mechanical Engineering, The University of Tokyo

Robotic Control Robot Navigation Vision-Language Navigation Vision-Language-Action Models Language-Conditioned Robot Control

Open paper AskPDF arXiv

Vision-Language Navigation has increasingly emphasized high-level instruction reasoning, memory, global map construction, and instruction decomposition, while the low-level action representation remains comparatively underexplored. We propose CoFL-S, a low-level vision-language-action framework that predicts a language-conditioned flow field over the robot's local visible sector and generates continuous trajectories by rolling out the predicted field. To train this low-level representation, we convert each VLN-CE episode, originally a whole-episode instruction paired with an action sequence, into frame-level local supervision with aligned sub-instructions and matched action, trajectory, and dense flow-field targets. For evaluation, we introduce a continuous-time Habitat benchmark that isolates low-level action interfaces from instruction decomposition and executes all methods through a shared velocity-command controller, enabling decomposition-independent closed-loop comparison across different planner frequencies rather than fixed discrete forward-and-turn transitions in VLN-CE. Under matched encoders and training settings, CoFL-S consistently outperforms action-token and action-chunk baselines across planner frequencies in the continuous-time Habitat benchmark, and zero-shot real-world closed-loop deployment further shows its advantage over both baselines beyond simulation.

Abstract

Vision-Language Navigation

Explore similar work

CoFL-S: Spatially Queryable Sector Flow Fields for Local Language-Conditioned Navigation

FlowDec: Temporal Conditional Flow Decorruptor for Robust Continuous Vision-Language Navigation

LiveVLN: Breaking the Stop-and-Go Loop in Vision-Language Navigation