DreamFlow: Local Navigation Beyond Observation via Conditional Flow Matching in the Latent Space
Authors: Jiwon Park, Dongkyu Lee, I Made Aswin Nahrendra, Jaeyoung Lim, Hyun Myung
Organizations: Robotics Program, KAIST (Korea Advanced Institute of Science and Technology), Daejeon 34141, South Korea. · School of Electrical Engineering, KAIST (Korea Advanced Institute of Science and Technology), Daejeon 34141, South Korea. · URobotics, Seoul, South Korea. · KRAFTON, Seoul, South Korea. · Dept. of Electrical Engineering and Computer Science, UC Berkeley, USA.
Local navigation in cluttered environments often suffers from dense obstacles and frequent local minima. Conventional local planners rely on heuristics and are prone to failure, while deep reinforcement learning(DRL)based approaches provide adaptability but are constrained by limited onboard sensing. These limitations lead to navigation failures because the robot cannot perceive structures outside its field of view. In this paper, we propose DreamFlow, a DRL-based local navigation framework that extends the robot's perceptual horizon through conditional flow matching(CFM). The proposed CFM based prediction module learns probabilistic mapping between local height map latent representation and broader spatial representation conditioned on navigation context. This enables the navigation policy to predict unobserved environmental features and proactively avoid potential local minima. Experimental results demonstrate that DreamFlow outperforms existing methods in terms of latent prediction accuracy and navigation performance in simulation. The proposed method was further validated in cluttered real world environments with a quadrupedal robot. The project page is available at https://dreamflow-icra.github.io.
Figures & tables
Fig. 1 : An illustration of local navigation to the goal point (yellow star) in a cluttered environment with obstacles. (a) Local navigation with only a limited onboard sensing range (blue box ed region ) may lead to local minima when obstacles lie beyond the sensor range. (b) DreamFlow enables the robot to find optimal actions within the observed sensing range (blue box ed region ) by leveraging conditional flow matching-based prediction of extended latent representations (green box ed region ). The red and green lines indicate the executed trajectories, respectively.
Fig. 2 : The overall framework of DreamFlow. During deployment, the local height map ote (blue dots, indicating observable terrain) is encoded into the local latent vector zte . Subsequently, the pre-trained conditional flow matching predicts the extended latent vector z^te conditioned on the context vector ct encoded from the proprioceptive observation s otp . The predicted latent vector z^te represents information from the extended height map region (green dots, indicating terrain beyond sensor range). The navigation policy πnav takes the concatenated ct and z^te as input to produce the high-level velocity action at . The locomotion policy then generates the low-level joint actions to control the robot. When training the navigation policy, the privileged states stp and the extended latent vector ztE from the extended height map otE are used to train the critic network.
Fig. 3 : The conditional flow matching training pipeline. The training dataset is collected using pre-trained height map encoders ( He and HE ) and a proprioceptive encoder (CENet). A l ocal height map ote (blue boxed region ) is encoded into a local latent representation zte , and a privileged extended height map otE (green boxed region ) is encoded into an extended latent representation ztE . During training, conditional flow matching learns a velocity field vθ(τ,ct,z) , conditioned on the contextual vector ct , that transports zte at τ0 towards ztE at τ1 . Consequently, conditional flow matching maps zte (blue dots) to the predicted latent vector z^te (red dots), which align s with ztE (greed dots) in the latent space.
Fig. 4 : (a) The simulation environments for evaluating navigation performance. The Maze environment consists of multiple corners leading to the goal, while the Hallway environment consists of narrow passages. (b) Our quadruped robot, used in real-world experiments, is equipped with a processing unit and a sensor module mounted on its body.
Fig. 5 : Comparison of local navigation performance across three simulation scenarios: (a) Maze ( Easy ), (b) Maze ( Hard ), and (c) Hallway. Robot trajectories are depicted from a top view and overlaid on a height map, starting from initial positions (yellow markers) and ending at goal locations (red stars). Trajectories are color-coded according to time progression, with a gradient from dark blue (start) to cyan (goal). DreamFlow demonstrates smoother trajectories, with better obstacle avoidance and more efficient path selection (highlighted by the red box). In contrast, other methods show frequent obstacle contacts and often get stuck in local minima.
Architecture
Cos. [mean±std]
Time [ms](↓)
ID
OOD
MLP
0.928 ± 0.012
0.702 ± 0.136
0.0007
Diffusion [ 31 ]
0.911 ± 0.349
0.919 ± 0.380
0.0218
FM
0.873 ± 0.020
0.784 ± 0.105
0.0117
CFM (Ours)
0.980 ± 0.015
0.973 ± 0.017
0.0122
TABLE I : Comparison of latent prediction architectures on in-distribution and out-of-distribution test sets. Cos. represents cosine similarity between predicted and ground-truth extended latents, where higher values indicate better prediction quality. Inference time is averaged per sample. The best results are indicated in bold .
Fig. 6 : Real-world navigation experiments on the quadruped robot, comparing baseline and DreamFlow. Time progresses from left to right across four snapshots. Yellow and red markers indicate goals and collision events, respectively. (a)–(b) Narrow environment with tight passages: the baseline policy frequently collides with walls at corners, whereas DreamFlow achieves collision-free navigation by anticipating corridor layouts beyond immediate perception. (c)–(d) Cluttered environment with box obstacles and a wall segment: the baseline collides with the wall due to limited perception, while DreamFlow successfully navigates without collisions through predictive terrain modeling.
Navigating a drone in unseen and cluttered environments requires reliable generalization to unseen scene layouts and understanding of environmental structure relative to the robot's capabilities. Previous methods, which assume the same environment configuration, often rely heavily on human-designed perception pipelines and predefined rules to guide the robot toward the target. This process is environment-dependent and generalizes poorly across environments. Inspired by animal navigation behavior, we design a navigation framework that navigates with a reinforcement-learning-based policy on top of a world-model-based environment understanding to overcome these issues. In addition, a sparse reward function without hand-crafted shaping terms is designed to avoid local minima traps and encourage yaw control behaviors. In simulation and on real drones, our method exhibits emergent capabilities for navigating complex, unseen environments and escaping local optima where other methods fail. In challenging maps, it achieves a 5.3% higher navigation success rate than best baseline. Furthermore, the proposed framework achieves effective sim-to-real transfer without any tuning during deployment. The code will be publicly available.
Zian Liu, Andong Yang, Chunkai Yang +3
Institute for AI Industry Research, Tsinghua University, Beijing, China. · Department of Electronic Engineering, Tsinghua University, Beijing, China. · School of Remote Sensing and Information Engineering, Wuhan University, Wuhan, China.
End-to-end diffusion-based policies have demonstrated strong performance in mapless visual navigation, but their iterative denoising process introduces substantial inference latency, while behavior cloning limits performance to the quality of expert demonstrations. We present NavCMPO, a two-stage adaptive navigation framework that combines few-step MeanFlow trajectory generation, critic-guided refinement, and reinforcement learning fine-tuning. During pre-training, an obstacle proximity prediction task encourages the visual representation to capture obstacle-aware spatial information. To compensate for the degradation in obstacle avoidance caused by few-step generation, Critic-Guided Trajectory Refinement (CGTR) uses gradients from a critic trained with obstacle-point-cloud supervision to refine intermediate trajectories. During adaptation, the MeanFlow policy is fine-tuned using Proximal Policy Optimization with behavior-cloning regularization, while the critic is updated to accommodate embodiment-specific observation changes. Under a matched training budget on the InternVLA-N1 benchmark, NavCMPO achieves an average success rate of 74.7%, exceeding the retrained NavDP baseline by 6.4 percentage points, while reducing inference latency from 85,ms to 60,ms. Experiments on a Unitree Go2 further demonstrate effective sim-to-real transfer.
Junjie An, Yi Wu, Xiao Liu +5
Zhejiang University, Hangzhou, China · Shandong University
Existing language-conditioned navigation systems typically rely on modular pipelines or trajectory generators, but the latter use each scene--instruction annotation mainly to supervise one start-conditioned rollout. To address these limitations, we present CoFL, an end-to-end policy that maps a bird's-eye view (BEV) observation and a language instruction to a continuous flow field for navigation. CoFL reformulates navigation as workspace-conditioned field learning rather than start-conditioned trajectory prediction: it learns local motion vectors at arbitrary BEV locations, turning each scene--instruction annotation into dense spatial control supervision. Trajectories are generated from any start by numerical integration of the predicted field, enabling simple real-time rollout and closed-loop recovery. To enable large-scale training and evaluation, we build a dataset of over 500k BEV image--instruction pairs, each procedurally annotated with a flow field and a trajectory derived from semantic maps built on Matterport3D and ScanNet. Evaluating on strictly unseen scenes, CoFL significantly outperforms modular Vision-Language Model (VLM)-based planners and trajectory generation policies in both navigation precision and safety, while maintaining real-time inference. Finally, we deploy CoFL zero-shot in real-world experiments with BEV observations across multiple layouts, maintaining feasible closed-loop control and a high success rate.