Learning Sidewalk Autopilot from Multi-Scale Imitation with Corrective Behavior Expansion

Honglin He; Yukai Ma; Brad Squicciarini; Wayne Wu; Bolei Zhou

cs.ROMar 23, 2026

Learning Sidewalk Autopilot from Multi-Scale Imitation with Corrective Behavior Expansion

Authors: Honglin He, Yukai Ma, Brad Squicciarini, Wayne Wu, Bolei Zhou

Organizations: Department of Computer Science, University of California, Los Angeles · Coco Robotics

Abstract

Sidewalk micromobility is a promising solution for last-mile transportation, but current learning-based control methods struggle in complex urban environments. Imitation learning (IL) learns policies from human demonstrations, yet its reliance on fixed offline data often leads to compounding errors, limited robustness, and poor generalization. To address these challenges, we propose a framework that advances IL through corrective behavior expansion and multi-scale imitation learning. On the data side, we augment teleoperation datasets with diverse corrective behaviors and sensor augmentations to enable the policy to learn to recover from its own mistakes. On the model side, we introduce a multi-scale IL architecture that captures both short-horizon interactive behaviors and long-horizon goal-directed intentions via horizon-based trajectory clustering and hierarchical supervision. Real-world experiments show that our approach significantly improves robustness and generalization in diverse sidewalk scenarios.

Figures & tables

Fig. 1 : This work aims to utilize corrective behavior expansion and multi-scale prediction to learn an autopilot model for sidewalk micromobility.

Fig. 2 : Illustration of the MIMIC framework . The model adopts an encoder–decoder architecture that combines coarse historical embeddings with fine-grained current visual observations as context. The context encoder converts the observation sequence by combining the coarse flattened features of historical frames with the fine patch-level features of the current frame, together with the goal point and camera features. The action decoder leverages time-horizon-specific anchors to produce actions parameterized by GMMs across multiple horizons, thereby enhancing the output’s diversity and robustness.

Fig. 3 : Illustration of the corrective behavior expansion. We first estimate the depth sequence and reconstruct a point cloud. Given the 3D point cloud, we perturb the trajectory using a deviation–recovery noise sequence. Then we synthesize corresponding observation-action pairs.

Fig. 4 : Illustration of the sensor augmentation. A pretrained relighting model is used to modify the scene guided by different lighting prompts. The original scenario is segmented into foreground and background regions where different relighting parameters are applied. The outputs are then blended to synthesize novel relighted observations.

Fig. 5 : Qualitative results of MIMIC on the CoS test set. The green trajectory denotes the one with the highest probability, while the others represent the top-6 trajectories filtered by non-maximum suppression (NMS).

	minADE 1s $\downarrow$	minFDE 1s $\downarrow$	mAP $\uparrow$	L2 1s $\downarrow$	L2 2s $\downarrow$
GNM ‡	0.594	0.988	-	0.988	-
ViNT ‡	0.638	1.056	-	1.056	-
NoMaD ‡	0.523	0.858	0.216	1.072	2.182
MBRA	0.617	1.019	-	1.019	2.034
CityWalker	0.648	1.125	-	1.125	-
ViNT*	0.247	0.425	-	0.425	0.925

TABLE I : Open-loop evaluation on CoS-Regular .

	minADE 1s $\downarrow$	minFDE 1s $\downarrow$	mAP $\uparrow$	L2 1s $\downarrow$	L2 2s $\downarrow$
MBRA	2.586	3.297	-	3.297	7.000
CityWalker	0.929	1.740	-	1.740	-
CityWalker*	0.398	0.695	-	0.695	1.397
MIMIC	0.196	0.328	0.565	0.645	1.348

TABLE II : Open-loop evaluation on CoS-Recovery .

$I$	$S$	$M$	$L$	$QF$	minADE 1s $\downarrow$	minFDE 1s $\downarrow$	mAP $\uparrow$	L2 2s $\downarrow$	L2 8s $\downarrow$
				$\checkmark$	0.188	0.371	-	0.789	4.083
	$\checkmark$				0.067	0.113	0.310	0.596	-
			$\checkmark$		0.081	0.132	0.670	0.711	3.805
$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$		0.074	0.135	0.680	0.637	4.068
$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	0.071	0.129	0.695	0.700	3.718

TABLE III : Ablation study on model design.

I,S,M,L

denote the prediction head at immediate, short, medium, and long horizons, respectively, and

QF

denotes the prediction head derived directly from the context features.

$\mathcal{D}_{S}$	$\mathcal{D}_{C}$	mAP $\uparrow$	L2 1s $\downarrow$	L2 2s $\downarrow$	L2 4s $\downarrow$	L2 8s $\downarrow$
		0.660	0.381	0.789	1.677	4.139
$\checkmark$		0.670	0.358	0.748	1.617	3.940
	$\checkmark$	0.679	0.355	0.754	1.621	4.028
$\checkmark$	$\checkmark$	0.695	0.342	0.700	1.434	3.718
$\checkmark$		0.374	0.914	1.949	4.817	9.137
$\checkmark$	$\checkmark$	0.565	0.645	1.348	4.499	8.562

TABLE IV : Ablation study on data expansions.

\mathcal{D}_{S}

denotes the set from sensor augmentation, and

\mathcal{D}_{C}

denotes the set from corrective behavior expansion.

Fig. 6 : Qualitative results of MIMIC in the real-world.

	Goal Reaching $\uparrow$	Pedestrian Avoidance $\uparrow$	SPL $\uparrow$	Intervention Times $\downarrow$
CityWalker	0.55	0.18	0.46	19
CityWalker*	0.70	0.29	0.44	11
MIMIC	0.90	0.76	0.69	4

TABLE V : Closed-loop evaluation in the real world.

Imitation Learning

Imitation learning trains agents to perform tasks by learning from expert demonstrations, feedback, or observed behavior.

See all 263 papers

From Imitation to Alignment: Human-Preference Flow Policies for Long-Horizon Sidewalk Navigation
Jun 10, 2026Honglin He, Zhizheng Liu, Yukai Ma +1Robot Navigation Social Robot Navigation
University of California, Los Angeles
Sim-to-Real Aware End-to-End Learning Environment for Micromobility
Sep 24, 2026Shouma Amano, Takuya AzumiEnd-to-End Autonomous Driving Sim-to-Real Transfer
Saitama University · Academic Association (Graduate School of Science and Engineering) Saitama University
Human-like autonomy emerges from self-play and a pinch of human data
Jun 11, 2026Daphne Cornelisse, Julian Hunt, Zixu Zhang +4Self-Play RL Autonomous Driving
NYU Tandon School of Engineering · NYU Courant · Princeton University +2

Jun 10, 2026J/K move · Enter open · S save

From Imitation to Alignment: Human-Preference Flow Policies for Long-Horizon Sidewalk Navigation

Honglin He, Zhizheng Liu, Yukai Ma, Bolei Zhou

University of California, Los Angeles

Robot Navigation Social Robot Navigation Imitation Learning

Open paper AskPDF arXiv

Autonomous long-horizon sidewalk navigation is essential for micro-mobility applications such as robotic food delivery and assistive electronic wheelchairs. Unlike autonomous driving on the road, long-horizon sidewalk navigation requires precise maneuvering through unpredictable sidewalk terrains and pedestrians, with a lightweight perception stack as minimal as a single monocular RGB camera. While imitation learning (IL) from demonstrations offers a practical solution, the resulting autopilot policy often suffers from compounding errors, a lack of social compliance on sidewalks, and deficiencies in counterfactual reasoning to handle complex situations. To address these challenges, we introduce FlowPilot, a mapless navigation policy that achieves robust and efficient long-horizon navigation performance using only a monocular RGB camera. We first propose to use anchored flow matching as an action representation for policy pre-training on large-scale robot fleet data and to capture the diverse, complex, multimodal distribution of sidewalk navigation behaviors. To bridge the gap between imitation and alignment, we further design a human-in-the-loop preference learning scheme to tune the policy on a small amount of human intervention data. It strengthens the model's counterfactual reasoning and social compliance on sidewalks. We evaluate FlowPilot through extensive simulation and real-world experiments in diverse sidewalk environments. FlowPilot achieves 42% success rate and 66% route completion in simulation, while FlowPilot-HP further improves real-world robustness and social compliance, reducing IR by 40.0% and NIR by 52.1% relative to the base model.

Abstract

Imitation Learning

Explore similar work

From Imitation to Alignment: Human-Preference Flow Policies for Long-Horizon Sidewalk Navigation

Sim-to-Real Aware End-to-End Learning Environment for Micromobility

Human-like autonomy emerges from self-play and a pinch of human data