Event cameras offer high temporal resolution and low latency, making them ideal sensors for high-speed robotic applications where conventional cameras suffer from motion blur. However, their widespread adoption in robot learning is severely bottlenecked by the computational cost of simulating high-frequency event data during online training. In this work, we present Approximate Imitation Learning, a novel framework that fundamentally resolves this bottleneck, reducing policy training time for complex, agile drone flight from 52.44 hours to just 1.86 hours - a 28x computational speedup. Our key insight is to separate representation learning from policy search. We first leverage a large-scale offline dataset to learn a task-specific representation space. Subsequently, the policy is fine-tuned through online interactions that rely solely on lightweight state information, completely eliminating the need to render events during the active policy search phase. This training paradigm drastically reduces development overhead and enables event-based control policies to scale to complex environments. Furthermore, our approach eliminates the reliance on standard cameras or intermediate representations during deployment, mapping events directly to control commands. In simulation, our method matches or exceeds the performance of standard imitation learning baselines that require full online event rendering. Finally, we successfully validate the framework in the real world, demonstrating that a policy trained via this ultra-efficient paradigm enables a quadrotor to fly through highly cluttered environments at remarkable speeds of up to 9.8 m/s.
Figures & tables
Figure 2: During offline training, teacher actions aT supervise the event encoder FS and the shared action decoder A . The approximate student, using teacher observations, is trained by aligning its features h^S and actions a^S with the features hS and actions aS of the event student. During online training, teacher observations obtained from lightweight simulations are used to fine-tune the shared action decoder A . Consistency losses in both the action and feature spaces are applied during an additional offline alignment step. For clarity, the auxiliary decoder Do is omitted.
Figure 2Figure 3
Poisson Delta: 0.04
Poisson Delta: 0.05
Methods
Training Input
Success Rate
Mean Velocity [ sm ]
Success Rate
Mean Velocity [ sm ]
BC
Off. Event
0.80
8.53
0.70
8.13
DAgger [ 42 ]
On. Event
0.30
6.66
0.10
5.72
BC+DAgger
Off. Event + On. Event
1.00
9.00
0.90
8.47
Approx IL (Ours)
Off. Event + On. State
1.00
8.76
0.90
8.53
State Teacher (Ours)
On. State + Dist. Map
1.00
8.77
0.90
8.60
Table 1: Simulation results for methods using offline (Off.) and/or online (On.) training input.
Table 5
Figure 7: Simulation results with dynamic obstacles moving randomly at 0.5ms−11.5ms−1 .
Figure 8: The simulation environment consists of tall cylindrical obstacles placed in a natural scene (a). The scene is captured by a simulated event camera mounted on the quadrotor (b). For reference, the corresponding grayscale images are also shown (c).
Figure 9: The event student receives events in the form of event representations, which are first encoded into features hSti by the event student encoder FS . The student encoder FS consists of a recurrent EfficientNet [ 53 ] followed by the Projection Layers . These event features are then concatenated with the outputs of the Vector Layers , which encode auxiliary inputs, i.e., the direction command vˉti , the previous action aSti−1 , and, optionally, the state information sti . The combined features are fused in the Fusion Layer , and the final actions aSti are produced by the Action Head .
Figure 10: Our vectorized event generation first quantizes log intensities into bands and computes band differences between adjacent timesteps. Neighboring non-zero values are then differenced again. Events are triggered where this second subtraction yields zero.
Figure 11: The maximum and mean GPU memory usage and the required computation time are shown for our vectorized event generation (Vectorized) and ESIM’s GPU-based event generation (ESIM) across different numbers of environments. Our method leads to a 34% reduction in mean runtime (top) and significantly lowers peak GPU memory usage (bottom).
Figure 12: Our proposed method yields a policy that consistently flies longer trajectories in simulation than BC, the closest baseline applicable in the same training setting.
Table 12
Figure 13: The ground truth distance maps and corresponding predictions are shown for the M3ED dataset (A), simulation data (B), and the real-world test environment (C). Importantly, the distance maps and associated auxiliary decoders are used only during pretraining and are not involved in action inference of the event student.
Figure 14: The platform used in the real-world experiments features an NVIDIA Jetson TX2 running the event-based student, which processes events from a DVXplorer Micro sensor.
Obstacle avoidance is a fundamental vision-based task essential for enabling quadrotors to perform advanced applications. When planning the trajectory, existing approaches both on optimization and learning typically regard quadrotor as a point-mass model, giving path or velocity commands then tracking the commands by outer-loop controller. However, at high speeds, planned trajectories sometimes become dynamically infeasible in actual flight, which beyond the capacity of controller. In this paper, we propose a novel end-to-end policy that directly maps depth images to low-level bodyrate commands by reinforcement learning via differentiable simulation. The high-fidelity simulation in training after parameter identification significantly reduces all the gaps between training, simulation and real world. Analytical process by differentiable simulation provides accurate gradient to ensure efficiently training the low-level policy without expert guidance. The policy employs a lightweight and the most simple inference pipeline that runs without explicit mapping, backbone networks, primitives, recurrent structures, or backend controllers, nor curriculum or privileged guidance. By inferring low-level command directly to the hardware controller, the method enables full flight envelope control and avoids the dynamic-infeasible issue.Experimental results demonstrate that the proposed approach achieves the highest success rate and the lowest jerk among state-of-the-art baselines across multiple benchmarks. The policy also exhibits strong generalization, successfully deploying zero-shot in unseen, outdoor environments while reaching speeds of up to 7.5m/s as well as stably flying in the super-dense forest. This work is released at https://github.com/Fanxing-LI/avoidance.
Fanxing Li, Shengyang Wang, Yuxiang Huang +5
Shanghai Jiao Tong University · Kunshan Duke University
Safe quadrotor navigation in cluttered and dynamic environments depends not only on instantaneous geometric perception, but more critically on anticipating collision risks induced by relative motion. Conventional modular pipelines frequently suffer from perception latency, while end-to-end learning methods relying on implicit scalar rewards often struggle to extract reliable spatio-temporal features without physics-grounded supervision. To address this, we propose an anticipatory risk-guided reinforcement learning framework. Leveraging privileged simulator states, we construct a directionally aligned future collision risk map based on the Closest Point of Approach (CPA). Through an asymmetric actor-critic architecture, the network is trained to self-predict this structured risk, which explicitly guides the visual policy during deployment. A lightweight spatio-temporal encoder extracts motion cues directly from onboard depth sequences, bypassing explicit object tracking or optical flow estimation. Extensive simulated and real-world experiments demonstrate that our method effectively improves safety margins and flight efficiency in dense dynamic clutters compared to existing baselines. Furthermore, the learned policy achieves robust zero-shot Sim-to-Real transfer on a physical quadrotor, relying purely on abstracted spatio-temporal depth sequences and its self-predicted risk priors, validating the effectiveness of our approach and its robust generalization from simulation to reality.
Yuchao Mei, Guohao Zhang, Luxia Ai +2
National Key Laboratory of Science and Technology on Multi-spectral Information Processing, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China
We present QuadAgent, a training-free agent system for agile quadrotor flight guided by vision-language inputs. Unlike prior end-to-end or serial agent approaches, QuadAgent decouples high-level reasoning from low-level control using an asynchronous multi-agent architecture: Foreground Workflow Agents handle active tasks and user commands, while Background Agents perform look-ahead reasoning. The system maintains scene memory via the Impression Graph, a lightweight topological map built from sparse keyframes, and ensures safe flight with a vision-based obstacle avoidance network. Simulation results show that QuadAgent outperforms baseline methods in efficiency and responsiveness. Real-world experiments demonstrate that it can interpret complex instructions, reason about its surroundings, and navigate cluttered indoor spaces at speeds up to 5 m/s.