Learning High-Speed Flight in the Wild

What this paper solves

High-speed flight through unstructured natural environments (forests, orchards, mountain terrain) presents a different problem than structured drone racing. There are no gates to detect, no known course geometry, and the obstacle structure is dense, irregular, and unpredictable. Prior systems required either 3D maps built in advance or sensors (LiDAR, depth cameras) that are too heavy or power-intensive for small platforms.

This paper trains a policy to fly at up to 10 m/s through previously unseen natural environments using only a forward-facing depth camera and an IMU. The key insight is that the policy does not need to reconstruct a 3D map: it maps directly from depth observations to body-rate commands, compressing the perception-planning-control pipeline into a single forward pass at 25 Hz.

Key mathematical framework

Privileged expert training

The training follows the same privileged learning pattern as Deep Drone Acrobatics (#12). A privileged expert ${\pi }^{\ast }$ has access to the full 3D point cloud from simulation and plans collision-free trajectories using sampling-based motion planning. The student policy ${\pi }_{\theta }$ observes only simulated depth images and learns to imitate the expert via DAgger:

where $o(s)$ is the depth image observable from state $s$ and $u^{\ast }={\pi }^{\ast }(s)$ is the expert body rate command. The student never sees the 3D point cloud at test time.

Uncertainty estimation

The policy outputs both a mean action and an uncertainty estimate. Using Monte Carlo dropout, the predicted uncertainty at test time is:

where ${\hat{u}}_m$ is the action sampled from the $m$-th stochastic forward pass and $\bar{u}=M^{-1}{\sum }_m{\hat{u}}_m$ is the mean action. When ${\hat{\sigma }}^2$ exceeds a threshold, the system reduces commanded speed, trading safety for speed.

Network architecture

The policy consists of two components operating asynchronously. A recurrent convolutional encoder processes depth images at 25 Hz:

The encoded state $z_t$ is concatenated with IMU measurements and fed to an MLP that outputs body rates and collective thrust. The recurrent unit maintains implicit state across frames, allowing the policy to reason about motion-induced depth changes.

Empirical results

• Speed: Successful flight at up to 10 m/s through previously unseen forest environments without any pre-built map
• Sensors: Forward-facing depth camera at 25 Hz and IMU only, no LiDAR, no GPS, no motion capture
• Zero-shot transfer: Policy trained entirely in simulation transfers to physical environments not seen during training
• Uncertainty behavior: The system autonomously slows down in high-uncertainty regions (dense obstacles, texture-poor zones) and accelerates in open areas
• Success rate: 80% course completion in natural environments at 7 m/s average speed, versus 0% for a model-based baseline without a map

What this means for robotic automation

The paper demonstrates that a single lean recurrent policy can subsume perception, planning, and control for high-speed navigation in unstructured environments. Constructing a map, planning a path through it, and executing that path are three sequential steps each of which can fail independently. Compressing all three into one network trained end-to-end eliminates the failure interfaces between them.

The uncertainty-based speed modulation is the correct failure mode for a navigation system: when the policy is uncertain, it slows down rather than maintaining speed and crashing. This is a safety-by-design property emergent from the uncertainty estimation, not from an explicit speed limiter. Any navigation system targeting real-world deployment should expose a calibrated uncertainty estimate for exactly this purpose.