Research › Analysis

Perception-Aware Receding Horizon Navigation for MAVs

Zhang, Z. & Scaramuzza, D. · ICRA 2534–2541 (IEEE, 2018) · Paper

Q2 Computing analysis. All mathematical results and empirical findings are attributed to the original authors. We present our reading of the work as it relates to robotic automation.

What this paper solves

Visual-inertial odometry fails when the camera cannot track enough visual features, which happens when the platform points away from textured surfaces, moves too fast for the feature tracker, or enters poorly lit regions. Standard trajectory planners treat the camera as a passive sensor and plan purely for dynamical feasibility, ignoring whether the resulting trajectory will degrade VIO quality.

This paper integrates perception quality directly into the trajectory optimization. The planner actively chooses trajectories that maintain good feature density in the camera field of view, preventing VIO failure before it happens. This is a co-optimization of motion and perception rather than treating them as independent layers.

Key mathematical framework

Perception quality model

Let $F (x)$ be a scalar measure of feature density in the camera field of view from state $x = (p, q)$ . One natural choice is the number of map points visible:

F (x) = ∣ {l \in M : π (R_{c}^{w} (q) (l - p) + p_{c}) \in [0, W] \times [0, H]} ∣

where $M$ is the sparse map, $π$ is the camera projection, $R_{c}^{w} (q)$ is the world-to-camera rotation from attitude $q$ , and $[0, W] \times [0, H]$ is the image plane. Higher $F$ means more features visible and better VIO quality.

Perception-aware MPC objective

The trajectory optimizer minimizes a composite objective that penalizes both tracking error and perception quality loss:

u_{0 : N} min k = 0 \sum N [∥ x_{k} - x_{k}^{r} ∥_{Q}^{2} + ∥ u_{k} ∥_{R}^{2} - α F (x_{k})]

subject to quadrotor dynamics $x_{k + 1} = f (x_{k}, u_{k})$ and feasibility constraints on thrust and angular rate. The term $- α F (x_{k})$ rewards trajectories that maintain high feature visibility, with $α$ balancing tracking performance against perception quality.

Receding horizon execution

The problem is solved at each timestep over a short horizon $N Δ t$ (typically 1-2 s) and only the first control in the sequence is applied. The horizon is short enough that the local feature map is approximately constant, making the perception quality term $F (x_{k})$ computable without global map updates during the optimization.

Empirical results

VIO survival: Perception-aware trajectories maintain active feature tracking in environments where standard MPC trajectories lose tracking completely within the mission window
Mission success: Navigation tasks that require pointing away from textured surfaces are completed successfully with the perception objective active; standard planners fail on the same tasks
Computational cost: Perception quality gradient is computed analytically by differentiating through the visibility model, adding modest overhead to the MPC solve time

What this means for robotic automation

Perception-aware planning makes explicit what is implicit in most navigation stacks: the localization system has performance requirements that depend on where the platform looks, and trajectory planning must respect those requirements. A system that navigates correctly but positions its camera where it cannot track features will fail when VIO loses tracking.

The architectural principle transfers directly: any system with a sensing modality whose quality depends on the platform state should include sensing quality as a term in the trajectory optimization. For anomaly-based localization, this means choosing trajectories that maximize anomaly observation rate rather than purely minimizing navigation cost. The planner and the sensor confidence model are coupled, not independent layers.