Research › Analysis

VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator

Qin, T., Li, P. & Shen, S. · IEEE Trans. Robot. 34, 1004–1020 (2018) · Paper

Q2 Computing analysis. All mathematical results and empirical findings are attributed to the original authors. We present our reading of the work as it relates to robotic automation.

What this paper solves

GPS-denied localization for mobile platforms requires fusing camera and IMU data without accumulated drift. VINS-Mono provides a complete, tightly-coupled monocular visual-inertial odometry pipeline: IMU preintegration for fast prediction, feature-based visual updates, sliding window bundle adjustment for local consistency, and optional loop closure for global drift correction. It is the most widely deployed open-source VIO baseline in the robotics field and is used directly as the feature frontend in Deep Drone Acrobatics (#12).

Key mathematical framework

IMU preintegration

Rather than integrating raw IMU measurements at every camera frame and re-integrating when IMU biases change, VINS-Mono preintegrates IMU measurements in the body frame between consecutive keyframes $b_{k}$ and $b_{k + 1}$ . The preintegrated terms are relative translation, velocity change, and rotation:

α_{b_{k + 1}}^{b_{k}} = \iint_{t \in [t_{k}, t_{k + 1}]} R_{t} (\overset{a}{^}_{t} - b_{a} - n_{a}) d t^{2}

β_{b_{k + 1}}^{b_{k}} = \int_{t \in [t_{k}, t_{k + 1}]} R_{t} (\overset{a}{^}_{t} - b_{a} - n_{a}) d t

γ_{b_{k + 1}}^{b_{k}} = \int_{t \in [t_{k}, t_{k + 1}]} \frac{1}{2} Ω (\overset{ω}{^}_{t} - b_{g} - n_{g}) γ_{t}^{b_{k}} d t

where $\overset{a}{^}_{t}$ and $\overset{ω}{^}_{t}$ are raw accelerometer and gyroscope measurements, $b_{a}, b_{g}$ are slowly varying biases, and $n_{a}, n_{g}$ are white noise terms. These preintegrated quantities can be updated cheaply when bias estimates change without re-integration.

IMU measurement residual

Given state estimates at keyframes $b_{k}$ and $b_{k + 1}$ , the IMU residual vector measures discrepancy between predicted and preintegrated motion:

r_{B} (z_{b_{k + 1}}^{b_{k}}, X) = R_{b_{k}}^{w}^{⊤} (p_{b_{k + 1}}^{w} - p_{b_{k}}^{w} - v_{b_{k}}^{w} Δ t + \frac{1}{2} g^{w} Δ t^{2}) - α_{b_{k + 1}}^{b_{k}} R_{b_{k}}^{w}^{⊤} (v_{b_{k + 1}}^{w} - v_{b_{k}}^{w} + g^{w} Δ t) - β_{b_{k + 1}}^{b_{k}} 2 [γ_{b_{k + 1}}^{b_{k}}^{- 1} \otimes R_{b_{k}}^{w}^{⊤} R_{b_{k + 1}}^{w}]_{x y z} δ b_{a} δ b_{g}

Visual measurement residual

For a landmark $l$ observed from camera frame $c_{j}$ , the visual residual is the reprojection error in normalized image coordinates:

r_{C} (z_{l}^{c_{j}}, X) = z_{l}^{c_{j}} - π (R_{c}^{b}^{⊤} R_{b_{j}}^{w}^{⊤} (p_{l}^{w} - p_{b_{j}}^{w}) - R_{c}^{b}^{⊤} p_{c}^{b})

where $π (\cdot)$ is the normalized projection function and $R_{c}^{b}, p_{c}^{b}$ are the camera-IMU extrinsic calibration.

Sliding window bundle adjustment

VINS-Mono maintains a sliding window of the most recent keyframes and optimizes the full state vector $X$ by minimizing the sum of IMU and visual residuals weighted by their respective covariances:

X min ⎩ ⎨ ⎧ ∥ r_{p} - H_{p} X ∥^{2} + k \in B \sum r_{B} (z_{b_{k + 1}}^{b_{k}}, X)_{P_{b_{k + 1}}^{b_{k}}}^{2} + (l, j) \in C \sum r_{C} (z_{l}^{c_{j}}, X)_{P_{l}^{c_{j}}}^{2} ⎭ ⎬ ⎫

The term $∥ r_{p} - H_{p} X ∥^{2}$ is the prior from marginalized states, maintained via the Schur complement to keep the window size bounded. This is the tightly-coupled formulation: IMU and visual measurements are jointly optimized rather than fused sequentially.

Empirical results

EuRoC benchmark: VINS-Mono achieves centimeter-level position accuracy on all EuRoC MAV sequences, competitive with or exceeding stereo and RGB-D systems using only a monocular camera and IMU
Initialization: Fully automatic. Estimates camera-IMU extrinsics and IMU biases online without pre-calibration
Loop closure: DBoW2-based place recognition with pose graph optimization eliminates long-term drift in environments with revisited regions
Deployment: Real-time on an Intel NUC CPU. Used as the feature frontend in Deep Drone Acrobatics and as the localization baseline in EGO-Planner and EGO-Swarm

What this means for robotic automation

VINS-Mono is the standard against which camera-dependent localization is measured. Its preintegration formulation is the correct way to handle asynchronous IMU and camera data in a sliding window estimator: IMU provides high-frequency prediction, camera provides low-frequency but drift-free correction, and the preintegrated terms allow both to be jointly optimized in a single nonlinear least-squares problem.

The sensor fusion architecture here is the transferable result. Camera input improves accuracy and precision over inertial sensing alone, and this benefit holds even at low resolution when the visual task is terrain conformation evaluation rather than high-fidelity feature tracking. Low-resolution frames are computationally cheap to process, which makes them compatible with edge-constrained deployments where full VIO pipelines are too expensive. The camera does not need to run a SLAM pipeline to be useful: it only needs to produce observations that correlate with the terrain prior.

VINS-Mono's deployment in Deep Drone Acrobatics as a feature track frontend rather than a full localization system illustrates this principle. Harris corner detection and Lucas-Kanade tracking are computationally simple relative to bundle adjustment, yet they provide enough scene-geometry information to satisfy the abstraction lemma's condition for minimizing the sim-to-real gap.