Enhancing feature tracking with gyro regularization *

• A 3-axis gyro mounted to a video camera can frequently predict the main component of optical flow.
• The gyro-predicted flow can be used to regularize feature tracking to help track ambiguous features through bad imagery.
• Gyro regularization does not require all features to belong to a rigid scene.
• Gyro regularization adds very little computational cost to feature tracking.
• The common practice of using gyros to initialize trackers offers no advantage over careful optical-only initialization.

We present a deeply integrated method of exploiting low-cost gyroscopes to improve general purpose feature tracking. Most previous methods use gyroscopes to initialize and bound the search for features. In contrast, we use them to regularize the tracking energy function so that they can directly assist in the tracking of ambiguous and poor-quality features. We demonstrate that our simple technique offers significant improvements in performance over conventional template-based tracking methods, and is in fact competitive with more complex and computationally expensive state-of-the-art trackers, but at a fraction of the computational cost. Additionally, we show that the practice of initializing template-based feature trackers like KLT (Kanade–Lucas–Tomasi) using gyro-predicted optical flow offers no advantage over using a careful optical-only initialization method, suggesting that some deeper level of integration, like the method we propose, is needed in order to realize a genuine improvement in tracking performance from these inertial sensors.

Figure optionsDownload high-quality image (419 K)Download as PowerPoint slide