Fusion of visual odometry and inertial navigation system on a smartphone

•We developed the monocular visual odometry algorithm.•We developed the inertial navigation system based on a pedometer and digital compass.•Both localization methods were fused with the Extended Kalman Filter.•The whole algorithm was implemented and tested on a smartphone.

The paper presents the monocular visual odometry, inertial navigation system and the fusion of both these localization approaches. The visual odometry algorithm consists of four other algorithms, namely the camera calibration algorithm, KLT algorithm, algorithm for the estimation of rigid transformation and RANSAC algorithm. The inertial navigation system is based on a pedometer and digital compass. Both visual odometry and the inertial navigation system can determine the incremental movements and the positions of a robot or a pedestrian according to the world coordinate system. In order to get an even more robust and accurate localization system, the advantages of each mentioned localization approaches were combined by using the Extended Kalman Filter. The algorithms were fully implemented on a smartphone, where they were divided into several threads that could be performed simultaneously on multiple processor cores. The proposed system, which fuses information from the camera and inertial sensors, can convert the smartphone into a powerful mobile sensor unit or the so-called virtual sensor that returns relative position in relation to the starting point. This virtual sensor can be used as an advanced sensor unit on mobile robots or as part of a smartphone application which requires personal navigation system. The operation of the localization system is proved by experimental results which were obtained by attaching a smartphone on a pedestrian who walked along the reference trajectory drawn on the floor. In the experiments the described system showed big potential in many aspects since very good results were obtained.