Calibration of inertial and vision systems as a prelude to multi-sensor fusion

Effective integration of vision and inertial sensing can be used to overcome the classical issue of inertial divergence in inertial systems over time. However, for such integration, two types of calibration need to be performed, namely, (a) calibration to estimate the intrinsic properties of the vision cameras, such as the focal length and lens distortion parameters, and (b) determination of the transformation between the camera and the inertial system. While there are a variety of approaches to calibrate a camera optically, only a few accomplishments have been reported on calibration of cameras with inertial systems. Even the latter ones have been developed under indoor conditions. Calibration of a two sensor system under indoor conditions does not provide an appropriate and practical transformation for use in outdoor maneuvers due to invariable differences between outdoor and indoor conditions. Also, use of custom calibration objects in outdoor operational conditions is not feasible due to larger field of view that requires relatively large calibration object sizes. Hence calibration becomes a critical issue particularly if the integrated system is used in Intelligent Transportation Systems applications. In such cases it is essential that the two sensor system be calibrated in an outdoor setting in the final configuration of the vehicle. This paper introduces a calibration technique to determine the optimized transformation between the inertial and vision sensor systems in an outdoor setting. The derived intrinsic properties and the transformation between individual sensors are verified during two separate test runs on a section of an actual roadway. Finally, on a third roadway section the inertial and converted vision data are successfully compared with data from a manual survey performed.