A Novel perspective invariant feature transform for RGB-D images

RGB-D cameras have been attracting increasing researches for solving traditional problems in the domain of computer vision and robotics. Among the existing local features, most are proposed for the color channel or depth channel separately, while little attention has been paid to designing new composite features based on the physical characteristics. In this work, we propose a novel perspective invariant feature transform (PIFT) for RGB-D images. We integrate the color and depth information together making full use of the intrinsic characteristics of the two types of information to enhance the robustness and adaptability to large spatial variations of local appearance. The depth information is used to project the feature patch to its tangent plane to make it consistent with different views. It also helps to filter out the “fake keypoints” which are unstable in 3D space. Binary descriptors are then generated in the feature patches using a color coding method. Experiments on publicly available RGB-D datasets show that the proposed method has the best precision and the second best recall rate comparing against state-of-the-art local features, when applied to feature matching with large spatial variations.