Monocular vision based 6D object localization for service robot’s intelligent grasping

Intelligent grasping is still a hard problem for home service robots. There are two major issues in the intelligent grasping, i.e. the object recognition and the pose estimation. To grasp casually placed objects, the robot needs the object’s full 6 degrees of freedom pose data. To deal with the challenges such as illumination changes, cluttered background, occlusion, etc., we propose a monocular vision based object recognition and 6D pose estimation method. The SIFT feature point matching and brute-force search algorithm is used to do a tentative object recognition. The object recognition result is then verified with the homography constraint. After passing the verification, the 6D pose estimation is obtained through the decomposition of the homography matrix and the result is refined using the Levenberg–Marquardt algorithm. We embed our pose estimation method in a tracking by detection framework to keep computing and refining the pose during the whole approaching procedure. To test our method, a robot arm of seven degrees of freedom was utilized for a group of grasping experiments. The experimental results showed that our approach successfully recognized and grasped a variety of household objects with decent accuracy.