On intensity-based 3D visual servoing

• 3D visual servoing without feature extraction or matching.
• A family of new techniques is proposed.
• Various experimental results.
• Common beliefs are refuted.
• New research directions are opened up.

This article investigates the problem of pose-based visual servoing whose equilibrium state is defined via a reference image. Differently from most solutions, this work directly exploits the pixel intensities without any feature extraction or matching. Intensity-based methods provide for higher accuracy and versatility. Another central idea of this work concerns the exploitation of the observability issue associated to monocular systems, which always occurs around the equilibrium. This overall framework allows for developing a family of new 3D visual servoing techniques with varying degrees of computational complexity and of prior knowledge. Importantly, they are arranged hierarchically from the simplest to the optimal ones, all in a unified scheme. Three new methods are then presented, and their closed-loop performances are experimentally assessed using a six degree-of-freedom robotic arm. As an additional contribution, these results refute the common belief that correct camera calibration and pose recovery are crucial to the accuracy of 3D visual servoing techniques.