Fundamental matrix estimation: A study of error criteria

The fundamental matrix (FM) describes the geometric relations that exist between two images of the same scene. Different error criteria are used for estimating FMs from an input set of correspondences. In this paper, the accuracy and efficiency aspects of the different error criteria are studied. We mathematically and experimentally proved that the most popular error criterion, the symmetric epipolar distance, is biased. It was also shown that despite the similarity between the algebraic expressions of the symmetric epipolar distance and Sampson distance, they have different accuracy properties. In addition, a new error criterion, Kanatani distance, was proposed and proved to be the most effective for use during the outlier removal phase from accuracy and efficiency perspectives. To thoroughly test the accuracy of the different error criteria, we proposed a randomized algorithm for Reprojection Error-based Correspondence Generation (RE-CG). As input, RE-CG takes an FM and a desired reprojection error value d. As output, RE-CG generates a random correspondence having that error value. Mathematical analysis of this algorithm revealed that the success probability for any given trial is 1 − (2/3)2 at best and is 1 − (6/7)2 at worst while experiments demonstrated that the algorithm often succeeds after only one trial.

Research highlights
► The article mathematically and experimentally compares the approximate error criteria that are used in the process of fundamental matrix (FM) estimation. The comparison is made from both the accuracy and efficiency perspectives.
► It is mathematically and experimentally proved that the most popular error criterion, the symmetric epipolar distance (SED), is biased.
► It is also shown that despite the similarity between the algebraic expressions of SED and Sampson distance (RE1), they have different accuracy properties.
► A new error criterion, Kanatani distance (REK), is proposed and is shown to be the most effective, from the accuracy and efficiency perspectives, for use during the outlier removal phase of FM estimation.
► The article proposes a randomized algorithm (RE-CG) for synthetic data set generation. As input, RE-CG accepts an FM F and a desired reprojection error d. As output, RE-CG produces a correspondence whose reprojection error, measured relative to F, is equal to d.