Robust multi-view L2 triangulation via optimal inlier selection and 3D structure refinement

• The uncertainty of epipolar transfer is analytically characterized.
• Two new algorithms to calculate the error bounds of 3D structure are proposed.
• A new robust formulation of direct multi-view triangulation is proposed.
• Our method can identify more inliers than existing robust triangulation strategies.
• Our method can reconstruct more unambiguous 3D points with higher accuracy.

This paper presents a new robust approach for multi-view L2 triangulation based on optimal inlier selection and 3D structure refinement. The proposed method starts with estimating the scale of noise in image measurements, which affects both the quantity and the accuracy of reconstructed 3D points but is overlooked or ignored in existing triangulation pipelines. A new residual-consensus scheme within which the uncertainty of epipolar transfer is analytically characterized by deriving its closed-form covariance is developed to robustly estimate the noise scale. Different from existing robust triangulation pipelines, the issue of outliers is addressed by directly searching for the optimal 3D points that are within either the theoretical correct error bounds calculated by second-order cone programming (SOCP) or the efficiently calculated approximate ranges. In particular, both the inlier selection and 3D structure refinement are realized in an optimal fashion using Differential Evolution (DE) optimization which allows flexibility in the design of the objective function. To validate the performance of the proposed method, extensive experiments using both synthetic data and real image sequences were carried out. Comparing with state-of-the-art robust triangulation strategies, the proposed method can consistently identify more reliable inliers and hence, reconstruct more unambiguous 3D points with higher accuracy than existing methods.