Structure and motion from line correspondences: Representation, projection, initialization and sparse bundle adjustment

• The Cayley representation of spatial line is developed.
• A novel line projection function is derived under the perspective camera model.
• An incremental initialization approach is designed to boot the bundle adjustment.
• The sparse bundle adjustment is employed to achieve higher efficiency and accuracy.

We address the problem of structure and motion from line correspondences, which ranges from the representation of lines, their projections and the initialization procedure to the final adjustment. The Cayley representation of spatial lines is developed, which is a nonlinear minimal parametrization circumventing the tiresome Plücker constraint. The relationships between different line representations are given. Based on these relationships, we derive a novel line projection function which is consistent with the previous results. After building the line observation model, we employ a closed-form solution for the first image triplet, then develop an incremental initialization approach to initialize the motion and structure parameters. Finally, the sparse bundle adjustment (SBA) is applied to refine the parameters, which updates the spatial lines by using the Cayley representation with an unconstrained optimization engine. The experiments show that the proposed algorithm outperforms the previous works both in efficiency and accuracy.