A differential geometry approach to camera-independent image correspondence

Projective geometry is a standard mathematical tool for image-based 3D reconstruction. Most reconstruction methods establish pointwise image correspondences using projective geometry. We present an alternative approach based on differential geometry using oriented patches rather than points. Our approach assumes that the scene to be reconstructed is observed by any camera, existing or potential, that satisfies very general conditions, namely, the differentiability of the surface and the bijective projection functions. We show how the notions of the differential geometry such as diffeomorphism, pushforward and pullback are related to the reconstruction problem. A unified theory applicable to various 3D reconstruction problems is presented. Considering two views of the surface, we derive reconstruction equations for oriented patches and pose equations to determine the relative pose of the two cameras. Then we discuss the generalized epipolar geometry and derive the generalized epipolar constraint (compatibility equation) along the epipolar curves. Applying the proposed theory to the projective camera and assuming that affine mapping between small corresponding regions has been estimated, we obtain the minimal pose equation for the case when a fully calibrated camera is moved with its internal parameters unchanged. Equations for the projective epipolar constraints and the fundamental matrix are also derived. Finally, two important nonlinear camera types, the axial and the spherical, are examined.