Homography-based partitioning of curved surface for stereo correspondence establishment

Planar homography (collineation) is an image-to-image mapping that could be used to pinpoint stereo correspondences, but its usage has been limited to only planar scenes. This paper describes a mechanism that generalizes the use of planar homography for establishing stereo correspondences over curved scenes. Piecewise-linear approximation is used to describe curved scene, so that stereo correspondences over images of the scene are captured by a collection of local homographies. Unlike the classical approaches, the mechanism does not employ the smoothness heuristic in establishing correspondences, but is instead based upon the interplay of two processes in an iterative manner: (1) partitioning of the scene into local planar patches based upon the most current set of confirmed stereo correspondences; and (2) prediction of new stereo correspondences by the use of the local homographies defined by the partitions, plus confirmation of such predictions by the image data, thereby enlarging the set of confirmed correspondences in every iteration until no more improvement could be obtained. A key step of the mechanism is to decide what local planar homography, and thereby what planar patch, is to be used for correspondence prediction of any given unmatched feature in any given iteration of the mechanism. With knowledge of the epipolar geometry, homography could be defined by any 3 non-collinear feature correspondences. Thus for any given unmatched feature, there are choices of the aforementioned local homography as defined by any set of three non-collinear matched features in the vicinity of the feature. In this paper, we explore what criteria are to be used to decide which of such triplet sets of matched features should be used. We analyze what errors in correspondence prediction could come with a planar homography. In particular, we classify the errors into scene-related geometric error and computation-related algebraic error. We examine their effects by simulated data experiments, and propose two ways of deciding which local homography to adopt for any given unmatched feature point. Real image experiments show that the methods could lead to promising results.