On the equations relating a three-dimensional object and its two-dimensional images

Our goal is to interpret computer vision in terms of the vanishing or non-vanishing of certain sets of polynomials. To this end, we establish the algebraic geometric foundations of computer vision using transformation groups. Let α be a three-dimensional object in whose structure m points and n lines play an important role. Let β be a possible two-dimensional image in whose structure are m points and n lines which may be an image of those chosen in α. We show that there is a set of polynomials Fi(α,β) which must vanish when β is a true image of α. The Fi are the maximal subdeterminants of a (3m+2n)×(12+m) matrix and their vanishing says that the rank of this matrix is at most 11+m. Let Z={(α,β):Fi(α,β)=0 for all i} and let Γ={(α,β):β is a true image of α}. We show that the Zariski-closure of Γ is an irreducible component of Z and give a precise description of the other components. We construct a set U′, defined by the non-vanishing of certain polynomials, so that U′∩Γ=U′∩Z. We conclude with two applications to object recovery: when (m,n)=(3,3), we show that different objects have the same image sets; for any (m, n), we show that objects cannot be distinguished by any polynomial function on their image sets.