Combined frequency and spatial domain-based patch propagation for image completion

Film and photography archives now have an accelerated rate of degradation. Because the preservation of cultural heritage plays an important role in our society, photograph and film restoration has recently drawn a substantial amount of attention. In this paper, an approach that involves exemplar-based inpainting, aimed at determining patch priority and patch matching, is proposed. Different image regions have different levels of importance for vision perception; hence, a priority score must be assigned. Patch priority is calculated by the energy of its distributed cosine transform coefficients (DCT term) and the edge term. The edge term prioritizes the edge patches and the energy of the DCT coefficients of the patch is used as a discriminator for patches with similar edge terms. Patch inpainting is performed by assessing the similarity between the patches in such a manner that the similarity measure is consistent with human visual judgment. Therefore, a structure-based similarity measure is developed. Further, the interpolated missing pixels at the patch are also considered for applying the structure-based patch matching criteria in finding the candidate patch. Experimental results on damaged digitized photographs and natural images are presented, which demonstrate the effectiveness of the image completion framework for tasks such as scratch/text, object removal and image inpainting.

Figure optionsDownload high-quality image (110 K)Download as PowerPoint slideHighlights
► A revised image completion algorithm based on resolution limits of the human vision system is developed.
► Patch priority is defined to encourage the filling of patches with high energy values.
► Patches at fill front are interpolated for better example based patch matching.
► Weighted patch matching criteria is developed for improved recovery of structural information.
► Image completion for recovery of damaged digitized photographs.