A color-gradient patch sparsity based image inpainting algorithm with structure coherence and neighborhood consistency

•A color-gradient patch sparsity based image inpainting algorithm is proposed.•Color-gradient structure sparsity is defined to determine filling order.•Weighted color-gradient distance is applied to search patches in local region.•Constraints are constructed to obtain sparse representation to fill target patch.

To better maintain structure coherence and neighborhood consistency, an exemplar-based inpainting algorithm is presented by introducing color-gradient patch sparsity (CGPS). Two concepts of CGPS including color-gradient structure sparsity (CGSS) and patch sparse representation are proposed to obtain the filling order, the search region size and the sparse representation of target patch, which are key steps in an exemplar-based inpainting algorithm. Firstly, the CGSS is designed based on weighted color-gradient distance (WCGD) to determine the filling order of all patches located at fill-front. Secondly, the WCGD is applied to search candidate patches and the CGSS is used to limit the search region size. Thirdly, the patch to be filled is sparsely represented under the local patch consistency constraints in color and gradient spaces. Differing from the exemplar-based inpainting approaches in which only color information is used, the proposed algorithm considers both color and gradient information, which ensures a better maintenance of structure coherence, texture clarity and neighborhood consistency. Moreover, the inpainting efficiency can be significantly improved by limiting the search region size via the CGSS. Experimental results on natural images are presented to demonstrate the advantages of the proposed approach for various tasks such as scratch removal, text removal, block removal and object removal.