Image retargeting with a 3D saliency model

• We propose to explore 3D information in image retargeting.
• In seam caving, 3D saliency works with L-1 norm of gradient to generate energy map for seam searching.
• In warping, 3D information is explored to build structure constraint and 3D saliency adjusts weight of mesh edges in warping.

Image retargeting is an attractive topic since variety devices are with different resolutions. So far, there are two main strategies for this problem. One is removing or adding appropriate pixels in image directly, such as the seam carving approach, and the other is warping the mesh of image and then mapping the texture. However, existing approaches may encounter difficulties on different types of images since most of them only consider 2D features. In this paper, we propose an adaptive 3D saliency model based on the 3D structure of images to incorporate 3D information into both seam carving and mesh warping for image retargeting. Considering the characteristics of the two strategies, the 3D information are used in different ways. In seam caving, the adaptive 3D saliency is combined with L1 norm of gradient to generate an energy map for searching the least important seam. In warping, depth information is explored to improve the detection of important edges in mesh generation. Experimental results demonstrate the advantages of our method in both the seam carving and the image warping algorithms.