Compositing images through light source detection

Compositing an image of an object into another image is a frequently occurring task in both image processing and augmented reality. To ensure a seamless composition, it is often necessary to infer the light conditions of the image to adjust the illumination of the inserted object. Here, we present a novel algorithm for multiple light detection that leverages the limitations of the human visual system (HVS) described in the literature and measured by our own psychophysical study. Finally, we show an application of our method to both image compositing and synthetic object insertion.

Graphical AbstractFigure optionsDownload full-size imageDownload high-quality image (76 K)Download as PowerPoint slideResearch Highlights► We have presented a novel light detection algorithm for single images that only requires the silhouette of any object in the image as additional user input. ► We are not limited to detecting just one light source, and no knowledge of the actual 3D geometry is required. To our knowledge no previous work has proposed a solution to such as under constrained problem. ► We have shown good results both with controlled lighting environments (where the light positions were measured and thus numerical data could be compared) and uncontrolled settings (with free images downloaded from the internet and with synthetic objects). ► We have introduced a novel image compositing method based on our light detection method, relighting and shape from shading. Our algorithm could help photographers mimic a given lighting scheme inspired by any other shot for which a reduced set of light directions (namely, the typical three-light setup made up of key, fill and rim lights) is preferable. Several existing applications could benefit from our automatic compositing system, specifically those based on combining pictures from an existing stack to create novel images (examples and results are shown in the paper). ► We have conducted a psychophysical study, confirming that our results are below a threshold where illumination inconsistencies tend to go unnoticed by human vision.