Learning cast shadow appearance for human posture recognition

This paper presents a system for human posture recognition using a camera and two infrared light sources. It uses as input the combination of the body silhouette and its (invisible to the eye) cast shadows. Conventional video-surveillance methods based on a single camera can fail to infer the correct posture since different postures can look similar under perspective projection. Fortunately, cast body shadows, generated by infrared lights, offer additional posture information that cannot be directly captured by a single camera. Each shadow can be projected on different surfaces (e.g. floor, walls, and furniture) generating complex body projections that represent various shapes within the same posture class. These shadow images are very challenging and difficult to describe with traditional handcrafted features that need to be somewhat invariant to these within-class changes. However, a deep convolution neural network (CNN) is able to learn a better data representation from a large-scale dataset. In the absence of a big real dataset, we propose to use synthetic data for training the CNN classifier. Learning from synthetic data is a challenging task due to the gap between synthetic and real feature distributions. Thus, we propose a normalization technique to bridge the gap and help the classifier to better generalize with real data. We evaluated the proposed system on a new real dataset captured in our laboratory and a simulated dataset generated with computer graphics tools. Experimental results validated the efficiency of the CNN model against other conventional methods. Furthermore, the combination of cast shadows and body silhouette had better performance than using only the body silhouette as expected.