Coupled person orientation estimation and appearance modeling using spherical harmonics

• Estimation of a person's orientation, 3D shape and texture from overlapping cameras
• Spherical harmonics are used as low-dimensional 3D shape- and texture-representation.
• Spherical harmonics properties allow to cope with orientation estimation elegantly.
• Outperformance of orientation estimation methods that use a fixed 3D shape model

We present a novel approach for the estimation of a person's overall body orientation, 3D shape and texture, from overlapping cameras. A distinguishing aspect of our approach is the use of spherical harmonics for 3D shape- and texture-representation; it offers a compact, low-dimensional representation, which elegantly copes with rotation estimation. The estimation process alternates between the estimation of texture, orientation and shape. Texture is estimated by sampling image intensities with the predicted 3D shape (i.e. torso and head) and the predicted orientation, from the last time step. Orientation (i.e. rotation around torso major axis) is estimated by minimizing the difference between a learned texture model in a canonical orientation and the current texture estimate. The newly estimated orientation allows to update the 3D shape estimate, taking into account the new 3D shape measurement obtained by volume carving.We investigate various components of our approach in experiments on synthetic and real-world data. We show that our proposed method has lower orientation estimation error than other methods that use fixed 3D shape models, for data involving persons.

Figure optionsDownload high-quality image (246 K)Download as PowerPoint slide