Lip segmentation and tracking under MAP-MRF framework with unknown segment number

This paper proposes a color lip segmentation method with unknown true segment number. Firstly, we build up a multi-layer hierarchical model, in which each layer corresponds to one segment cluster. Subsequently, a Markov random field derived from this model is obtained such that the segmentation problem is formulated as a labeling optimization problem under the maximum a posteriori Markov random field (MAP-MRF) framework. Suppose the pre-assigned number of segment clusters may over-estimate the ground truth, whereby leading to the over-segmentation. We present a rival penalized iterative algorithm capable of performing segment clusters and over-segmentation elimination simultaneously. Based upon this algorithm, we propose a lip segmentation and tracking scheme, featuring the robust performance to the estimate of the number of segment clusters. Experimental results show the efficacy of the proposed method in comparison with the existing counterparts.

► Formulate the segmentation problem into a labeling optimization problem under the MAP-MRF framework. ► Present a rival penalized iterative algorithm to perform the segment clusters without knowing the true cluster number. ► Propose a lip segmentation and tracking scheme, featuring the robust performance to the pre-assigned number of segment clusters.