Cross-corpus speech emotion recognition based on transfer non-negative matrix factorization

• In this paper, we investigate the NMF algorithms for cross-corpus speech emotion recognition.
• We propose a transfer NMF scheme for cross-corpus speech emotion recognition, in which the NMF and MMD algorithms are jointly optimized.
• Two novel transfer NMF approaches, called transfer graph regularized NMF (TGNMF) and transfer constrained NMF (TCNMF), are presented, respectively.
• The optimization algorithms for TGNMF and TCNMF approaches are presented in detail.

Automatic emotion recognition from speech has received an increasing amount of interest in recent years, and many speech emotion recognition methods have been presented, in which the training and testing procedures are often conducted on the same corpus. However, in practice, the training and testing speech utterances are collected from different conditions or devices, which will have adverse effects on recognition performance. To address this problem, in this paper, a novel cross-corpus speech emotion recognition method, called transfer non-negative matrix factorization (TNMF) is proposed. Specifically, the NMF approach, which is popular in computer vision and pattern recognition fields, is utilized to obtain low dimensional representations of emotional features. Meanwhile, the discrepancies between source and target data sets are considered, and the maximum mean discrepancy (MMD) algorithm is used for similarity measurement. Then, the TNMF method, which jointly optimizes the NMF and MMD algorithms, is presented. Moreover, to further improve the recognition performance, two variants of TNMF, called transfer graph regularized NMF (TGNMF) and transfer constrained NMF (TCNMF), are proposed, respectively. Several experiments are carried out on three popular emotional databases, and the results demonstrate the effectiveness and robustness of our scheme.