Orthogonal nonnegative matrix tri-factorization for co-clustering: Multiplicative updates on Stiefel manifolds

Matrix factorization-based methods become popular in dyadic data analysis, where a fundamental problem, for example, is to perform document clustering or co-clustering words and documents given a term-document matrix. Nonnegative matrix tri-factorization (NMTF) emerges as a promising tool for co-clustering, seeking a 3-factor decomposition X≈USV⊤X≈USV⊤ with all factor matrices restricted to be nonnegative, i.e., U⩾0,S⩾0,V⩾0.U⩾0,S⩾0,V⩾0. In this paper we develop multiplicative updates for orthogonal NMTF where X≈USV⊤X≈USV⊤ is pursued with orthogonality constraints, U⊤U=I,U⊤U=I, and V⊤V=IV⊤V=I, exploiting true gradients on Stiefel manifolds. Experiments on various document data sets demonstrate that our method works well for document clustering and is useful in revealing polysemous words via co-clustering words and documents.