On the identifiability of overcomplete dictionaries via the minimisation principle underlying K-SVD

This article gives theoretical insights into the performance of K-SVD, a dictionary learning algorithm that has gained significant popularity in practical applications. The particular question studied here is when a dictionary Φ∈Rd×KΦ∈Rd×K can be recovered as local minimum of the minimisation criterion underlying K-SVD from a set of N   training signals yn=Φxnyn=Φxn. A theoretical analysis of the problem leads to two types of identifiability results assuming the training signals are generated from a tight frame with coefficients drawn from a random symmetric distribution. First, asymptotic results showing that in expectation the generating dictionary can be recovered exactly as a local minimum of the K-SVD criterion if the coefficient distribution exhibits sufficient decay. Second, based on the asymptotic results it is demonstrated that given a finite number of training samples N  , such that N/log⁡N=O(K3d)N/log⁡N=O(K3d), except with probability O(N−Kd)O(N−Kd) there is a local minimum of the K-SVD criterion within distance O(KN−1/4)O(KN−1/4) to the generating dictionary.