Structured deep Fisher pruning for efficient facial trait classification

High efficiency is desirable for many interactive biometrics tasks, including facial trait recognition. Although deep convolutional nets are effective for a multitude of classification tasks, their high space and time demands make them impractical for PCs and mobile devices without a powerful GPU. In this paper, we propose a structured filter-level pruning approach based on Fisher LDA [1], which boosts efficiency while maintaining accuracy for facial trait classification. It starts from the last convolutional layer where we find filter activations are less correlated. Through Fisher's LDA, we show that this decorrelation makes it safe to discard directly filters with high within-class variance and low between-class variance. The pruning goes on by tracing deconvolution based dependency over layers. Combined with light classifiers, the reduced CNNs can achieve comparable accuracies on example facial traits from the LFWA(+) and CelebA datasets, but with large reductions in model size (96%-98% for VGG-16, 81% for GoogLeNet) and computation (as high as 80% for VGG-16, 61% for GoogLeNet).