A distributed approach for large-scale classifier training and image classification

• A visual concept network is constructed to measure the inter-concept similarity.
• Inter-related classifiers are trained jointly rather than independently.
• A distributed approach is developed for parallel image classification.
• An MPI-based framework is developed to train inter-related classifiers.
• We perform our experiments on image set with 1000 classes for algorithm evaluation.

In this paper, a distributed approach is developed for achieving large-scale classifier training and image classification. First, a visual concept network is constructed for determining the inter-related learning tasks automatically, e.g., the inter-related classifiers for the visually similar object classes in the same group should be trained in parallel by using multiple machines to enhance their discrimination power. Second, an MPI-based distributed computing approach is constructed by using a master–slave mode to address two critical issues of huge computational cost and huge storage/memory cost for large-scale classifier training and image classification. In addition, an indexing-based storage method is developed for reducing the sizes of intermediate SVM models and avoiding the repeated computations of SVs (support vectors) in the test stage for image classification. Our experiments have also provided very positive results on 2010 ImageNet database for Large Scale Visual Recognition Challenge.