Feature coding for image classification combining global saliency and local difference

• We propose an efficient and robust coding method for image classification.
• Global descriptor space saliency is applied for improving stability and robustness.
• Local difference is used for exploring the latent structure information of the codebook.
• Our global descriptor space saliency is complementary to the previous local saliency based coding.

Saliency1 based coding proposed recently have been proven to perform well in both performance and efficiency for image classification. However, we find that they are sensitive to unusual features, e.g., noisy features, which we call poor robustness. To address this problem, we propose a novel coding scheme by combining global saliency and local difference together, which are applied for improving stability or robustness and exploring the latent structure information of the codebook respectively. Thorough experiments on various datasets show that our coding consistently performs better than local saliency based coding, in terms of both accuracy and computation cost. Furthermore, it is more robust to unusual features than localized soft-assignment coding. In addition, a combination of our global saliency with local saliency based coding can usually improve both.