Two-dimensional subspace alignment for convolutional activations adaptation

In real-world computer vision applications, many intrinsic and extrinsic variations can cause a significant domain shift. Although deep convolutional models have provided us with better domain-invariant features, existing mechanisms to adapt convolutional activations are still limited. Notice that convolutional activations are intrinsically represented as tensors, in this paper we develop a two-dimensional subspace alignment (2DSA) approach based on 2D principal component analysis (PCA) to better adapt convolutional activations. Extensive experiments demonstrate the advantages of 2DSA over its counterpart SA in both effectiveness and efficiency. In particular, when trying to explain why 2DSA works well, we find that the best classification performance has low correlation with the global domain discrepancy measure. In an effort to find a better way to compare domains, we introduce within- and between-class domain divergence measures to characterize the class-level differences. The proposed measures somewhat shed light on what a good alignment might be for classification. Furthermore, we also demonstrate a novel domain adaptation application in agriculture and create a dataset for the problem.