On automated source selection for transfer learning in convolutional neural networks

•This study demonstrates that automatically ranking pre-trained source CNNs for a given target task, is possible.•This study presents an information theoretic framework to rank source CNNs in an efficient, reliable, and zero-shot manner.•This study presents a thorough experimental evaluation of the proposed theory using the Places-MIT database, CalTech-256 database, MNIST database and a real-world MRI database.

Transfer learning, or inductive transfer, refers to the transfer of knowledge from a source task to a target task. In the context of convolutional neural networks (CNNs), transfer learning can be implemented by transplanting the learned feature layers from one CNN (derived from the source task) to initialize another (for the target task). Previous research has shown that the choice of the source CNN impacts the performance of the target task. In the current literature, there is no principled way for selecting a source CNN for a given target task despite the increasing availability of pre-trained source CNNs. In this paper we investigate the possibility of automatically ranking source CNNs prior to utilizing them for a target task. In particular, we present an information theoretic framework to understand the source-target relationship and use this as a basis to derive an approach to automatically rank source CNNs in an efficient, zero-shot manner. The practical utility of the approach is thoroughly evaluated using the Places-MIT dataset, MNIST dataset and a real-world MRI database. Experimental results demonstrate the efficacy of the proposed ranking method for transfer learning.