Local receptive field constrained deep networks

• Extension of RBMs with Gaussian local receptive field constraints on hidden nodes.
• Method for automatically learning advantageous, non-uniform feature detector placement from data.
• Novel biologically inspired deep network models (LRF-DBN and LRF-DNN) with features of gradually increasing size.
• Superior face completion and dimensionality reduction results achieved.
• Feature hierarchy of the proposed model demonstrates part-based compositionality.

Automatic extraction of distinctive features from a visual information stream is challenging due to the large amount of information contained in most image data. In recent years deep neural networks (DNNs) have gained outstanding popularity for solving visual information processing tasks. This study reports novel contributions, including a new DNN architecture and training method, which increase the fidelity of DNN-based representations to encodings extracted by visual processing neurons. Our local receptive field constrained DNNs (LRF-DNNs) are pre-trained with a modified restricted Boltzmann machine, the LRF-RBM, which utilizes biologically inspired Gaussian receptive field constraints to encourage the emergence of local features. Moreover, we propose a method for concurrently finding advantageous receptive field centers, while training the LRF-RBM. By utilizing LRF-RBMs with gradually increasing receptive field sizes on each layer, our LRF-DNN learns features of increasing complexity and demonstrates hierarchical part-based compositionality. We show superior face completion and reconstruction results on the challenging LFW face dataset.