Early experiments with neural diversity machines

The current paper introduces the concept of neural diversity machines (NDM) which, refers to hybrid artificial neural networks (HANN) with conditions on the minimum number of functions available to the network, amongst several other properties. The paper demonstrates how NDM networks can be optimized for solving different problems. The results demonstrate the feasibility of the approach and bolster some of the biological and computational arguments in favor of neural diversity. A substantial number of optimization experiments were conducted, generating a corresponding number of diverse neural architectures, which revealed several unexpected statistics, including the relative commonality of nodes combining inner-product and Gaussian functions. The paper confirms the advantages of HANN, demonstrates the potential of increasing the focus on neural diversity and hints at possible new neural computational strategies.