Learning with hidden variables

- Work in machine learning has made learning in deep neuronal architectures possible.
- Single neuron non-linearities make a strong impact on the success of learning.
- Biological implementation of these learning rules are being suggested.
- Dynamic nets with hidden nodes capture long-time correlations, relevant in biology.

Learning and inferring features that generate sensory input is a task continuously performed by cortex. In recent years, novel algorithms and learning rules have been proposed that allow neural network models to learn such features from natural images, written text, audio signals, etc. These networks usually involve deep architectures with many layers of hidden neurons. Here we review recent advancements in this area emphasizing, amongst other things, the processing of dynamical inputs by networks with hidden nodes and the role of single neuron models. These points and the questions they arise can provide conceptual advancements in understanding of learning in the cortex and the relationship between machine learning approaches to learning with hidden nodes and those in cortical circuits.