A Bayesian model for canonical circuits in the neocortex for parallelized and incremental learning of symbol representations

We present a Bayesian model for parallelized canonical circuits in the neocortex, which can partition a cognitive context into orthogonal symbol representations. The model is capable of learning from infinite sensory streams, updating itself with every new instance and without having to keep instances older than the last seen instance per symbol. The inherently incremental and parallel qualities of the model, allow it to scale to any number of symbols as they appear in the sensory stream, and to transparently follow non-stationary distributions for existing symbols. These qualities are made possible in part by a novel Bayesian inference method, which can run Metropolis-Hastings incrementally on a data stream, and significantly outperforms particle filters in a Bayesian neural network application.