Representing, binding, retrieving and unifying relational knowledge using pools of neural binders

For a long time, connectionist architectures have been criticized for having propositional fixation, lack of compositionality and, in general, for their weakness in representing sophisticated symbolic information and processing it. This work offers a novel approach that allows full integration of symbolic AI with the connectionist paradigm. We show how to encode and process relational knowledge using Artificial Neural Networks (ANNs), such as Boltzmann Machines. The neural architecture uses a working memory (WM), consisting of pools of “binders”, and a long-term synaptic-memory that can store a large relational knowledge-base (KB). A compact variable binding mechanism is proposed which dynamically allocates ensembles of neurons when a query is clamped; retrieving KB items till a solution emerges in the WM. We illustrate the proposal through non-trivial predicate unification problems: knowledge items are only retrieved into the WM upon need, and unified, graph-like structures emerge at equilibrium as an activation pattern of the neural network. Our architecture is based on the fact that some attractor-based ANNs may be viewed as performing constraint satisfaction, where, at equilibrium, fixed-points maximally satisfy a set of weighted constraints. We show how to encode relational graphs as neural activation in WM and how to use constraints that are encoded in synapses, in order to retrieve and process such complex structures. Both procedural (the unification algorithm) and declarative knowledge (logic formulae) are first expressed as constraints and then used to generate (or learn) weighted synaptic connections. The architecture has no central control and is inherently robust to unit failures. Contrary to previous connectionist suggestions, this approach is expressive, compact, accurate, and goal directed. The mechanism is universal and has a simple underlying computational principle. As such, it may be further adapted for applications that combine the advantages of both connectionist and traditional symbolic AI and may be used in modeling aspects of human reasoning.