On the implicit acquisition of a context-free grammar by a simple recurrent neural network

The performance of a simple recurrent neural network on the implicit acquisition of a context-free grammar is re-examined and found to be significantly higher than previously reported by Elman. This result is obtained although the previous work employed a multilayer extension of the basic form of simple recurrent network and restricted the complexity of training and test corpora. The high performance is traced to a well-organized internal representation of the grammatical elements, as probed by a principal-component analysis of the hidden-layer activities. From the next-symbol-prediction performance on sentences not present in the training corpus, a capacity of generalization is demonstrated.