Sequential learning using temporal context

The temporal context model (TCM) has been extensively applied to recall phenomena from episodic memory. Here we use the same formulation of temporal context to construct a sequential learning model called the predictive temporal context model (pTCM) to extract the generating function of a language from sequentially-presented words. In pTCM, temporal context is used to generate a prediction vector at each step of learning and these prediction vectors are in turn used to construct semantic representations of words on the fly. The semantic representation of a word is defined as the superposition of prediction vectors that occur prior to the presentation of the word in the sequence. Here we create a formal framework for pTCM and prove several useful results. We explore the effect of manipulating the parameters of the model on learning a sequence of words generated by a bi-gram generating function. In this simple case, we demonstrate that feeding back the constructed semantic representation into the temporal context during learning improves the performance of the model when trained with a finite training sequence from a language with equivalence classes among some words. We also describe pTCMA, a variant of the model that is identical to pTCM at steady state. pTCMA has significant computational advantages over pTCM and can improve the quality of its prediction for some training sequences.