Finding Semantic Equivalence of Text Using Random Index Vectors

The challenges of machine semantic understanding have not yet been satisfactorily solved by automated methods. In our approach, the semantics and syntax of words, phrases and documents are represented by deep semantic vectors that capture both the structure and semantic meaning of the language. Our experiment reproduces the experiment done by Patwardhan and Pedersen 2006, but uses random index vectors for the words, glosses and tweets. Our model first determines random index vectors from glosses and definitions for words from WordNet. From these foundational semantic vectors, random index vectors that represent phrases, sentences or tweets are determined. Our set of algorithms relies on high-dimensional distributed representations, and their effectiveness and versatility derive from the unintuitive properties of such representations: from the mathematical properties of high-dimensional spaces. High-dimensional vector representations have been used successfully in modeling human cognition, such as memory and learning. Our semantic vectors are high-dimensional and capture the meaning of a language expression, such as a word, phrase, query, news article, story or a message. A key benefit of our method is that the dimensionality of the vectors remains constant as we add data; this also allows good generalization to rarely seen words, which “borrow strength” from their more frequent neighbors.