Learning multi-prototype word embedding from single-prototype word embedding with integrated knowledge

• A mini context word sense disambiguation with adapted Lesk algorithm is proposed
• New initialization approach is proposed to balance speed and performance.
• Improved approximation algorithm is proposed to help supervised learning
• The framework may utilize any sense inventory with word-sense definition.

Distributional semantic models (DSM) or word embeddings are widely used in prediction of semantic similarity and relatedness. However, context aware similarity and relatedness prediction is still a challenging issue because most DSM models or word embeddings use one vector per word without considering polysemy and homonym. In this paper, we propose a supervised fine tuning framework to transform the existing single-prototype word embeddings into multi-prototype word embeddings based on lexical semantic resources. As a post-processing step, the proposed framework is compatible with any sense inventory and any word embedding. To test the proposed learning framework, both intrinsic and extrinsic evaluations are conducted. Experiments results of 3 tasks with 8 datasets show that the multi-prototype word representations learned by the proposed framework outperform single-prototype word representations.