Direct modelling of output context dependence in discriminative hidden Markov model

This paper proposes a discriminative HMM to directly model output context dependence. The discriminative HMM assumes mutual information independence in its output model that a “hidden” state is only dependent on the outputs and independent on other “hidden” states. As a result, it overcomes the output context independent assumption in the traditional generative HMM. In addition, a dynamic back-off modelling algorithm using constraint relaxation principle is proposed to resolve the data sparseness problem in the discriminative HMM due to the direct modelling of the output context dependence in its output model. The evaluations on part-of-speech tagging and phrase chunking show that the discriminative HMM can effectively capture the output context dependence through its output context dependent output model and the dynamic back-off modelling algorithm.