Entropy of high-order Markov chains beyond the pair correlations

The main goal of this paper is to develop an estimate for the entropy of random stationary ergodic symbolic sequences with elements belonging to a finite alphabet. We present here the detailed analytical study of the entropy for the high-order Markov chain in the bilinear approximation. In the limit of weak correlations, we present analytically the entropy of sequence by means of correlation functions of the second and third orders. The appendix contains a short comprehensive introduction into the subject of study.