On Improving Probability Models in System Identification

A solution of the following problem is presented. Given a known model represented by a discrete probability distribution = (q1, q2, … , qn), and information Δ, a new improved model represented by a probability distribution is found which is associated with such that Δ = D( || ), where D( || ) represents the relative information between the two probability distributions. The improved probability distributions have been chosen to be those which become local minima of the discrete entropy. The fundamental properties of such models and natural effects how to use information are studied.