Solving partially observable problems with inaccurate PSR models

Modeling dynamical systems is a commonly used technique to solve partially observable problems in the artificial intelligence field. Predictive state representations (PSRs) have been proposed as an alternative to partially observable Markov decision processes (POMDPs) to model dynamical systems. Although POMDPs and PSRs provide general frameworks to solve partially observable problems, they rely heavily on a known and accurate model of the environment. However, in real world applications it is extremely difficult to build an accurate model. In this paper, we propose an algorithm to solve partially observable problems using an inaccurate PSR model which is learned from samples. The proposed algorithm can also improve the accuracy of the learned model. Given the inaccurate PSR model, the PSR state is identified firstly. Then the traditional Markov decision processes (MDP) techniques are used to solve the partially observable problem. Furthermore, the learned model, which may get off-track as often happens when the model is learned from samples, can be reset. The effectiveness of our proposed algorithm is demonstrated based on a standard set of POMDP test problems.