Improved anticipatory classifier system with internal memory for POMDPs with aliased states

ACSM (Hayashida et al., 2014) consists of a method of discerning the aliased states in a POMDP (Partially Observable Markov Decision Process) which is one of Markov decision process such that an agent observes local information about the environment, and choosing the appropriate action based on the internal memory and the sensory information which an agent obtains from the environment. Though ACSM achieves the highest performance in the existing methods based on classifier systems, it requires a huge number of memories for the internal memories, and spends long time for some large scaled problems. This paper improves a classifier system, ACSM (Anticipatory Classifier System with Memory) focused on the process of learning of ACSM, and the aim of this paper is to make the system more efficient. The improved method is named ACSMr in this paper, and some numerical experiments using 5 kinds of maze problems which are well used as benchmark problems for POMDPs are executed. ACSMr achieves greater experimental result than the existing classifier systems for the maze problems.