Functional Systems Network Outperforms Q-learning in Stochastic Environment

Learning of the goal-directed behavior is one of the central problems in the field of artificial intelligence. Functional system network (FSN) is biologically inspired algorithm proposed in [3] that demonstrated successful learning in deterministic multi-goal environments. Here we extend it be applicable in stochastic environments. Important feature of the FSN algorithm is ability to learn many optional goal-directed action sequences and switch between then during behavior execution. To optimize reuse of alternative behaviors in stochastic environments we extended original FSN with functionality that allows to rank competing options by estimated usefulness. Extended model was studied in the grid world of different sizes with stochastic transitions updated between trials. Results demonstrate that FSN is able to solve this task and outperforms significantly standard Q-learning algorithm.