From movements to actions: Two mechanisms for learning action sequences

When other individuals move, we interpret their movements as discrete, hierarchically-organized, goal-directed actions. However, the mechanisms that integrate visible movement features into actions are poorly understood. Here, we consider two sequence learning mechanisms – transitional probability-based (TP) and position-based encoding computations – that have been studied extensively in the domain of language learning, and investigate their potential for integrating movements into actions. If these learning mechanisms integrate movements into actions, then they should create memory units that contain (i) movement information, (ii) information about the order in which movements occurred, and (iii) information allowing actions to be recognized from different viewpoints. We show that both mechanisms retain movement information. However, only the position-based mechanism creates movement representations that are view-invariant and contain order information. The TP-based mechanism creates movement representations that are view-dependent and contain no order information. We therefore suggest that the TP-based mechanism is unlikely to play an important role for integrating movements into actions. In contrast, the position-based mechanism retains some of the types of information needed to represent goal-directed actions, which makes it an attractive target for further research to explore what, if any, role it plays in the perception of goal-directed actions.

► We examine how humans form memories of action sequences.
► We find that movement memory is computed by two sequence learning mechanisms.
► The first encodes unordered co-occurrences of visible movement features.
► The second encodes ordered, viewpoint-invariant sequential positions of movements.
► We conclude that the second mechanism might be used for action representation.