Understanding the structure of skill through a detailed analysis of Individuals' performance on the Space Fortress game

• We contrasted individual and group level analyses in the domain of Space Fortress.
• We use aspects of the expert-performance approach to detail the performance of a small set of individual participants.
• Individual-level analyses can reveal important aspects of skill that are not captured through group-level techniques.
• We uncovered important behavioral differences between participants that are not captured through traditional techniques.
• We suggest that individual-level data can help uncover the mechanisms of superior performance.

In this paper we describe a novel approach to the study of individual differences in acquired skilled performance in complex laboratory tasks based on an extension of the methodology of the expert-performance approach (Ericsson & Smith, 1991) to shorter periods of training and practice. In contrast to more traditional approaches that study the average performance of groups of participants, we explored detailed behavioral changes for individual participants across their development on the Space Fortress game. We focused on dramatic individual differences in learning and skill acquisition at the individual level by analyzing the archival game data of several interesting players to uncover the specific structure of their acquired skill. Our analysis revealed that even after maximal values for game-generated subscores were reached, the most skilled participant's behaviors such as his flight path, missile firing, and mine handling continued to be refined and improved (Participant 17 from Boot et al., 2010). We contrasted this participant's behavior with the behavior of several other participants and found striking differences in the structure of their performance, which calls into question the appropriateness of averaging their data. For example, some participants engaged in different control strategies such as “world wrapping” or maintaining a finely-tuned circular flight path around the fortress (in contrast to Participant 17′s angular flight path). In light of these differences, we raise fundamental questions about how skill acquisition for individual participants should be studied and described. Our data suggest that a detailed analysis of individuals' data is an essential step for generating a general theory of skill acquisition that explains improvement at the group and individual levels.