Game-based versus storyboard-based evaluations of crew support prototypes for long duration missions

The Mission Execution Crew Assistant (MECA) is developing a distributed system of electronic partners (ePartners) to support astronauts performing nominal and off- nominal actions in long duration missions. The ePartners’ support should adequately deal with the dynamics of the context, operations, team and personal conditions, which will change over time substantially. Such support—with the concerning context effects—should be thoroughly tested in all stages of the development process. A major question is how to address the context effects of in-space operations for evaluations of crew support prototypes. Via game-technology, the prototype can be tested with astronauts or their representatives, immersed in the envisioned, simulated context. We investigated if a game-based evaluation better addresses the context effects by producing a more elaborate, in-depth and realistic user experience than a “classical” storyboard-based evaluation. In the game-based evaluation, the participants showed higher arousal levels where expected, a more intense feeling of spatial presence, better situation awareness, and faster performance where needed. Such an evaluation can be used as an alternative or complement of field or micro-world tests when context dynamics cannot be simulated in these last tests cost-efficiently.