EISEval, a generic reconfigurable environment for evaluating agent-based interactive systems

The evaluation of interactive systems has been an active subject of research for many years. Many methods have been proposed, but most of them do not take the architectural specificities of an agent-based interactive system into account, nor do they focus on the link between architecture and evaluation. In this paper, we present an agent-based architecture model for interactive systems. Then, based on this architecture, we propose a generic, reconfigurable evaluation environment, called EISEval, designed and developed to help evaluators analyze and evaluate certain aspects of interactive systems in general and of agent-based architecture interactive systems in particular: User Interface (UI), non-functional properties (e.g., response time, complexity) and user characteristics (e.g., abilities, preferences, progress). System designers can draw useful conclusions from the evaluation results to improve the system. This environment was applied to evaluate an agent-based interactive system used to supervise an urban transport network in a study organized in laboratory.

Graphical abstractFigure optionsDownload full-size imageDownload as PowerPoint slideHighlights► We consider agent-based architecture model in evaluation. ► A new type of evaluation tool is described. ► The feasibility of this approach has been successfully showed in a study in transport domain.