A CSCW Requirements Engineering CASE Tool: Development and usability evaluation

•Six metrics were used to measure effectiveness, efficiency and satisfaction.•Facial expression analysis was used to assess users’ satisfaction throughout the test.•The evaluation of tool was reported by applying the ISO/IEC 25062:2006.•We found some flaws in the tool, despite its good usability figures.•The lessons learnt are presented to help in other usability assessment studies.

ContextCSRML Tool 2012 is a Requirements Engineering CASE Tool for the Goal-Oriented Collaborative Systems Requirements Modeling Language (CSRML).ObjectiveThis work aims at evaluating the usability of CSRML Tool 2012, thus identifying any possible usability flaw to be solved in the next releases of the application, as well as giving a general overview on how to develop a DSL tool similar to the one evaluated in this work by means of Visual Studio Modelling SDK.MethodIn this evaluation, which was reported by following the ISO/IEC 25062:2006 Common Industry Format for usability tests, 28 fourth-course Computer Science students took part. They were asked to carry out a series of modifications to an incomplete CSRML requirements specification. Usability was assessed by measuring the task’s completion rate, the elapsed time, number of accesses to the help system of the tool and the instructor’s verbal assistance. The participants’ arousal and pleasantness were assessed by analyzing both facial expressions and a USE questionnaire.ResultsIn spite of obtaining high usability levels, the test outcome revealed some usability flaws that should be addressed.ConclusionThe important lessons learnt from this evaluation are also applicable to the success of other usability tests as well as to the development of new CASE tools.

Graphical abstractFigure optionsDownload full-size imageDownload as PowerPoint slide