Evaluation of model transformation approaches for model refactoring

• A measurement-based comparison of leading model transformation approaches on a challenging transformation case study.
• Provides a rigorous method for comparative evaluation of transformation approaches, based on quality characteristics and empirical measurement.
• A wide range of quality characteristics are evaluated, from correctness to complexity, modularity, usability and portability.

This paper provides a systematic evaluation framework for comparing model transformation approaches, based upon the ISO/IEC 9126-1 quality characteristics for software systems. We apply this framework to compare five transformation approaches (QVT-R, ATL, Kermeta, UML-RSDS and GrGen.NET) on a complex model refactoring case study: the amalgamation of apparent attribute clones in a class diagram.The case study highlights the problems with the specification and design of the refactoring category of model transformations, and provides a challenging example by which model transformation languages and approaches can be compared. We take into account a wide range of evaluation criteria aspects such as correctness, efficiency, flexibility, interoperability, re-usability and robustness, which have not been comprehensively covered by other comparative surveys of transformation approaches.The results show clear distinctions between the capabilities and suitabilities of different approaches to address the refactoring form of transformation problem.