Replication and comparison of computational experiments in applied evolutionary computing: Common pitfalls and guidelines to avoid them

• Pitfalls in the comparison of computational experiments within the field of applied evolutionary computing are quite common.
• Common pitfalls found in replication and comparison of computational experiments for economic load dispatch problem are presented.
• Guidelines on setting and conducting computational experiments for evolutionary algorithms are provided.

Replicating and comparing computational experiments in applied evolutionary computing may sound like a trivial task. Unfortunately, it is not so. Namely, many papers do not document experimental settings in sufficient detail, and hence replication of experiments is almost impossible. Additionally, some work fails to satisfy the thumb rules for Experimentation throughout all disciplines, such that all experiments should be conducted and compared under the same or stricter conditions. Also, because of the stochastic properties inherent in evolutionary algorithms (EAs), experimental results should always be rich enough with respect to Statistics. Moreover, the comparisons conducted should be based on suitable performance measures and show the statistical significance of one approach over others. Otherwise, the derived conclusions may fail to have scientific merits. The primary objective of this paper is to offer some preliminary guidelines and reminders for assisting researchers to conduct any replications and comparisons of computational experiments when solving practical problems, by the use of EAs in the future. The common pitfalls are explained, that solve economic load dispatch problems using EAs from concrete examples found in some papers.

Figure optionsDownload as PowerPoint slide