A comprehensive approach, and a case study, for conducting attack detection experiments in Cyber-Physical Systems

Several methods have been proposed by researchers to detect cyber attacks in Cyber-Physical Systems (CPSs). This paper proposes a comprehensive approach for conducting experiments to assess the effectiveness of such methods in the context of a robot (Amigobot) that includes both cyber and physical components. The proposed approach includes a method for performing vulnerability analysis, several methods for attack detection, and guidelines for conducting experimental studies in the context of cyber security. The method for vulnerability analysis makes use of the Failure-Attack-CounTermeasure (FACT) graph. The experimental study to evaluate methods for attack detection comprises of three experiments. These methods have been implemented and evaluated, within and across all three experiments, with respect to their effectiveness, detection speed, and durability for injection, scaling, and stealthy attacks. The proposed guidelines define key phases and artifacts for conducting such experiments and are an adaptation of those used in Software Engineering.