The CARE platform for the analysis of behavior model inference techniques

• We provide a benchmarking mechanism for FSM inference techniques.
• CARE provides the framework for the intensive evaluation of the techniques.
• Real case studies as well as artificial programs were used for evaluation.
• These programs cover 7 classes of program structures, for representativeness.
• We compare the performances of five different behavior inference techniques.

ContextFinite State Machine (FSM) inference from execution traces has received a lot of attention over the past few years. Various approaches have been explored, each holding different properties for the resulting models, but the lack of standard benchmarks limits the ability of comparing the proposed techniques. Evaluation is usually performed on a few case studies, which is useful for assessing the feasibility of the algorithm on particular cases, but fails to demonstrate effectiveness in a broad context. Consequently, understanding the strengths and weaknesses of inference techniques remains a challenging task.ObjectiveThis paper proposes CARE, a general, approach-independent, platform for the intensive evaluation of FSM inference techniques.MethodGrounded in a program specification scheme that provides a good control on the expected program structures, it allows the production of large benchmarks with well identified properties.ResultsThe CARE platform demonstrates the following features: (1) providing a benchmarking mechanism for FSM inference techniques, (2) allowing analysis of existing techniques w.r.t. a class of programs and/or behaviors, and (3) helping users in choosing the best suited approach for their objective. Moreover, our extensive experiments on different FSM inference techniques highlight that they do not behave in the same manner on every class of program. Characterizing different classes of programs thus helps understanding the strengths and weaknesses of the studied techniques.ConclusionExperiments reported in this paper show examples of use cases that demonstrate the ability of the platform to generate large and diverse sets of programs, which allows to carry out meaningful inference techniques analysis. The analysis strategies the CARE platform offers open new opportunities for program behavior learning, particularly in conjunction with model checking techniques. The CARE platform is available at http://care.lip6.fr.

Figure optionsDownload as PowerPoint slide