Validation of Systems Engineering Methods and Techniques in Industry

Systems Engineering methods and techniques are mostly taught under the label “Best practices”. Unfortunately, for most methods and techniques we have little support that they are good or even best. In this paper we describe an industry-as-laboratory approach that is applied to evaluate methods and techniques in industrial practice. Our students with at least two years working experience in their company choose one method or technique to study in the daily practice in their company. They apply that technique in one or more projects, observe or measure the results, and evaluate the method or technique. In that way we gradually build up a collection of case studies that can be used to support (or invalidate) claims about the applicability systems engineering methods and techniques in different circumstances and domains. In this paper we report our current insights after 40 master projects in 3 years. The choice of methods and techniques so far has resulted in two methods that have been studied at least six times: concept selection by Pugh matrix and A3 reports. We see that using Pugh matrices helps to evolve the thinking about problem and solution. A3 reports are a promising technique to improve communication and discussion.