The Conundrum of Verification and Validation of Social Science-based Models

In the systems engineering lexicon, definitions for the terms “verification” and “validation” are settled; consistent with the definitions promulgated by the Department of Defense (DoD) [1], ; and quite distinct from one another. Verification confirms that all elements of the system meet technical requirements (the product was built right). Validation confirms that the realized system complies with stakeholder requirements (the right system was built). The distinction becomes blurred, however, when one considers verification and validation (V&V) of social science-based models and simulations. Unlike physics-based models, the theoretical underpinnings of Human, Social, Cultural, and Behavioral (HSCB) or other social science models are not readily verified through observation of real-world events or empirical testing. As a result, the theoretical claims on which the models are built are often contested. As noted by Lustick and Tubin [2], when experts do not agree on what the right thing is, determining that the model is built right cannot be separated from tests of whether the right thing has been built. Because systems engineers may encounter social-science based models either in the context of system design or verification, where they may be used as substitutes for human operators or users, or when they are components of a physical system, as is the case when HSCB models are embedded into enhanced persistent surveillance systems for military or intelligence applications, it is important that they understand the limitations and controversies surrounding V&V of these types of models. In this paper, the literature on V&V of models is reviewed, with an emphasis on social science models and some recently developed constructs for their verification and validation. Future directions for social science-based model development and V&V are briefly outlined.