Exploring causal relationships in an innovation program with Robust Portfolio Modeling

Many countries seek to foster the commercial exploitation of science-based research results through selective policy instruments. Typically, these instruments involve processes of follow-up data collection where the results of ex ante and ex post assessments are systematically recorded. Yet, several factors – such as the presence of multiple objectives, predominance of qualitative data and missing observations – may complicate the use of such data for adjusting the management practices of these instruments. With the aim of addressing these challenges, we adopt Robust Portfolio Modeling 1 (RPM) as an evaluation framework to the analysis of longitudinal data: specifically, we (i) determine subsets of outperforming and underperforming projects through the development of an explicit multicriteria model for ex post evaluation, and (ii) carry out comparative analyses between these subsets, in order to identify which ex ante interventions and contextual characteristics may have contributed to later performance. We also report experiences from the application of RPM-evaluation to a Finnish innovation program and outline extensions of this approach that provide further decision support to the managers of innovation programs.