Examining mixing methods in an evaluation of a smoking cessation program

• Surveys, focus groups, and phenomenological interviews were used in an evaluation.
• Methods were analyzed individually and mixed at the point of interpretation pragmatically and dialectically.
• Strongly interpretive methods produced different findings than the weakly interpretive ones.
• Mixed methods appear optimized when method paradigms differ substantially.
• The cost of mixed method may be justified for this study based on knowledge gains from divergent findings.

Three different methods were used in an evaluation of a smoking cessation study: surveys, focus groups, and phenomenological interviews. The results of each method were analyzed separately and then combined using both a pragmatic and dialectic stance to examine the effects of different approaches to mixing methods. Results show that the further apart the methods are philosophically, the more diverse the findings. Comparisons of decision maker opinions and costs of the different methods are provided along with recommendations for evaluators’ uses of different methods.