Methodological insights from a rigorous small scale design experiment

This paper discusses the methods used to conduct high quality small-scale design experiments. It aims to provide a demonstrator promoting the uptake of more rigorous methods in design research and based on this it aims to specify a body of further work for linking study types and contexts. A small-scale experiment was conducted using methods specifically developed to mitigate four core problem areas identified from review: context, system understanding, methods and controls. The techniques were then critiqued in detail and used to draw several insights for design researchers including the value of control techniques and triangulation of metrics. Finally, the critique is used to specify further research aimed at linking design experimentation and design practice more effectively for design research.

► We conduct and then critique a high quality small-scale design experiment.
► We identify methods currently underutilised in design research.
► We demonstrate benefits of triangulation, deviant case analysis and placebo control.
► We provide a demonstrator for improved methods quality in small-scale experiments.
► We specify further work for externally validating small-scale design experiments.