Levels of automation and user participation in usability testing

This paper identifies a number of factors involved in current practices of usability testing and presents profiles for three prototype methods: think-aloud, subjective ratings, and history files. We then identify ideal levels to generate the profile for new methods. These methods involve either a human observer or a self-administration of the test by the user. We propose methods of automating the evaluation form by dynamically adding items and modifying the form and the tasks in the process of the usability test. For self-administration of testing, we propose similar ideas of dynamically automating the forms and the tasks. Furthermore, we propose methods of eliciting the user's goals and focus of attention. Finally, we propose that user testing methods and interfaces should be subjected to usability testing.