Studying teacher selection of resources in an ultra-large scale interactive system: Does metadata guide the way?

Ultra-large-scale interactive systems on the Internet have begun to change how teachers prepare for instruction, particularly in regards to resource selection. Consequently, it is important to look at how teachers are currently selecting resources beyond content or keyword search. We conducted a two-part observational study of an existing popular system called TeachersPayTeachers hypothesizing that ‘evaluative metadata’ (i.e. comments, ratings, and popularity measures) would drive selection of resources. The first part examined patterns in tens of thousands of sales overall, and the second part focused on patterns of sales in one focal topic that could be expert coded. We find that there are significant gaps in available metadata, that some aspects of metadata are closely associated with sales, and that metadata are weak correlates of expert-determined quality. We conclude by making suggestions for additional research and suggesting how ultra-large scale-interactive systems such as TeachersPayTeachers could be used to improve teacher education.

► We studied user created metadata in an ultra-large-scale interactive system.
► We hypothesized that evaluative metadata would drive teacher selection of resources.
► Some aspects of metadata are closely associated with selection of resources.
► Metadata are weak correlates of expert-determined quality.