The effect of task on classification accuracy: Using gesture recognition techniques in free-sketch recognition

Generating, grouping, and labeling free-sketch data is a difficult and time-consuming task for both user study participants and researchers. To simplify this process for both parties, we would like to have users draw isolated shapes instead of complete sketches that must be hand-labeled and grouped, and then use this data to train our free-sketch symbol recognizer. However, it is an open question whether shapes drawn in isolation accurately reflect the way users draw shapes in a complete diagram. To answer this question, we present a systematic exploration of the effect of task on recognition accuracy using three different recognizers. Our study examines how task affects accuracy in the context of user-independent, user semi-dependent and user-dependent training data. We find that as the amount of user-specific training data increases, the effect of task on recognition accuracy also increases. We also show that the best overall recognition results are obtained by using user semi-dependent, task-specific training data. These results hold across three different domains: circuit diagrams, entity relationship diagrams and process diagrams. Finally, we introduce a variant of a popular and simple gesture recognition algorithm that recognizes freely drawn shapes as well as a highly accurate but more complex recognizer designed explicitly for free-sketch recognition.

Graphical AbstractFigure optionsDownload high-quality image (190 K)Download as PowerPoint slideResearch Highlights
► Training data collection methods have a significant impact on recognizer performance.
► Introduces a simple new multi-stroke symbol recognizer, multi-dollar.
► Compares the performance of multi-dollar, $N, and an image-based symbol recognizer.