Automatic understanding of sketch maps using context-aware classification

•We present the first comprehensive system for offline classifying sketch map objects.•We created a database of labeled sketch maps for training and evaluation purposes.•Context-awareness improves the classification of sketch map objects greatly.

Sketching is a natural and easy way for humans to express visual information in everyday life. Despite a number of approaches to understand online sketch maps, the automatic understanding of offline, hand-drawn sketch maps still poses a problem. This paper presents a new approach for novel sketch map understanding. To our knowledge, this is the first comprehensive work dealing with this task in an offline way. This paper presents a system for automatic understanding of sketch maps and the underlying algorithms for all steps. Major parts are a region-growing segmentation for sketch map objects, a classification for isolated objects, and a context-aware classification. The context-aware classification uses probabilistic relaxation labeling to integrate dependencies between objects into the recognition. We show how these algorithms can deal with the major problems of sketch map understanding, such as vagueness in interpretation. Our experiments demonstrate the importance of context-aware classification for sketch map understanding. In addition, a new database of annotated sketch maps was developed and is made publicly available. This can be used for training and evaluation of sketch map understanding algorithms.