Painterly animation using motion maps

Starting from an input video, we replicate the manual technique of paint-on-glass animation. Motion maps are used to represent the regions where changes occur between frames. Edges are the key to identifying frame-to-frame changes, and a strong motion map is constructed from the edges in each frame, displaced by the motion vector. A second, weak motion map records the other pixels where there is significant movement between frames. These maps are used to generate the brush strokes necessary to convert one ‘painted’ frame into the next. Local gradient interpolation, based robustly on the edges, is used to determine the orientation of the brush strokes, and we avoid holes in the image by making additional strokes with smaller brushes. We also employ MSE data in evaluating temporal coherence between frames.