On-line, interactive view synthesis and augmentation

Our on-line, interactive system for enhanced tele-teaching starts from a network of fixed cameras, placed around an instructor. A ‘virtual’ camera can move freely between these ‘real’ cameras. The creation of a virtual view requires on-line foreground/background segmentation and 3D reconstructions of the foreground. An off-line, user-interactive method is used for the initial 3D reconstruction of the background. An on-line update of the background model is done automatically.This on-line analysis also allows the system to place the instructor in a ‘virtual or mixed environment’. The surroundings can be replaced by a more suitable background, like an outdoor scene or large-sized presentation slides. Given a desktop and a set of consumer grade cameras, a lecture can be given from an arbitrary place, e.g. an office. Virtual objects, in particular ‘virtual post-its’ or labels, can be used to augment the scene. They can be on-line selected slide cut-outs or predefined elements. The former are based on simple gestures with a laser pointer.