Advanced interactive medical visualization on the GPU

Interactive visual analysis of a patient’s anatomy by means of computer-generated 3D imagery is crucial for diagnosis, pre-operative planning, and surgical training. The task of visualization is no longer limited to producing images at interactive rates, but also includes the guided extraction of significant features to assist the user in the data exploration process. An effective visualization module has to perform a problem-specific abstraction of the dataset, leading to a more compact and hence more efficient visual representation. Moreover, many medical applications, such as surgical training simulators and pre-operative planning for plastic and reconstructive surgery, require the visualization of datasets that are dynamically modified or even generated by a physics-based simulation engine.In this paper we present a set of approaches that allow interactive exploration of medical datasets in real time. Our method combines direct volume rendering via ray-casting with a novel approach for isosurface extraction and re-use directly on graphics processing units (GPUs) in a single framework. The isosurface extraction technique takes advantage of the recently introduced Microsoft DirectX®10 pipeline for dynamic surface extraction in real time using geometry shaders. This surface is constructed in polygonal form and can be directly used post-extraction for collision detection, rendering, and optimization. The resulting polygonal surface can also be analyzed for geometric properties, such as feature area, volume and size deviation, which is crucial for semi-automatic tumor analysis as used, for example, in colonoscopy. Additionally, we have developed a technique for real-time volume data analysis by providing an interactive user interface for designing material properties for organs in the scanned volume. Combining isosurface with direct volume rendering allows visualization of the surface properties as well as the context of tissues surrounding the region and gives better context for navigation. Our application can be used with CT and MRI scan data, or with a variety of other medical and scientific applications. The techniques we present are general and intuitive to implement and can be used for many other interactive environments and effects, separately or together.