A heterogeneous computing system for coupling 3D endomicroscopy with volume rendering in real-time image visualization

Modern microscopic volumetric imaging processes lack capturing flexibility and are inconvenient to operate. Additionally, the quality of acquired data could not be assessed immediately during imaging due to the lack of a coherent real-time visualization system. Thus, to eliminate the requisition of close user supervision while providing real-time 3D visualization alongside imaging, we propose and describe an innovative approach to integrate imaging and visualization into a single pipeline called an online incrementally accumulated rendering system. This system is composed of an electronic controller for progressive acquisition, a memory allocator for memory isolation, an efficient memory organization scheme, a compositing scheme to render accumulated datasets, and accumulative frame buffers for displaying non-conflicting outputs. We implement this design using a laser scanning confocal endomicroscope, interfaced with an FPGA prototyping board through a custom hardware circuit. Empirical results from practical implementations deployed in a cancer research center are presented in this paper.