Towards robust 3D visual tracking for motion compensation in beating heart surgery

In the context of minimally invasive cardiac surgery, active vision-based motion compensation schemes have been proposed for mitigating problems related to physiological motion. However, robust and accurate visual tracking remains a difficult task. The purpose of this paper is to present a robust visual tracking method that estimates the 3D temporal and spatial deformation of the heart surface using stereo endoscopic images. The novelty is the combination of a visual tracking method based on a Thin-Plate Spline (TPS) model for representing the heart surface deformations with a temporal heart motion model based on a time-varying dual Fourier series for overcoming tracking disturbances or failures. The considerable improvements in tracking robustness facing specular reflections and occlusions are demonstrated through experiments using images of in vivo porcine and human beating hearts.

In this paper, we present a robust visual tracking method that estimates the 3D temporal and spatial deformation of the heart surface using stereo endoscopic images. The proposed method is a combination of a visual tracking method based on a Thin-Plate Spline (TPS) model for representing the heart surface deformations with a temporal heart motion model based on a time-varying dual Fourier series for overcoming tracking disturbances or failures.Figure optionsDownload high-quality image (156 K)Download as PowerPoint slideResearch highlights
► This paper focuses on robust and accurate visual tracking for mitigating problems related to physiological motion in minimally invasive cardiac surgery.
► The paper presents a novel robust visual tracking framework that estimates the 3D temporal and spatial deformation of the heart surface using stereo endoscopic images.
► By combining a visual tracking method based on Thin-Plate Splines and a temporal motion model based on a dual Fourier series, tracking disturbances and failures can be bridged.
► The robustness improvements are demonstrated through experiments using in vivo porcine and human beating heart images.