Where should cameras look at soccer games: Improving smoothness using the overlapped hidden Markov model

Automatic camera planning for sports has been a long term goal in computer vision and machine learning. In this paper, we study camera planning for soccer games using pan, tilt and zoom (PTZ) cameras. Two important problems have been addressed. First, we propose the Overlapped Hidden Markov Model (OHMM) method which effectively optimizes the camera trajectory in overlapped local windows. The OHMM method significantly improves the smoothness of the camera planning by optimizing the camera trajectory in the temporal space, resulting in much more natural camera movements present in real broadcasts. We also propose CalibMe which is a highly automatic camera calibration method for soccer games. CalibMe enables users to collect large amounts of training data for learning algorithms. The precision of CalibMe is evaluated on a motion blur affected sequence and outperforms several strong existing methods. The performance of the OHMM method is extensively evaluated on both synthetic and real data. It outperforms the state-of-the-art algorithms in terms of smoothness without sacrificing accuracy.