A framework for dynamic restructuring of semantic video analysis systems based on learning attention control

Current semantic video analysis systems are usually hierarchical and consist of some levels to overcome semantic gaps between low-level features and high-level concepts. In these systems, some features, descriptors, objects or concepts are extracted in each level and therefore, total computational complexity of such systems is huge. In this paper, we present a new general framework to impose attention control on a video analysis system using Q-learning. Thus, our proposed framework restructures a given system dynamically to direct attention to the blocks extracting the most informative features/concepts and reduces computational complexity of the system. In other words, the proposed framework directs flow of processing actively using a learning attention control method. The proposed framework is evaluated for event detection in broadcast soccer videos using limited numbers of training samples. Experiments show that the proposed framework is able to learn how to direct attention to informative features/concepts and restructure the initial structure of the system dynamically to reach the final goal with less computational complexity.