Hierarchical ELM ensembles for visual descriptor fusion

Extreme Learning Machines (ELM) have been successfully applied to variety of classification problems by utilizing a single descriptor type. However, a single descriptor may be insufficient for the visual classification task, due to the high level of intra-class variability coupled with low inter-class distance. Although several studies have investigated methods for combining multiple descriptors by ELM, they predominantly apply a simple concatenation of descriptors before classifying them. This type of descriptor fusion may impose problems of descriptor compatibility, high dimensionality and restricted accuracy. In this paper, we propose a hierarchical descriptors fusion strategy at the decision level (“late-fusion”), which relies on ELM ensembles (ELM-E). The proposed method, denoted as H-ELM-E, effectively combines multiple complementary descriptors by a two-level ELM-E based architecture, which ensures that a more informative descriptors will gain more impact on the final decision. In the first level, a separate ELM-E classifier is trained for every image descriptor. In the second level, the output scores from the previous level are aggregated into the mid-level representation which is conducted to an additional ELM-E classifier. The exhaustive experimental evaluation confirmed that the proposed hierarchical ELM-E based strategy is superior to the single-descriptor methods as well as “early fusion” of multiple descriptors, for the visual classification task. Additionally, it was shown that significant accuracy improvement is achieved by integrating ensembles of ELM as a basic classifier, instead of using a single ELM.