Optimal hyperparameter tuning of convolutional neural networks based on the parameter-setting-free harmony search algorithm

Hyperparameters determine layer architecture in the feature extraction step of a convolutional neural network (CNN), and this affects classification accuracy and learning time. In this paper, we propose a method to improve CNN performance by hyperparameter tuning in the feature extraction step of CNN. In the proposed method, the hyperparameter is adjusted using a parameter-setting-free harmony search (PSF-HS) algorithm, which is a metaheuristic optimization method. In the PSF-HS algorithm, the hyperparameter to be adjusted is set as the harmony, and harmony memory is generated after generating the harmony. Harmony memory is updated based on the loss of a CNN. A simulation using CNN architecture with reference to LeNet-5 and a MNIST dataset, and a simulation using the CNN architecture with reference to CifarNet and a Cifar-10 dataset are performed. By two simulations, it is possible to improve the performance by tuning the hyperparameters in CNN architectures proposed in the past.