A Survey of Hyper-parameter Optimization Methods in Convolutional Neural Networks
Journal Title: Gazi Üniversitesi Fen Bilimleri Dergisi Part C: Tasarım ve Teknoloji - Year 2019, Vol 7, Issue 2
Abstract
Convolutional neural networks (CNN) are special types of multi-layer artificial neural networks in which convolution method is used instead of matrix multiplication in at least one of its layers. Although satisfactory results have been achieved by CNN especially in computer vision studies, they still have some difficulties. As the proposed network architectures become deeper with the aim of much better accuracy and the resolution of the input images increases, this results in a need for more computational power. Reducing the computational cost while at the same time still having high accuracy rates depend on the use of powerful equipments and the selection of hyper-parameter values in CNN. In this study, we examined methods like Genetic Algorithms, Particle Swarm Optimization, Differential Evolution and Bayes Optimization that has been used extensively to optimize CNN hyper-parameters, and also listed the hyper-parameters selected to be optimized in those studies, ranges of those parameter values and the results obtained by each of those studies. These studies reveal that the number of layers, number and size of the kernels at each layer, learning rate and the batch size parameters are among the hyper-parameters that affect the performance of the CNNs the most. When the studies that use the same datasets are compared in terms of accuracy, Genetic Algorithms and Particle Swarm Optimization which are both population-based methods achieve the best results for the majority of the datasets. It is also shown that the performance of the models found in these studies are competitive or sometimes better than those of the “state of the art” models. In addition, the CNNs produced in these studies are prevented from being overgrown by imposing limits on the hiper-parameter values. Thus simpler and easier to train models have been obtained. These computationally advantageous simpler models were able to achieve competitive results compared to complicated models.
Authors and Affiliations
Ayla GÜLCÜ, Zeki KUŞ
Keywords
Related Articles
- EP ID EP605916
- DOI 10.29109/gujsc.514483
- Views 122
- Downloads 0
Real Time Measurement of Average Winding Resistances of Transformers
The monitoring of transformers that are expensive equipment of power systems is the one of important issues. Transformer monitoring systems are usually used in order to detect failures, to determine the lifetime, to meas...
Numerical investigation of steam addition into the oxidizer on combustion behaviours of coke oven gas
The effects of steam addition into the oxidizer on combustion behaviours of coke oven gas has been numerically investigated within the present study. Ansys Fluent that is a commercial computational fluid dynamics code ha...
Discrete Cosine Transformation in Human Face Images with Noise - Subband Based Face Recognition
In this work, face recognition performances of eigenface and Fisherface algorithms in different face subspaces obtained from Discrete Cosine Transform (DCT) coefficients and original images were evaluated. DCT coefficien...
Effects of Boron Phosphate Additive on Ceramics with Anorthite Phase
In this study, the anorthite phase was chosen as a ceramic type that can withstand thermal shock. First, boron phosphate was synthesised, boron phosphate was calcined at 1000°C and characterized by XRD and chemical analy...
Risk Factors Identification and Estimation for Glass Fiber Reinforced Concrete Production Sector
Glass fiber reinforced concrete (GRC) usage in construction industry has been becoming very popular due to its aesthetic, mechanical properties, fast and reliable production in the last decade. However, it has more compl...