A Survey of Hyper-parameter Optimization Methods in Convolutional Neural Networks
Journal Title: Gazi Üniversitesi Fen Bilimleri Dergisi Part C: Tasarım ve Teknoloji - Year 2019, Vol 7, Issue 2
Abstract
Convolutional neural networks (CNN) are special types of multi-layer artificial neural networks in which convolution method is used instead of matrix multiplication in at least one of its layers. Although satisfactory results have been achieved by CNN especially in computer vision studies, they still have some difficulties. As the proposed network architectures become deeper with the aim of much better accuracy and the resolution of the input images increases, this results in a need for more computational power. Reducing the computational cost while at the same time still having high accuracy rates depend on the use of powerful equipments and the selection of hyper-parameter values in CNN. In this study, we examined methods like Genetic Algorithms, Particle Swarm Optimization, Differential Evolution and Bayes Optimization that has been used extensively to optimize CNN hyper-parameters, and also listed the hyper-parameters selected to be optimized in those studies, ranges of those parameter values and the results obtained by each of those studies. These studies reveal that the number of layers, number and size of the kernels at each layer, learning rate and the batch size parameters are among the hyper-parameters that affect the performance of the CNNs the most. When the studies that use the same datasets are compared in terms of accuracy, Genetic Algorithms and Particle Swarm Optimization which are both population-based methods achieve the best results for the majority of the datasets. It is also shown that the performance of the models found in these studies are competitive or sometimes better than those of the “state of the art” models. In addition, the CNNs produced in these studies are prevented from being overgrown by imposing limits on the hiper-parameter values. Thus simpler and easier to train models have been obtained. These computationally advantageous simpler models were able to achieve competitive results compared to complicated models.
Authors and Affiliations
Ayla GÜLCÜ, Zeki KUŞ
Keywords
Related Articles
- EP ID EP605916
- DOI 10.29109/gujsc.514483
- Views 123
- Downloads 0
Determination of Optimum Cutting Parameters Affecting the Surface Form Properties in the Ductile Stainless Steels with Gray Relational Analysis Method
In this study, the effects of cutting parameters with different coatings (TiAlN and TiN) on drilling performance were investigated. Drilling operations were carried out on duplex stainless steels by drilling holes 20 mm...
Detection of Islamophobic Tweets on Twitter Using Sentiment Analysis
Social networks, becoming more accessible as the Internet usage increases, have turned into platforms where people share their feelings and opinions on various subjects. The 90 per cent of the data we have today has been...
Z in ’Sequential Model’ Spin Analysis for New Adjustment
In the e + e multiplier, the spin analysis for the Z boz new tuning boson was studied. In the Sequential Model, a normalized cross-section of different spin states was investigated. This study shows that the new setting...
The Lattice Boltzmann Method for Fluid Flows and Its Applications
A numerical method in Computational Fluid Dynamics (CFD), which is called “Lattice Boltzmann Method (LBM)”, is studied in this paper. The main difference between LBM and conventional CFD methods is how they investigate p...
Effects of Boron Phosphate Additive on Ceramics with Anorthite Phase
In this study, the anorthite phase was chosen as a ceramic type that can withstand thermal shock. First, boron phosphate was synthesised, boron phosphate was calcined at 1000°C and characterized by XRD and chemical analy...