A Survey of Hyper-parameter Optimization Methods in Convolutional Neural Networks
Journal Title: Gazi Üniversitesi Fen Bilimleri Dergisi Part C: Tasarım ve Teknoloji - Year 2019, Vol 7, Issue 2
Abstract
Convolutional neural networks (CNN) are special types of multi-layer artificial neural networks in which convolution method is used instead of matrix multiplication in at least one of its layers. Although satisfactory results have been achieved by CNN especially in computer vision studies, they still have some difficulties. As the proposed network architectures become deeper with the aim of much better accuracy and the resolution of the input images increases, this results in a need for more computational power. Reducing the computational cost while at the same time still having high accuracy rates depend on the use of powerful equipments and the selection of hyper-parameter values in CNN. In this study, we examined methods like Genetic Algorithms, Particle Swarm Optimization, Differential Evolution and Bayes Optimization that has been used extensively to optimize CNN hyper-parameters, and also listed the hyper-parameters selected to be optimized in those studies, ranges of those parameter values and the results obtained by each of those studies. These studies reveal that the number of layers, number and size of the kernels at each layer, learning rate and the batch size parameters are among the hyper-parameters that affect the performance of the CNNs the most. When the studies that use the same datasets are compared in terms of accuracy, Genetic Algorithms and Particle Swarm Optimization which are both population-based methods achieve the best results for the majority of the datasets. It is also shown that the performance of the models found in these studies are competitive or sometimes better than those of the “state of the art” models. In addition, the CNNs produced in these studies are prevented from being overgrown by imposing limits on the hiper-parameter values. Thus simpler and easier to train models have been obtained. These computationally advantageous simpler models were able to achieve competitive results compared to complicated models.
Authors and Affiliations
Ayla GÜLCÜ, Zeki KUŞ
Keywords
Related Articles
- EP ID EP605916
- DOI 10.29109/gujsc.514483
- Views 116
- Downloads 0
The Investigation of viscosity values of Aluminum Powder Reinforced Polypropylene.
In this study, aluminum (Al) powder reinforced polypropylene based composites were produced and viscosity changes were investigated for the determination of flow properties of composites. Experiments have been carried ou...
Comparison of Drum-type and Disc-type Magnetorheological Brakes by Computational Methods
This paper presents a comparison between drum-type and disc-type magnetorheological brakes, based on torque density and efficiency. Magnetic simulations of the parametric brake designs are carried out in FEMM finite elem...
A Detailed Study for the Determination of Phase Inductances of a Shaded-Pole Induction Motor with Variable Air Gap
Shaded-pole induction motors (SPIMs) are extensively used in industrial applications, home appliances and ventilation systems due to their simple structure, low cost and low maintenance requirement. Formation of elliptic...
COMPARISON OF FLAT AND INCARCERATING SURFACE SOLAR SUPPORTED DRYING SYSTEMS FOR APRICOT DRYING
In this study, drying systems with flat and incarcerating surface absorbent plates were designed and manufactured for drying apricots, then their performances were compared. Experiments were carried out with slices 4-5 m...
AN INVESTİGATION ON THE USAGE OF THE PATENT SEARCH ENGİNES
Patent documents are very valuable scientific resources. They store recent and detailed technical knowledge in all technological fields. While the importance of the patent rights is understood better day by day, the numb...