Regularization Activation Function for Extreme Learning Machine
Journal Title: International Journal of Advanced Computer Science & Applications - Year 2019, Vol 10, Issue 3
Abstract
Extreme Learning Machine (ELM) algorithm based on single hidden layer feedforward neural networks has shown as the best time series prediction technique. Furthermore, the algorithm has a good generalization performance with extremely fast learning speed. However, ELM facing overfitting problem that can affect the model quality due to the implementation using empirical risk minimization scheme. Therefore, this study aims to improve ELM by introducing an Activation Functions Regularization in ELM called RAF-ELM. The experiment has been conducted in two phases. First, investigating the modified RAF-ELM performance using four type of activation functions which is Sigmoid, Sine, Tribas and Hardlim. In this study, input weight and bias for hidden layers are randomly selected, whereas the best neurons number of hidden layer is determined from 5 to 100. This experiment used UCI benchmark datasets. The number of neurons (99) using Sigmoid activation function shown the best performance. The proposed methods has improved the accuracy performance and learning speed up to 0.016205 MAE and processing time 0.007 seconds respectively compared with conventional ELM and has improved up to 0.0354 MSE for accuracy performance compare with state of the art algorithm. The second experiment is to validate the proposed RAF-ELM using 15 regression benchmark dataset. RAF-ELM has been compared with four neural network techniques namely conventional ELM, Back Propagation, Radial Basis Function and Elman. The results show that RAF-ELM technique obtain the best performance compared to other techniques in term of accuracy for various time series data that come from various domain.
Authors and Affiliations
Noraini Ismail, Zulaiha Ali Othman, Noor Azah Samsudin
Keywords
Related Articles
- EP ID EP498526
- DOI 10.14569/IJACSA.2019.0100331
- Views 63
- Downloads 0
An Improved Grunwald-Letnikov Fractional Differential Mask for Image Texture Enhancement
Texture plays an important role in identification of objects or regions of interest in an image. In order to enhance this textural information and overcome the limitations of the classical derivative operators a tw...
Comparative Study of Bayesian and Energy Detection Including MRC Under Fading Environment in Collaborative Cognitive Radio Network
The most important component of Cognitive Radio Network (CRN) is to sense the underutilised spectrum efficiently in fading environment for incorporating the increasing demand of wireless applications. The result of spect...
Crypto-Steganographic LSB-based System for AES-Encrypted Data
The purpose of this work is to increase the level of concealment of information from unauthorized access by pre-encrypting and hiding it in multimedia files such as images. A crypto-steganographic information protection...
An Investigation and Comparison of Invasive Weed, Flower Pollination and Krill Evolutionary Algorithms
Being inspired by natural phenomena and available biological processes in the nature is one of the difficult methods of problem solving in computer sciences. Evolutionary methods are a set of algorithms that are inspired...
Create a Virtual Mannequin Through the 2-D Image-based Anthropometric Measurement and Radius Distance Free Form Deformation
3-D human body models are used in a wild spectrum of applications, such as film and entertainment industry, that require images of human replicas, but the computer generated models of human body generally do not ad...