Regularization Activation Function for Extreme Learning Machine
Journal Title: International Journal of Advanced Computer Science & Applications - Year 2019, Vol 10, Issue 3
Abstract
Extreme Learning Machine (ELM) algorithm based on single hidden layer feedforward neural networks has shown as the best time series prediction technique. Furthermore, the algorithm has a good generalization performance with extremely fast learning speed. However, ELM facing overfitting problem that can affect the model quality due to the implementation using empirical risk minimization scheme. Therefore, this study aims to improve ELM by introducing an Activation Functions Regularization in ELM called RAF-ELM. The experiment has been conducted in two phases. First, investigating the modified RAF-ELM performance using four type of activation functions which is Sigmoid, Sine, Tribas and Hardlim. In this study, input weight and bias for hidden layers are randomly selected, whereas the best neurons number of hidden layer is determined from 5 to 100. This experiment used UCI benchmark datasets. The number of neurons (99) using Sigmoid activation function shown the best performance. The proposed methods has improved the accuracy performance and learning speed up to 0.016205 MAE and processing time 0.007 seconds respectively compared with conventional ELM and has improved up to 0.0354 MSE for accuracy performance compare with state of the art algorithm. The second experiment is to validate the proposed RAF-ELM using 15 regression benchmark dataset. RAF-ELM has been compared with four neural network techniques namely conventional ELM, Back Propagation, Radial Basis Function and Elman. The results show that RAF-ELM technique obtain the best performance compared to other techniques in term of accuracy for various time series data that come from various domain.
Authors and Affiliations
Noraini Ismail, Zulaiha Ali Othman, Noor Azah Samsudin
Keywords
Related Articles
- EP ID EP498526
- DOI 10.14569/IJACSA.2019.0100331
- Views 82
- Downloads 0
Decision Tree Classification of Remotely Sensed Satellite Data using Spectral Separability Matrix
In this paper an attempt has been made to develop a decision tree classification algorithm for remotely sensed satellite data using the separability matrix of the spectral distributions of probable classes in respe...
ComEx Miner: Expert Mining in Virtual Communities
The utilization of Web 2.0 as a platform to comprehend the arduous task of expert identification is an upcoming trend. An open problem is to assess the level of expertise objectively in the web 2.0 communities formed. We...
The Impact on Effectiveness and User Satisfaction of Menu Positioning on Web Pages
The authors of this paper are conducting research into the usability of menu positioning on web pages. Other researchers have also done work in this area, but the results are not conclusive and therefore more work still...
Linking Context to Data Warehouse Design
Data warehouses are now widely used for analysis and decision support purposes. The availability of software solutions, which are more and more user-friendly and easy to manipulate has made it possible to extend their us...
Accident Detection and Smart Rescue System using Android Smartphone with Real-Time Location Tracking
A large number of deaths are caused by Traffic accidents worldwide. The global crisis of road safety can be seen by observing the significant number of deaths and injuries that are caused by road traffic accidents. In ma...