Adaptive Learning Rate SGD AlgorithmforSVM
Journal Title: Scholars Journal of Physics, Mathematics and Statistics - Year 2017, Vol 4, Issue 4
Abstract
Stochastic gradient descent (SGD) is a simple and effective algorithm for solving the optimization problem of support vector machine, where each iteration operates on a single training example. The run-time of SGD does not depend directly on the size of the training set, the resulting algorithm is especially suited for learning from large datasets. However, the problem of stochastic gradient descent algorithm is that it is difficult to choose the proper learning rate. A learning rate is too small, which leads to slow convergence, while a learning rate that is too large can hinder convergence and cause fluctuate. In order to improve the efficiency and classification ability of SVM based on stochastic gradient descent algorithm, three algorithms of adaptive learning rate SGD are used to solve support vector machine, which are Adagrad, Adadelta and Adam. The experimental results show that the algorithm based on Adagrad, Adadelta and Adam for solving the linear support vector machine has faster convergence speed and higher testing precision.
Authors and Affiliations
Shuxia Lu, Zhao Jin
Keywords
Related Articles
- EP ID EP385890
- DOI -
- Views 82
- Downloads 0
Determination of Radioactivity Concentration and Annual Committed Effective Dose in Drinking Water Collected from Local Borehole in Gombe, Nigeria
The gross alpha and beta data were generated from EURISYS MEASURE IN20 low Background multiple (eight) channels alpha and beta detector stationed at Center for Energy Research and Training (CERT), Ahmadu Bello University...
On The Ternary Quadratic Diophantine Equation X2+Y2=37Z2
The Ternary Quadratic Diophantine Equation given by is analyzed for its patterns of non-zero distinct integer solutions. A few interesting relations between the solutions and special polygonal and pyramidal numbers ar...
Analysis of Hydrocarbon Loss Based on Neural Network Geochemical Recovery Correction
Data obtained by rock pyrolysis analysis is only part of the hydrocarbon content in the stratum, which will affect the credibility of information and evaluation of oil-gas-water layers. Therefore, it is necessary to rest...
Skillful application of difference equation in a mathematical puzzle
Unimportant and uncertain decisions can be made by flipping a coin in real-life. And apparently, toss a coin is a question of random probability. In the present paper, the author has made use of difference equation in so...
A New Dimension Reduction Approach Based on Distance for Mixture Discriminant Analysis of the High-Dimensional Data
In this study, we proposed a novel dimension reduction approach for mixture discriminant analysis on based mixture of multivariate normal distributions of high-dimensional data. We considered case of a classification pro...