Adaptive Learning Rate SGD AlgorithmforSVM
Journal Title: Scholars Journal of Physics, Mathematics and Statistics - Year 2017, Vol 4, Issue 4
Abstract
Stochastic gradient descent (SGD) is a simple and effective algorithm for solving the optimization problem of support vector machine, where each iteration operates on a single training example. The run-time of SGD does not depend directly on the size of the training set, the resulting algorithm is especially suited for learning from large datasets. However, the problem of stochastic gradient descent algorithm is that it is difficult to choose the proper learning rate. A learning rate is too small, which leads to slow convergence, while a learning rate that is too large can hinder convergence and cause fluctuate. In order to improve the efficiency and classification ability of SVM based on stochastic gradient descent algorithm, three algorithms of adaptive learning rate SGD are used to solve support vector machine, which are Adagrad, Adadelta and Adam. The experimental results show that the algorithm based on Adagrad, Adadelta and Adam for solving the linear support vector machine has faster convergence speed and higher testing precision.
Authors and Affiliations
Shuxia Lu, Zhao Jin
Keywords
Related Articles
- EP ID EP385890
- DOI -
- Views 69
- Downloads 0
A Specific Formula to Compute the Determinant of One Matrix of Order
Let be an matrix, where , In this paper, we establish a specific formula to calculate the determinant of matrix .
Observation on the Binary Quadratic Equation 3x2-8xy+3y2+2x+2y+6=0
The binary quadratic equation is studied for its non-trivial integral solutions. The recurrence relations satisfied by the solutions x and y are given. A few interesting properties among the solutions are presented.
A Study on Reasonable Energy Consumption Quota of Hotel
The study on reasonable energy consumption quota of hotel can be a hot topic in smart city. Based on the STIRPAT model, in the paper we establish the identity with the impact factors which effect the energy consumption o...
Forecasting Monthly Water Production in Gaza City Using a Seasonal ARIMA Model
This paper aimed to analyze seasonal time series data on monthly water production in the Gaza Strip during the period between 2006 and 2012 using the Box-Jenkins methodology. A model was proposed that could forecast futu...
Research of Imbalanced Data Classification in Data Mining
Classification is one of the most important research contents in data mining and traditional classification methods are relatively mature, when dealing with well-balanced data they can make good performances. But in real...