Revisit of Logistic Regression
Journal Title: International Journal of Advanced Computer Science & Applications - Year 2013, Vol 4, Issue 5
Abstract
Logistic regression (LR) is widely applied as a powerful classification method in various fields, and a variety of optimization methods have been developed. To cope with large-scale problems, an efficient optimization method for LR is required in terms of computational cost and memory usage. In this paper, we propose an efficient optimization method using non-linear conjugate gradient (CG) descent. In each CG iteration, the proposed method employs the optimized step size without exhaustive line search, which significantly reduces the number of iterations, making the whole optimization process fast. In addition, on the basis of such CG-based optimization scheme, a novel optimization method for kernel logistic regression (KLR) is proposed. Unlike the ordinary KLR methods, the proposed method optimizes the kernel-based classifier, which is naturally formulated as the linear combination of sample kernel functions, directly in the reproducing kernel Hilbert space (RKHS), not the linear coefficients. Subsequently, we also propose the multiple-kernel logistic regression (MKLR) along with the optimization of KLR. The MKLR effectively combines the multiple types of kernels with optimizing the weights for the kernels in the framework of the logistic regression. These proposed methods are all based on CG-based optimization and matrix-matrix computation which is easily parallelized such as by using multi-thread programming. In the experimental results on multi-class classifications using various datasets, the proposed methods exhibit favorable performances in terms of classification accuracies and computation times.
Authors and Affiliations
Takumi Kobayashi, Kenji Watanabe, Nobuyuki Otsu
Keywords
Related Articles
- EP ID EP87475
- DOI 10.14569/IJACSA.2013.040519
- Views 98
- Downloads 0
Repository of Static and Dynamic Signs
Gesture-based communication is on the rise in Human Computer Interaction. Advancement in the form of smart phones has made it possible to introduce a new kind of communication. Gesture-based interfaces are increasingly g...
On the Projection Matrices Influence in the Classification of Compressed Sensed ECG Signals
In this paper the classification results of compressed sensed ECG signals based on various types of projection matrices is investigated. The compressed signals are classified using the KNN (K-Nearest Neighbour) algorithm...
Adaptive Cache Replacement:A Novel Approach
Cache replacement policies are developed to help insure optimal use of limited resources. Varieties of such algorithms exist with relatively few that dynamically adapt to traffic patterns. Algorithms that are tunable typ...
Efficient Retrieval of Text for Biomedical Domain using Data Mining Algorithm
Data mining, a branch of computer science [1], is the process of extracting patterns from large data sets by combining methods from statistics and artificial intelligence with database management. Data mining is se...
The threshold EM algorithm for parameter learning in bayesian network with incomplete data
Bayesian networks (BN) are used in a big range of applications but they have one issue concerning parameter learning. In real application, training data are always incomplete or some nodes are hidden. To deal with this p...