Revisit of Logistic Regression
Journal Title: International Journal of Advanced Computer Science & Applications - Year 2013, Vol 4, Issue 5
Abstract
Logistic regression (LR) is widely applied as a powerful classification method in various fields, and a variety of optimization methods have been developed. To cope with large-scale problems, an efficient optimization method for LR is required in terms of computational cost and memory usage. In this paper, we propose an efficient optimization method using non-linear conjugate gradient (CG) descent. In each CG iteration, the proposed method employs the optimized step size without exhaustive line search, which significantly reduces the number of iterations, making the whole optimization process fast. In addition, on the basis of such CG-based optimization scheme, a novel optimization method for kernel logistic regression (KLR) is proposed. Unlike the ordinary KLR methods, the proposed method optimizes the kernel-based classifier, which is naturally formulated as the linear combination of sample kernel functions, directly in the reproducing kernel Hilbert space (RKHS), not the linear coefficients. Subsequently, we also propose the multiple-kernel logistic regression (MKLR) along with the optimization of KLR. The MKLR effectively combines the multiple types of kernels with optimizing the weights for the kernels in the framework of the logistic regression. These proposed methods are all based on CG-based optimization and matrix-matrix computation which is easily parallelized such as by using multi-thread programming. In the experimental results on multi-class classifications using various datasets, the proposed methods exhibit favorable performances in terms of classification accuracies and computation times.
Authors and Affiliations
Takumi Kobayashi, Kenji Watanabe, Nobuyuki Otsu
Keywords
Related Articles
- EP ID EP87475
- DOI 10.14569/IJACSA.2013.040519
- Views 82
- Downloads 0
A Fast Method to Estimate Partial Weights Enumerators by Hash Techniques and Automorphism Group
BCH codes have high error correcting capability which allows classing them as good cyclic error correcting codes. This important characteristic is very useful in communication and data storage systems. Actually after alm...
Plethora of Cyber Forensics
As threats against digital assets have risen and there is necessitate exposing and eliminating hidden risks and threats. The ability of exposing is called “cyber forensics.” Cyber Penetrators have adopted more sophistic...
Improving Classification Accuracy of Heart Sound Signals Using Hierarchical MLP Network
Classification of heart sound signals to normal or their classes of disease are very important in screening and diagnosis system since various applications and devices that fulfilling this purpose are rapidly design and...
Challenges in Designing Ethical Rules for Infrastructures in Internet of Vehicles
Vehicular Ad-hoc Networks (VANETs) have seen significant advancements in technology. Innovation in connectivity and communication has brought substantial capabilities to various components of VANETs such as vehicles, inf...
Features Optimization for ECG Signals Classification
A new method is used in this work to classify ECG beats. The new method is about using an optimization algorithm for selecting the features of each beat then classify them. For each beat, twenty-four higher order statist...