Revisit of Logistic Regression
Journal Title: International Journal of Advanced Computer Science & Applications - Year 2013, Vol 4, Issue 5
Abstract
Logistic regression (LR) is widely applied as a powerful classification method in various fields, and a variety of optimization methods have been developed. To cope with large-scale problems, an efficient optimization method for LR is required in terms of computational cost and memory usage. In this paper, we propose an efficient optimization method using non-linear conjugate gradient (CG) descent. In each CG iteration, the proposed method employs the optimized step size without exhaustive line search, which significantly reduces the number of iterations, making the whole optimization process fast. In addition, on the basis of such CG-based optimization scheme, a novel optimization method for kernel logistic regression (KLR) is proposed. Unlike the ordinary KLR methods, the proposed method optimizes the kernel-based classifier, which is naturally formulated as the linear combination of sample kernel functions, directly in the reproducing kernel Hilbert space (RKHS), not the linear coefficients. Subsequently, we also propose the multiple-kernel logistic regression (MKLR) along with the optimization of KLR. The MKLR effectively combines the multiple types of kernels with optimizing the weights for the kernels in the framework of the logistic regression. These proposed methods are all based on CG-based optimization and matrix-matrix computation which is easily parallelized such as by using multi-thread programming. In the experimental results on multi-class classifications using various datasets, the proposed methods exhibit favorable performances in terms of classification accuracies and computation times.
Authors and Affiliations
Takumi Kobayashi, Kenji Watanabe, Nobuyuki Otsu
Keywords
Related Articles
- EP ID EP87475
- DOI 10.14569/IJACSA.2013.040519
- Views 95
- Downloads 0
Implementation of a Hierarchical Hybrid Intrusion Detection Mechanism in Wireless Sensors Network
During the last years, Wireless Sensor Networks (WSNs) have attracted considerable attention within the scientific community. The applications based on Wireless Sensor Networks, whose areas include, agriculture, military...
Implementation of a Formal Software Requirements Ambiguity Prevention Tool
The success of the software engineering process depends heavily on clear unambiguous software requirements. Ambiguity refers to the possibility to understand a requirement in more than one way. Unfortunately, ambiguity i...
Billing System Design Based on Internet Environment
This paper deals with the design of Internet billing system, in which it is possible pay invoices electronically. This approach is implemented via virtual banks, in which the process of money transfer can be implemented....
A Novel Assessment to Achieve Maximum Efficiency in Optimizing Software Failures
Software Reliability is a specialized area of software engineering which deals with the identification of failures while developing the software. Effective analysis of the reliability helps to signify the number of failu...
Urdu Sentiment Analysis
Internet is the most significant source of getting up thoughts, surveys for a product, and reviews for any type of service or activity. A Bulky amount of reviews are produced on daily basis on the cyberspace about online...