Revisit of Logistic Regression
Journal Title: International Journal of Advanced Computer Science & Applications - Year 2013, Vol 4, Issue 5
Abstract
Logistic regression (LR) is widely applied as a powerful classification method in various fields, and a variety of optimization methods have been developed. To cope with large-scale problems, an efficient optimization method for LR is required in terms of computational cost and memory usage. In this paper, we propose an efficient optimization method using non-linear conjugate gradient (CG) descent. In each CG iteration, the proposed method employs the optimized step size without exhaustive line search, which significantly reduces the number of iterations, making the whole optimization process fast. In addition, on the basis of such CG-based optimization scheme, a novel optimization method for kernel logistic regression (KLR) is proposed. Unlike the ordinary KLR methods, the proposed method optimizes the kernel-based classifier, which is naturally formulated as the linear combination of sample kernel functions, directly in the reproducing kernel Hilbert space (RKHS), not the linear coefficients. Subsequently, we also propose the multiple-kernel logistic regression (MKLR) along with the optimization of KLR. The MKLR effectively combines the multiple types of kernels with optimizing the weights for the kernels in the framework of the logistic regression. These proposed methods are all based on CG-based optimization and matrix-matrix computation which is easily parallelized such as by using multi-thread programming. In the experimental results on multi-class classifications using various datasets, the proposed methods exhibit favorable performances in terms of classification accuracies and computation times.
Authors and Affiliations
Takumi Kobayashi, Kenji Watanabe, Nobuyuki Otsu
Keywords
Related Articles
- EP ID EP87475
- DOI 10.14569/IJACSA.2013.040519
- Views 94
- Downloads 0
SVM Optimization for Sentiment Analysis
Exponential growth in mobile technology and mini computing devices has led to a massive increment in social media users, who are continuously posting their views and comments about certain products and services, which ar...
Impact of Distributed Generation on the Reliability of Local Distribution System
With the growth of distributed generation (DG) and renewable energy resources the power sector is becoming more sophisticated, distributed generation technologies with its diverse impacts on power system is becoming attr...
Clustering Students’ Arabic Tweets using Different Schemes
In this paper, Twitter has been chosen as a platform for clustering the topics that have been mentioned by King Abdulaziz University students to understand students’ behaviours and answer their inquiries. The aim of the...
Towards the Performance Investigation of Automatic Melanoma Diagnosis Applications
Melanoma is a type of skin cancer, one of the fatal diseases, that appears as an abnormal growth of skin cells and the lesion part often looks like a mole on the skin. Early detection of melanoma from skin lesion by mean...
A Comparative Study of various Secure Routing Protocols based on AODV
The paper surveyed and compared various secure routing protocols for the mobile ad hoc networks (MANETs). MANETs are vulnerable to various security threats because of its dynamic topology and selfconfigurable nature. Sec...