Revisit of Logistic Regression
Journal Title: International Journal of Advanced Computer Science & Applications - Year 2013, Vol 4, Issue 5
Abstract
Logistic regression (LR) is widely applied as a powerful classification method in various fields, and a variety of optimization methods have been developed. To cope with large-scale problems, an efficient optimization method for LR is required in terms of computational cost and memory usage. In this paper, we propose an efficient optimization method using non-linear conjugate gradient (CG) descent. In each CG iteration, the proposed method employs the optimized step size without exhaustive line search, which significantly reduces the number of iterations, making the whole optimization process fast. In addition, on the basis of such CG-based optimization scheme, a novel optimization method for kernel logistic regression (KLR) is proposed. Unlike the ordinary KLR methods, the proposed method optimizes the kernel-based classifier, which is naturally formulated as the linear combination of sample kernel functions, directly in the reproducing kernel Hilbert space (RKHS), not the linear coefficients. Subsequently, we also propose the multiple-kernel logistic regression (MKLR) along with the optimization of KLR. The MKLR effectively combines the multiple types of kernels with optimizing the weights for the kernels in the framework of the logistic regression. These proposed methods are all based on CG-based optimization and matrix-matrix computation which is easily parallelized such as by using multi-thread programming. In the experimental results on multi-class classifications using various datasets, the proposed methods exhibit favorable performances in terms of classification accuracies and computation times.
Authors and Affiliations
Takumi Kobayashi, Kenji Watanabe, Nobuyuki Otsu
Keywords
Related Articles
- EP ID EP87475
- DOI 10.14569/IJACSA.2013.040519
- Views 80
- Downloads 0
DoS/DDoS Detection for E-Healthcare in Internet of Things
Internet of Things (IoT) has emerged as a new horizon in communication age. IoT has provided platform to various emerging technologies and applications for growth. E-Health services have also been integrated and greatly...
Determination of Weighting Assessment on DREAD Model using Profile Matching
Web application creators often get lack understanding of security threats that can occur in applications that are made, while security threats can create new problems that are more complex. These security threats will po...
Identifying Dynamic Topics of Interest across Social Networks
Information propagation plays a significant role in online social networks, mining the latent information produced became crucial to understand how information is disseminated. It can be used for market prediction, rumor...
A Multiple-Objects Recognition Method Based on Region Similarity Measures: Application to Roof Extraction from Orthophotoplans
In this paper, an efficient method for automatic and accurate detection of multiple objects from images using a region similarity measure is presented. This method involves the construction of two knowledge databases: Th...
Establishing News Credibility using Sentiment Analysis on Twitter
The widespread use of Internet has resulted in a massive number of websites, blogs and forums. People can easily discuss with each other about different topics and products, and can leave reviews to help out others. This...