Workload Optimization by Horizontal Aggregation in SQL for Data Mining Analysis
Journal Title: International Journal of Advanced Research in Computer Engineering & Technology(IJARCET) - Year 2012, Vol 1, Issue 8
Abstract
— preparing a data set for analysis is generally the most time consuming task in a data mining project, requiring many complex SQL queries, joining tables, and aggregating columns. Existing SQL aggregations have limitations to prepare data sets because they return one column per aggregated group. In general, a significant manual effort is required to build data sets, where a horizontal layout is required. We propose simple, yet powerful, methods to generate SQL code to return aggregated columns in a horizontal tabular layout, returning a set of numbers instead of one number per row. This new class of functions is called horizontal aggregations. Horizontal aggregations build data sets with a horizontal denormalized layout (e.g., point-dimension, observation variable, instance-feature), which is the standard layout required by most data mining algorithms. We propose three fundamental methods to evaluate horizontal aggregations: CASE: Exploiting the programming CASE construct; SPJ: Based on standard relational algebra operators (SPJ queries); PIVOT: Using the PIVOT operator which is offered by some DBMSs. Experiments with large tables compare the proposed query evaluation methods. Our CASE method has similar speed to the PIVOT operator and it is much faster than the SPJ method. In general, the CASE and PIVOT methods exhibit linear scalability, whereas the SPJ method does not. For query optimization the distance computation and nearest cluster in the k-means are based on SQL. Work load balancing is the assignment of work to processors in a way that maximizes application performance. The process of load balancing can be generalized into four basic steps: 1. Monitoring processor load and state; 2. Exchanging workload and state information between processors; 3. Decision making; 4. Data migration. The decision phase is triggered when the load imbalance is detected to calculate optimal data redistribution. In the fourth and last phase, data migrates from overloaded processors to under-loaded ones
Authors and Affiliations
Mr. Prasanna M. Rathod , Prof. Karuna G. Bagde,
Keywords
Related Articles
- EP ID EP130940
- DOI -
- Views 74
- Downloads 0
DATA SHARING IN THE CLOUD USING DISTRIBUTED ACCOUNTABILITY
Now a day’s Cloud Computing is the rapid growing technology. Now most of the persons are using Cloud Computing technology .Cloud computing enables highly scalable services to be easily consumed over the Internet on an as...
A Survey on Clustering Based Image Segmentation
In computer vision, segmentation refers to the process of partitioning a digital image into multiple segments (Sets of pixels, also known as super pixels). This paper is a survey on various clustering techniques...
Data-warehousing on Cloud Computing
Our everyday data processing activities create massive amounts of data.Cloud Computing has emerged as a new paradigm for hosting and delivering services over the internet. Cloud computing is attractive to busin...
Design of Colour and Texture Based Relevant Image Search Engine
Online Navigation behavior grows each passing day, due to the interest of people in digital images is growing day by day; so the Users in many professional fields are exploiting the opportunities offered by the abi...
EFFICIENT LOCALIZED DEPLOYMENT ALGORITHM WITH BALANCED ENERGY CONSUMPTION IN WIRELESS SENSOR NETWORKS
A Wireless Sensor Network (WSN) consists of spatially distributed autonomous sensors to cooperatively monitor physical or environmental conditions , such as temperature, sound, pressure, motion or pollutants. WSN...