Big Data Cluster Processing Through Optimized Speculative Execution
Journal Title: International journal of Emerging Trends in Science and Technology - Year 2017, Vol 4, Issue 9
Abstract
A big parallel processing job can be delayed substantially as long as one of its many tasks is being assigned to an unreliable or congested machine. To tackle this so-called straggler problem, most parallel processing frameworks such as MapReduce have adopted various strategies under which the system may speculatively launch additional copies of the same task if its progress is abnormally slow when extra idling resource is available. In this paper, we focus on the design of speculative execution schemes for parallel processing clusters from an optimization perspective under different loading conditions. For the lightly loaded case, we analyze and propose one cloning scheme, namely, the Smart Cloning Algorithm (SCA) which is based on maximizing the overall system utility. We also derive the workload threshold under which SCA should be used for speculative execution. For the heavily loaded case, we propose the Enhanced Speculative Execution (ESE) algorithm which is an extension of the Microsoft Mantri scheme to mitigate stragglers. Our simulation results show SCA reduces the total job flowtime, i.e., the job delay/ response time by nearly 6% comparing to the speculative execution strategy of Microsoft Mantri. In addition, we show that the ESE Algorithm outperforms the Mantri baseline scheme by 71% in terms of the job flowtime while consuming the same amount of computation resource.
Authors and Affiliations
D. Sasi Redkha
Keywords
Related Articles
- EP ID EP245626
- DOI -
- Views 107
- Downloads 0
Dual Sentiment Analysis with Three-Stage Model for Complex Polarity Shift Patterns with Two Sides of One Review
Sentiment classification is a unique process of text categorization whose objective is to categorize a text related to the sentimental polarities of opinions it consists of constructive or adverse, positive or negative....
Smart Accident Detection & Assistance System
The numbers of deaths due to road accidents are increasing day by day. Even the vehicles are becoming more sophisticated with features like GPS navigation and airbags for safety. In spite of this the victims of the accid...
Seismic Performance for Fixed Base and Base Isolated Reinforced Concrete Structure
The base isolation systems are gaining large attention as a mean to protect structure form seismic hazard. It is a suitable technology for earthquake resistant design of variety of structure like buildings, bridges, airp...
Execution of Soft Handover in UMTS Using OPNET Simulator
Mobility is the main advantage of mobile cellular systems. Ability to communicate anywhere, at any time was the great success of wireless communications in 90’s. Now a day’s, continuous service is achieved by supporting...
Back Propagation Neural Network Based Image Compression
Compression algorithms are methods that reduce the number of symbols used to represent source information, therefore reducing the amount of space needed to store the source information or the amount of time necessary to...