Big Data Cluster Processing Through Optimized Speculative Execution
Journal Title: International journal of Emerging Trends in Science and Technology - Year 2017, Vol 4, Issue 9
Abstract
A big parallel processing job can be delayed substantially as long as one of its many tasks is being assigned to an unreliable or congested machine. To tackle this so-called straggler problem, most parallel processing frameworks such as MapReduce have adopted various strategies under which the system may speculatively launch additional copies of the same task if its progress is abnormally slow when extra idling resource is available. In this paper, we focus on the design of speculative execution schemes for parallel processing clusters from an optimization perspective under different loading conditions. For the lightly loaded case, we analyze and propose one cloning scheme, namely, the Smart Cloning Algorithm (SCA) which is based on maximizing the overall system utility. We also derive the workload threshold under which SCA should be used for speculative execution. For the heavily loaded case, we propose the Enhanced Speculative Execution (ESE) algorithm which is an extension of the Microsoft Mantri scheme to mitigate stragglers. Our simulation results show SCA reduces the total job flowtime, i.e., the job delay/ response time by nearly 6% comparing to the speculative execution strategy of Microsoft Mantri. In addition, we show that the ESE Algorithm outperforms the Mantri baseline scheme by 71% in terms of the job flowtime while consuming the same amount of computation resource.
Authors and Affiliations
D. Sasi Redkha
Keywords
Related Articles
- EP ID EP245626
- DOI -
- Views 145
- Downloads 0
Control Strategy for Single-Phase Inverters in Distributed Generation Systems for improving Power Quality
This project deals with reactive power compensation and harmonic reduction by single-phase inverter for grid connected DG systems. The main theme is to integrate DG unit with shunt active Power filters. so that, the inve...
Asbestos Roofing” as “Housing Pattern” and Its Implications on Health of the Households in Sub Urban Area of Chennai
Background: Health implications of asbestos roofing are the first of its kind and there are no studies on it. Objectives: To see prevalence of health implications of the household members related to the asbestos roofing...
Madhya Pradesh Public Service Guarantees Act 2010 and Transparency of Administration
India comprises statutory laws which guarantee time bound delivery of services for various public services rendered by the Government to citizen and provides mechanism for punishing the errant public servant who is defic...
Phenology of Fingermillet (Eleusine coracana L.) in Relation to AgroClimatic Indices under Different Sowing Dates
A field experiment was conducted during kharif 2015 at Agricultural College Farm, Bapatla on sandy loam soil to study the phenology, accumulated growing degree days, photo thermal unit, helio-thermal unit, heat use effic...
Double Negative Metamaterial of Copper Split Ring and Graphite Materials
The work of this paper is to design and simulate a novel rectangular split ring and graphite wire structured having simultaneous negative permittivity and permeability so called double negative metamaterial or left hande...