Big Data Cluster Processing Through Optimized Speculative Execution
Journal Title: International journal of Emerging Trends in Science and Technology - Year 2017, Vol 4, Issue 9
Abstract
A big parallel processing job can be delayed substantially as long as one of its many tasks is being assigned to an unreliable or congested machine. To tackle this so-called straggler problem, most parallel processing frameworks such as MapReduce have adopted various strategies under which the system may speculatively launch additional copies of the same task if its progress is abnormally slow when extra idling resource is available. In this paper, we focus on the design of speculative execution schemes for parallel processing clusters from an optimization perspective under different loading conditions. For the lightly loaded case, we analyze and propose one cloning scheme, namely, the Smart Cloning Algorithm (SCA) which is based on maximizing the overall system utility. We also derive the workload threshold under which SCA should be used for speculative execution. For the heavily loaded case, we propose the Enhanced Speculative Execution (ESE) algorithm which is an extension of the Microsoft Mantri scheme to mitigate stragglers. Our simulation results show SCA reduces the total job flowtime, i.e., the job delay/ response time by nearly 6% comparing to the speculative execution strategy of Microsoft Mantri. In addition, we show that the ESE Algorithm outperforms the Mantri baseline scheme by 71% in terms of the job flowtime while consuming the same amount of computation resource.
Authors and Affiliations
D. Sasi Redkha
Keywords
Related Articles
- EP ID EP245626
- DOI -
- Views 143
- Downloads 0
Mucoadhesive Microspheres Based Formulation Development of Ziprasidone Hydrochloride for Nasal Delivery
The most important criteria for developing novel drug delivery system are to achieve clinical efficacy. Mucoadhesive polymer like chitosan can be employed to increase the residence time of formulation in the nasal cavity...
Qualities and Skills of Leaders (With Reference To Kautilya Arthashastra)
Qualities and Skills are the most decisive factors for any leader to direct any organization systematically. In this article researchers made an endevour to understand the viewpoint of Acharya Chanakya relating to the qu...
Shetkari Bazar: An Alternative to the Problems of Unorganized Vegetable Market System in Latur City
In India, Rythu Bazaar (Farmers’ Market) concept was introduced in the state of Andhra Pradesh. This marketing system has given good results, with regard to prices of vegetables, benefits to the farmers and customers. At...
Effect of Monopole field on the Gravitational Collapse of Husain Space-Time
We study the effect of the monopole field on the occurrence of the naked singularities arising in Husain space-time. For an appropriate choice of the arbitrary functions, the outgoing radial null geodesics, emanating fro...
Experimental Study on the Hardened Properties of Concrete by Using Soft Drink Bottle Caps as Partial Replacement for Coarse Aggregates
Cement concrete is the most extensively used construction material in the world because of its great workability and can be moulded to any shape. Ordinary cement concrete possesses a very low tensile strength, limited du...