Big Data Cluster Processing Through Optimized Speculative Execution
Journal Title: International journal of Emerging Trends in Science and Technology - Year 2017, Vol 4, Issue 9
Abstract
A big parallel processing job can be delayed substantially as long as one of its many tasks is being assigned to an unreliable or congested machine. To tackle this so-called straggler problem, most parallel processing frameworks such as MapReduce have adopted various strategies under which the system may speculatively launch additional copies of the same task if its progress is abnormally slow when extra idling resource is available. In this paper, we focus on the design of speculative execution schemes for parallel processing clusters from an optimization perspective under different loading conditions. For the lightly loaded case, we analyze and propose one cloning scheme, namely, the Smart Cloning Algorithm (SCA) which is based on maximizing the overall system utility. We also derive the workload threshold under which SCA should be used for speculative execution. For the heavily loaded case, we propose the Enhanced Speculative Execution (ESE) algorithm which is an extension of the Microsoft Mantri scheme to mitigate stragglers. Our simulation results show SCA reduces the total job flowtime, i.e., the job delay/ response time by nearly 6% comparing to the speculative execution strategy of Microsoft Mantri. In addition, we show that the ESE Algorithm outperforms the Mantri baseline scheme by 71% in terms of the job flowtime while consuming the same amount of computation resource.
Authors and Affiliations
D. Sasi Redkha
Keywords
Related Articles
- EP ID EP245626
- DOI -
- Views 122
- Downloads 0
An Amazing Traditional Healer of Manipur Solanum Xanthocarpum (Leipungkhanga)
Solanum xanthocarpum (Solananceae) is traditionally used as pungent, bitter, digestive, diuretic, alternative, astringent and anthelmintic . It was also used in fever, cough, flatulence and heart diseases. Phytochemistry...
A Local Position of Recognition of Node Duplication Attacks in Wireless Networks
IP spoofing-based submerging attacks are a serious and open security problem on the current Internet. The best current anti-spoofing practices have long been implemented in modern routers. However, they are not sufficien...
Impact of FDI Inflows on Productivity of Commercial Banks in India- An Analysis
Foreign direct investment is considered to be the life blood of economic development, especially for the developing and underdeveloped countries. Foreign Direct Investment has become sin-quo-non for the economic developm...
Evolution of the SELF in Lady Oracle by Margaret Atwood
Feminism is essentially linked with cultural moorings and social taboos. Every country has its own cultural heritage. As fiction is bound to the mirror of a particular country, feminism has assumed new shapes and name. W...
Noble Metal Functionalized MoO3: NiO Nanocomposite for Fabrication of CO2Gas Sensor
Over the past few years, considerable interest has been focused on semiconducting nanoparticles due to their potential applications in diverse fields including catalysis, magnetic...