Big Data Analysis: Challenges and Solutions
Journal Title: International Journal of Scientific Research and Management - Year 2015, Vol 3, Issue 2
Abstract
We live in on - demand, on - command Digital universe with data prolife ring by Institutions, Individuals and Machines at a very high rate. This data is categories as "Big Data " due to its sheer Volume, Variety, Velocity and Veracity. Most of this data is unstructured, quasi structured or semi structured and it is heterogeneous in nature. The volume and the heterogeneity of data with the speed it is generated, makes it difficult for the present computing infrastructure to manage Big Data. Traditional data management, warehousing and analysis systems fall short of tools to analyze this data. Due to its specific nature of Big Data, it is stored in distributed file system architectu res. Hadoop and HDFS by Apache is widely used for storing and managing Big Data. Analyzing Big Data is a challenging task as it involves large distributed file system s which should be fault tolerant, flexible and scalable. Map Reduce is widely been used fo r the efficient analysis of Big Data. Traditional DBMS techniques like Joins and Indexing and other techniques like graph search is used for classification and clustering of Big Da ta. These techniques are being adopted to be used in Map Reduce. In this res earch paper the authors suggest various methods for catering to the problems in hand through Map Reduce framework over Hadoop Distributed File System (HDFS). Map Reduce is a Minimization technique which makes use of file indexing with mapping, sorting, shu ffling and finally reducing. Map Reduce techniques have been studied at in this paper which is implemented for Big Data analysis using HDFS.
Authors and Affiliations
Dipak M. Durgude
Keywords
Related Articles
- EP ID EP214706
- DOI -
- Views 78
- Downloads 0
Asse s sment on Cr, Cd, As, Ni and Pb uptake and phy remediation potential of Scirpus mucronatus
A laboratory experiment was conducted to examine the heavy metals (Cr, Cd, As, Ni and Pb) phytoaccumulation potential of an aquatic macrophytes Scirpus mucronatus. The macrophyte were transferr...
Impact Of Social Media Marketing In Business Engagemen
The social media era was started around ten years ago. It began with LinkedIn, which was launched in 2003, followed by MySpace and Facebook in 2004, YouTube in 2005, and Twitter in 2 006. In less than a decade, its...
Peace Education: A Pedagogical Emergent For Student Wellbeing
In present the entire world is running behind the materialistic living and only self - satisfaction of modern needs instead of thinking about the social world where they exist, which has made the living more complex and...
Operational Impact of ICTs in Front - Office of Selected Hotels in South - East and South - South Nigeria
The operation of Front - Office of a hotel or a holiday centre is dependent on the effectiveness and efficiency of services rendered. When hotel services are predominantly manual, timely services may not be witnessed at...
Impact of a Health E ducation Program on Improving Barakat Basic School Childre n's Knowledge about Malaria Prevention and Management , South Gezira
Background Malaria is an ancient scourge of humanity. Almost half the world's population lives in countries where the disease is endemic. Children are the worst affected by this disease. It may cause as m...