Big Data Analysis: Challenges and Solutions
Journal Title: International Journal of Scientific Research and Management - Year 2015, Vol 3, Issue 2
Abstract
We live in on - demand, on - command Digital universe with data prolife ring by Institutions, Individuals and Machines at a very high rate. This data is categories as "Big Data " due to its sheer Volume, Variety, Velocity and Veracity. Most of this data is unstructured, quasi structured or semi structured and it is heterogeneous in nature. The volume and the heterogeneity of data with the speed it is generated, makes it difficult for the present computing infrastructure to manage Big Data. Traditional data management, warehousing and analysis systems fall short of tools to analyze this data. Due to its specific nature of Big Data, it is stored in distributed file system architectu res. Hadoop and HDFS by Apache is widely used for storing and managing Big Data. Analyzing Big Data is a challenging task as it involves large distributed file system s which should be fault tolerant, flexible and scalable. Map Reduce is widely been used fo r the efficient analysis of Big Data. Traditional DBMS techniques like Joins and Indexing and other techniques like graph search is used for classification and clustering of Big Da ta. These techniques are being adopted to be used in Map Reduce. In this res earch paper the authors suggest various methods for catering to the problems in hand through Map Reduce framework over Hadoop Distributed File System (HDFS). Map Reduce is a Minimization technique which makes use of file indexing with mapping, sorting, shu ffling and finally reducing. Map Reduce techniques have been studied at in this paper which is implemented for Big Data analysis using HDFS.
Authors and Affiliations
Dipak M. Durgude
Keywords
Related Articles
- EP ID EP214706
- DOI -
- Views 75
- Downloads 0
Analysis of consumer behavior in a small size market entity: case study for Vlora District, Albania
In standard econometric application all variables are analyzed statistically before being used in mathematical models. In this framework we considered non-stationary distribution as an starting procedure on the study of...
Present Status of Agriclinics and Agribusiness Centers Scheme in India: An Analysis
Agriclinics and agribusiness centres scheme is a subsidy based credit linked scheme for setting up agriventure by agricultural graduates launched by government of India towards strengthen tech...
Comparative Evaluation Of Anticancer Potential Of Moringaoleifera , Ganodermalucidumand Silver Nanoparticles Against B reast And Liver Cancer Cell Lines And Related Pro And Anti Apoptotic Genes Profile
The present work aimed to evaluate the anticancer potentials concerning the cytotoxicity and anti - proliferative activity of L - amino acid Oxidase (LAAO) ,MoringaOleifera(MO) Methanol (MLM) and water extracts (MLW), Ga...
Food Safety - The Need Of The Hour
Access to sufficient a mount of safe and nutritious food is key to sustaining life and promoting good health. Unsafe food containing harmful viruses, parasites or chemicals cause more than 20...
Watermarking Using Bit Plane Complexity Segmentation and Artificial Neural Network
Digital Watermarking is th e act of hiding a message related to an image within the image itself. Watermarking has many desirable properties like effectiveness, image fidelity and robustness. Multilayer artificial neu...