Log Analysis with Hadoop MapReduce
Journal Title: Journal of Emerging Computer Technologies - Year 2021, Vol 1, Issue 1
Abstract
Pretty much every part of life now results in the generation of data. Logs are documentation of events or records of system activities and are created automatically through IT systems. Log data analysis is a process of making sense of these records. Log data often grows quickly and the conventional database solutions run short for dealing with a large volume of log files. Hadoop, having a wide area of applications for Big Data analysis, provides a solution for this problem. In this study, Hadoop was installed on two virtual machines. Log files generated by a Python script were analyzed in order to evaluate the system activities. The aim was to validate the importance of Hadoop in meeting the challenge of dealing with Big Data. The performed experiments show that analyzing logs with Hadoop MapReduce makes the data processing and detection of malfunctions and defects faster and simpler.
Authors and Affiliations
Mihiri Chathurika,Gligor Risteski,Beyza Ali,Atanas Hristov,
Keywords
Related Articles
- EP ID EP706078
- DOI -
- Views 115
- Downloads 0
Digital Twin Based Disaster Management System Proposal: DT-DMS
The damage and the impact of natural disasters are becoming more destructive with the increase of urbanization. Today's metropolitan cities are not sufficiently prepared for the pre and post-disaster situations. Digital...
Development of An Efficient Tool to Convert Regular Expressions to NFA
In the computing theory, while the term “Language” specifies the string set, the term “Regular Expressions” means the notation that builds, creates and generates these languages. Also, the term “Regular Expressions” crea...
Semantic Web and Hospital Appointment Ontology
Ontologies are a very important element for applications. They are used in many areas such as management information systems, e-commerce applications. Ontologies also play a major role in the new technology called Web 3....
Ensemble Regression-Based Gold Price (XAU/USD) Prediction
The prediction of any commodities such as cryptocurrency, stocks, silver, gold is a challenging task for the investors, researchers, and analysts. In this work, we propose a model that forecasts the value of 1 ounce of g...
Vehicle Detection with HOG and Linear SVM
In this paper, we present a vehicle detection system by employing Histogram of Oriented Gradients (HOG) for feature extraction and linear SVM for classification. We study the influence of the colour space on the performa...