Log Analysis with Hadoop MapReduce
Journal Title: Journal of Emerging Computer Technologies - Year 2021, Vol 1, Issue 1
Abstract
Pretty much every part of life now results in the generation of data. Logs are documentation of events or records of system activities and are created automatically through IT systems. Log data analysis is a process of making sense of these records. Log data often grows quickly and the conventional database solutions run short for dealing with a large volume of log files. Hadoop, having a wide area of applications for Big Data analysis, provides a solution for this problem. In this study, Hadoop was installed on two virtual machines. Log files generated by a Python script were analyzed in order to evaluate the system activities. The aim was to validate the importance of Hadoop in meeting the challenge of dealing with Big Data. The performed experiments show that analyzing logs with Hadoop MapReduce makes the data processing and detection of malfunctions and defects faster and simpler.
Authors and Affiliations
Mihiri Chathurika,Gligor Risteski,Beyza Ali,Atanas Hristov,
Keywords
Related Articles
- EP ID EP706078
- DOI -
- Views 130
- Downloads 0
Ensemble Regression-Based Gold Price (XAU/USD) Prediction
The prediction of any commodities such as cryptocurrency, stocks, silver, gold is a challenging task for the investors, researchers, and analysts. In this work, we propose a model that forecasts the value of 1 ounce of g...
Development of An Efficient Tool to Convert Regular Expressions to NFA
In the computing theory, while the term “Language” specifies the string set, the term “Regular Expressions” means the notation that builds, creates and generates these languages. Also, the term “Regular Expressions” crea...
Automatic Movie Rating by Using Twitter Sentiment Analysis and Monitoring Tool
Today, due to the intense use of social media platforms such as Twitter by all segments of today's technology, people have begun to share their views, ideas, and feelings through these media. It is possible to discover m...
A Dynamic Method and Program for Multiple Password Generation and Management
Authentication is a process that users must do first to gain access to the websites/services they receive service from. Today, the most common method used for authentication is still text-based passwords. But, some diffi...
Edge Computing Security with an IoT device
Information technologies are changing every aspect of human life day by day. In this context, Edge Computing, Internet of Things, Machine Learning and Big Data Analytics technologies are thought to be a part of this chan...