Log Analysis with Hadoop MapReduce
Journal Title: Journal of Emerging Computer Technologies - Year 2021, Vol 1, Issue 1
Abstract
Pretty much every part of life now results in the generation of data. Logs are documentation of events or records of system activities and are created automatically through IT systems. Log data analysis is a process of making sense of these records. Log data often grows quickly and the conventional database solutions run short for dealing with a large volume of log files. Hadoop, having a wide area of applications for Big Data analysis, provides a solution for this problem. In this study, Hadoop was installed on two virtual machines. Log files generated by a Python script were analyzed in order to evaluate the system activities. The aim was to validate the importance of Hadoop in meeting the challenge of dealing with Big Data. The performed experiments show that analyzing logs with Hadoop MapReduce makes the data processing and detection of malfunctions and defects faster and simpler.
Authors and Affiliations
Mihiri Chathurika,Gligor Risteski,Beyza Ali,Atanas Hristov,
Keywords
Related Articles
- EP ID EP706078
- DOI -
- Views 117
- Downloads 0
Social Media Content Review of Popular MMORPG Games: Reddit Comment Scraping and Sentiment Analysis
Social media is a system that provides access from one-way information sharing to two-way and simultaneous information sharing, introducing Web 2.0 as a service to users. Social media is a set of dialogues and exchanges...
GoHammer Blockchain Performance Test Tool
Decentralized services are increasingly being developed and their Decentralized applications are increasingly developed but their performance metrics are not tested enough. The total number of transactions that can be su...
Face Recognition Based Multifunction IoT Smart Mailbox
The abundance and variety of the Internet of Things (IoT) applications in the recent couple of years has reach a new record. From smart home, smart traffic, smart city, to industry, logistics and agriculture IoT applicat...
Log Analysis with Hadoop MapReduce
Pretty much every part of life now results in the generation of data. Logs are documentation of events or records of system activities and are created automatically through IT systems. Log data analysis is a process of m...
Semantic Web and Hospital Appointment Ontology
Ontologies are a very important element for applications. They are used in many areas such as management information systems, e-commerce applications. Ontologies also play a major role in the new technology called Web 3....