A New Architecture for Real Time Data Stream Processing
Journal Title: International Journal of Advanced Computer Science & Applications - Year 2017, Vol 8, Issue 11
Abstract
Processing a data stream in real time is a crucial issue for several applications, however processing a large amount of data from different sources, such as sensor networks, web traffic, social media, video streams and other sources, represents a huge challenge. The main problem is that the big data system is based on Hadoop technology, especially MapReduce for processing. This latter is a high scalability and fault tolerant framework. It also processes a large amount of data in batches and provides perception blast insight of older data, but it can only process a limited set of data. MapReduce is not appropriate for real time stream processing, and is very important to process data the moment they arrive at a fast response and a good decision making. Ergo the need for a new architecture that allows real-time data processing with high speed along with low latency. The major aim of the paper at hand is to give a clear survey of the different open sources technologies that exist for real-time data stream processing including their system architectures. We shall also provide a brand new architecture which is mainly based on previous comparisons of real-time processing powered with machine learning and storm technology.
Authors and Affiliations
Soumaya Ounacer, Mohamed Amine TALHAOUI, Soufiane Ardchir, Abderrahmane Daif, Mohamed Azouazi
Keywords
Related Articles
- EP ID EP240422
- DOI 10.14569/IJACSA.2017.081106
- Views 106
- Downloads 0
Mobile Phone Operations using Human Eyes Only and its Applications
Mobile phone operations using human eyes only is proposed together with its applications for cooking with referring to recipes and manufacturing with referring to manuals, production procedure, so on. It is found that mo...
Addressing the Future Data Management Challenges in IoT: A Proposed Framework
Internet of Thing (IoT) has been attracting the interest of researchers in recent years. Traditionally, only handful types of devices had the capability to be connected to internet/intranet, but due to the latest develop...
Sentiment Analysis of Arabic Jordanian Dialect Tweets
Sentiment Analysis (SA) of social media contents has become one of the growing areas of research in data mining. SA provides the ability of text mining the public opinions of a subjective manner in real time. This paper...
Integrated Information System for reserving rooms in Hotels
It is very important to build new and modern flexible dynamic effective compatible reusable information systems including database to help manipulate different processes and deal with many parts around it. One of these i...
Image Enhancement Using Homomorphic Filtering and Adaptive Median Filtering for Balinese Papyrus (Lontar)
Balinese papyrus (Lontar) is one of the most popular media to write for more than a hundred years in Indonesia. Balinese papyrus are used to document things that are considered important in the past. Most of the balinese...