A New Architecture for Real Time Data Stream Processing
Journal Title: International Journal of Advanced Computer Science & Applications - Year 2017, Vol 8, Issue 11
Abstract
Processing a data stream in real time is a crucial issue for several applications, however processing a large amount of data from different sources, such as sensor networks, web traffic, social media, video streams and other sources, represents a huge challenge. The main problem is that the big data system is based on Hadoop technology, especially MapReduce for processing. This latter is a high scalability and fault tolerant framework. It also processes a large amount of data in batches and provides perception blast insight of older data, but it can only process a limited set of data. MapReduce is not appropriate for real time stream processing, and is very important to process data the moment they arrive at a fast response and a good decision making. Ergo the need for a new architecture that allows real-time data processing with high speed along with low latency. The major aim of the paper at hand is to give a clear survey of the different open sources technologies that exist for real-time data stream processing including their system architectures. We shall also provide a brand new architecture which is mainly based on previous comparisons of real-time processing powered with machine learning and storm technology.
Authors and Affiliations
Soumaya Ounacer, Mohamed Amine TALHAOUI, Soufiane Ardchir, Abderrahmane Daif, Mohamed Azouazi
Keywords
Related Articles
- EP ID EP240422
- DOI 10.14569/IJACSA.2017.081106
- Views 110
- Downloads 0
Improving Forecasting Accuracy in the Case of Intermittent Demand Forecasting
In making forecasting, there are many kinds of data. Stationary time series data are relatively easy to make forecasting but random data are very difficult in its execution for forecasting. Intermittent data are often se...
Frequency Domain Analysis for Assessing Fluid Responsiveness by Using Instantaneous Pulse Rate Variability
In the ICU, fluid therapy is conventional strategy for the patient in shock. However, only half of ICU patients have well-responses to fluid therapy, and fluid loading in non-responsive patient delays definitive therapy....
Path Planning in a Dynamic Environment
Path planning is an important area in the control of autonomous mobile robots. Recent work has focused on aspects reductions in processing time than the memory requirements. A dynamic environment uses a lot of memory and...
English-Arabic Hybrid Machine Translation System using EBMT and Translation Memory
The availability of a machine translation to translate from English-to-Arabic with high accuracy is not available because of the difficult morphology of the Arabic Language. A hybrid machine translation system between Ex...
Improvement of the Handover and Quality of Service on Software Defined Wireless Networks
The Wireless Fidelity (WiFi) is the business name given to the 802.11b and 802.11g IEEE standard by the WiFi Alliance, formerly known as Weca industry with more than 200 member companies dedicated to supporting the growt...