A New Architecture for Real Time Data Stream Processing
Journal Title: International Journal of Advanced Computer Science & Applications - Year 2017, Vol 8, Issue 11
Abstract
Processing a data stream in real time is a crucial issue for several applications, however processing a large amount of data from different sources, such as sensor networks, web traffic, social media, video streams and other sources, represents a huge challenge. The main problem is that the big data system is based on Hadoop technology, especially MapReduce for processing. This latter is a high scalability and fault tolerant framework. It also processes a large amount of data in batches and provides perception blast insight of older data, but it can only process a limited set of data. MapReduce is not appropriate for real time stream processing, and is very important to process data the moment they arrive at a fast response and a good decision making. Ergo the need for a new architecture that allows real-time data processing with high speed along with low latency. The major aim of the paper at hand is to give a clear survey of the different open sources technologies that exist for real-time data stream processing including their system architectures. We shall also provide a brand new architecture which is mainly based on previous comparisons of real-time processing powered with machine learning and storm technology.
Authors and Affiliations
Soumaya Ounacer, Mohamed Amine TALHAOUI, Soufiane Ardchir, Abderrahmane Daif, Mohamed Azouazi
Keywords
Related Articles
- EP ID EP240422
- DOI 10.14569/IJACSA.2017.081106
- Views 86
- Downloads 0
A Proposed Integrated Approach for BI and GIS in Health Sector to Support Decision Makers (BIGIS-DSS)
This paper explores the possibilities of adopting Business Intelligence (BI), and Geographic Information System (GIS) to build a spatial intelligence and predictive analytical approach. The proposed approach will help in...
Learners’ Attitudes Towards Extended-Blended Learning Experience Based on the S2P Learning Model
Within the Moroccan context, the Higher Education Institutions have realized the importance of the integration of information technologies into the formal learning curriculum. However, the risks of demotivation remain la...
Real Time RNA Sequence Edition with Matrix Insertion Deletion for Improved Bio Molecular Computing using Template Match Measure
The RNA sequence editing has become a challenging task in the molecular computing. There are number of approaches that have been discussed earlier for the problem RNA editing in bio molecular computing, but they suffer t...
A Game Theoretic Approach to Demand Side Management in Smart Grid with Multiple Energy Sources and Storage
A smart grid is an advancement in electrical grid which includes a variety of operational and energy measures. To utilize energy distribution in an efficient manner, demand side management has become the fore-runner. In ou...
E-Learning for Secondary and Higher Education Sectors: A Survey
Electronic learning (e-learning) has gained reasonable acceptance from educational institutions at all levels. There are various studies conducted by researchers considering different aspects of e-learning to investigate...