Micro Agent and Neural Network based Model for Data Error Detection in a Real Time Data Stream
Journal Title: International Journal of Advanced Computer Science & Applications - Year 2019, Vol 10, Issue 7
Abstract
In this paper, we present a model for learning and detecting the presence of data type errors in a real time big data stream processing context. The proposed approach is based on a collection of micro-agents. Each micro-agent is trained to detect a specific type of error using an atomic neural network based on a sample multilayer perceptron. The supervised learning process is based on a binary classifier where the training data inputs are represented by data types and data values. The Micro-Agent for Error Detection (MAED) is deployed at several instances depending on the number of error types to be handled. The orchestration mechanism of data streams to be examined is performed by a special Host Micro Agent (HMA). This later receives in real time a data stream, splits the current record into several elementary fields. Each field value is streamed to an instance of MAED Agent which responds with a signal of presence or not of a specific data type error of the corresponding data field. For each detected data type error, the HMA Agent selects and performs the appropriate cleaning algorithm from a repository to correct the present errors of the data stream. To validate this approach, we propose an implementation based on Framework Deep Learning 4j for the Machines Learning part and JADE as a Multi Agent System (MAS) platform. The used dataset is generated by an events generator for smart highways.
Authors and Affiliations
Sidi Mohamed Snineh, Mohamed Youssfi, Abdelaziz Daaif, Omar Bouattane
Keywords
Related Articles
- EP ID EP611252
- DOI 10.14569/IJACSA.2019.0100725
- Views 111
- Downloads 0
Fuzzy Based Evaluation of Software Quality Using Quality Models and Goal Models
Software quality requirements are essential part for the success of software development. Defined and guaranteed quality in software development requires identifying, refining, and predicting quality properties by approp...
Automatic Classification and Segmentation of Brain Tumor in CT Images using Optimal Dominant Gray level Run length Texture Features
Tumor classification and segmentation from brain computed tomography image data is an important but time consuming task performed manually by medical experts. Automating this process is challenging due to the high divers...
An Agent-based Simulation for Studying Air Pollution from Traffic in Urban Areas: The Case of Hanoi City
In urban areas, traffic is one of the main causes of air pollution. Establishing an effective solution to raise public awareness of this phenomenon could help to significantly reduce the level of pollution in urban areas...
LNG Import Contract in the perspective of Associated Technical and Managerial Challenges for the Distribution Companies of Pakistan
Energy Managers and Government Office Holders in Pakistan are nowadays pondering over multiple options for the resolution of ongoing Energy crises in the country. LNG (Liquefied Natural Gas) import has been finalized for...
Impact of Medical Technology on Expansion in Healthcare Expenses
The impact of medical technology on expansion in health care expenses has long been a subject of essential interest, mainly in the context of long-term outcrops of health spending, which must deal with the issue of the a...