Micro Agent and Neural Network based Model for Data Error Detection in a Real Time Data Stream
Journal Title: International Journal of Advanced Computer Science & Applications - Year 2019, Vol 10, Issue 7
Abstract
In this paper, we present a model for learning and detecting the presence of data type errors in a real time big data stream processing context. The proposed approach is based on a collection of micro-agents. Each micro-agent is trained to detect a specific type of error using an atomic neural network based on a sample multilayer perceptron. The supervised learning process is based on a binary classifier where the training data inputs are represented by data types and data values. The Micro-Agent for Error Detection (MAED) is deployed at several instances depending on the number of error types to be handled. The orchestration mechanism of data streams to be examined is performed by a special Host Micro Agent (HMA). This later receives in real time a data stream, splits the current record into several elementary fields. Each field value is streamed to an instance of MAED Agent which responds with a signal of presence or not of a specific data type error of the corresponding data field. For each detected data type error, the HMA Agent selects and performs the appropriate cleaning algorithm from a repository to correct the present errors of the data stream. To validate this approach, we propose an implementation based on Framework Deep Learning 4j for the Machines Learning part and JADE as a Multi Agent System (MAS) platform. The used dataset is generated by an events generator for smart highways.
Authors and Affiliations
Sidi Mohamed Snineh, Mohamed Youssfi, Abdelaziz Daaif, Omar Bouattane
Keywords
Related Articles
- EP ID EP611252
- DOI 10.14569/IJACSA.2019.0100725
- Views 104
- Downloads 0
An Improved Transformer for LLC Resonant Inverter for Induction Heating Applications
A new trend in power converters is to design a planar transformer that aims for low profile. However, at high frequency, the planar transformer AC losses become significant due to the proximity and skin effects. In this...
Modeling and Control by Multi-Model Approach of the Greenhouse Dynamical System with Multiple Time-delays
This paper presents the Internal Multi-Model Control IMMC for a multivariable discrete-time system with variable multiple delays. This work focus on the Greenhouse climate model as a multivariable time-delay system. In f...
Exploiting Document Level Semantics in Document Clustering
Document clustering is an unsupervised machine learning method that separates a large subject heterogeneous collection (Corpus) into smaller, more manageable, subject homogeneous collections (clusters). Traditional metho...
Comparative Analysis of Evolutionary Algorithms for Multi-Objective Travelling Salesman Problem
The Evolutionary Computation has grown much in last few years. Inspired by biological evolution, this field is used to solve NP-hard optimization problems to come up with best solution. TSP is most popular and complex pr...
Arijo: Location-Specific Data Crowdsourcing Web Application as a Curriculum Supplement
Smart devices are quickly becoming more accessible to the general public. With the proper tools, they can be used to supplement the work of educators. According to studies by Beeland Jr. and Roussou, learning through int...