Big Data Analytics for Net Flow Analysis in Distributed Environment using Hadoop
Journal Title: International Journal of Research in Computer and Communication Technology - Year 2015, Vol 4, Issue 7
Abstract
Network traffic measurement and analysis have been regularly performed on a high performance server that collects and analysis packet flow. When we monitor a large volume of network traffic data for detailed statistics, a large-scale network, it is not easy to handle Tera or Peta byte data with a single server, there is need to thousands of machines. As distributed parallel processing scheme have been recently developed due to the cluster file system, which beneficially applied to analyzing big network traffic data. Hadoop is a popular parallel processing framework that is widely used for working with large datasets. We analyze the netflow data monitoring single node to multi nodes hadoop cluster and provide an algorithm to calculate packet count and packet size of each source ip address for every fix interval of time, with low rate of false positives to detect malicious activity. Finally, we highlight performance and benefits of hadoop distributed cluster when we used large data sets as well as small data sets.
Authors and Affiliations
Amreesh Kumar Patel, D. S. Bhilare, Sushil buriya, Satyendra Singh Yadav
Keywords
Related Articles
- EP ID EP28209
- DOI -
- Views 283
- Downloads 2
Application of Component-Based Localization in Sparse Wireless Networks
Localization is crucial for wireless ad hoc and sensor networks. As the istance-measurement ranges are often less than the communication ranges for many ranging systems, most communication-dense wireless networks ar...
Routing Scheme for Exploring Opportunities in Ad Hoc Networks
OPPORTUNISTIC routing for multihop wireless ad hoc networks has seen recent research interest to overcome deficiencies of conventional routing. The proposed scheme utilizes a reinforcement learning framework to oppo...
Enhanced Sparse Coding Technique For Top Image List
Image reranking is successful for enhancing the execution of a content based picture seek. Be that as it may, existing reranking algorithms are constrained for two principle reasons: 1) the literary meta-information...
COMPARISON AND STATISTICAL ANALYSIS OF NAM AND NORMAL SPEECH PROCESSING USING WAVELET TRANSFORM
In this, we present statistical approaches to enhance body-conducted unvoiced speech for silent speech communication using wavelet transform. So far Analysis of NAM speech has been made only using HMM (Hidden Markov...
Evaluation of A Prototype ATPG System Using Rulesets From The Stanford And Internet2 Backbones
The most important sources of transparency for ATPG (Automatic Test Packet Generation) are polling the network periodically for forwarding state and performing all pair’s reachability. First we have freshly speeded u...