A Novel Approach for Eliminating Duplicates in Large Dataset
Journal Title: International Journal for Research in Applied Science and Engineering Technology (IJRASET) - Year 2016, Vol 4, Issue 8
Abstract
One of the serious problems faced in several applications with personal details management, customer affiliation management, data mining, etc is duplicate detection. This survey deals with the various duplicate record detection techniques in both small and large datasets. To detect the duplicity with less time of execution and also without disturbing the dataset quality, methods like Progressive Blocking and Progressive Neighborhood are used. Progressive sorted neighborhood method also called as PSNM is used in this model for finding or detecting the duplicate in a parallel approach. Progressive Blocking algorithm works on large datasets where finding duplication requires immense time. These algorithms are used to enhance duplicate detection system. The efficiency can be doubled over the conventional duplicate detection method using this algorithm. Several different methods of data analysis are studied here with various approaches for duplicate detection.
Authors and Affiliations
N Chaitanya, Appini Narayanarao, M. Srinivasulu
Keywords
Related Articles
- EP ID EP22510
- DOI -
- Views 234
- Downloads 5
Implementation of Wireless Sensor Networks Using ZigBee
WSNs are usually composed of small, low cost devices that communicate wirelessly and have the capabilities of processing, sensing and storing. A sensor network consists of an array of numerous sensor networks of diverse...
Model Survey on Policy Making for Software Defined Networks
As the nature of threat is evolving day by day so it very important that network defence method should also evolve. This lead in increased demand of Software Defined Network(SDN) and OpenFlow, the policy based network m...
Recent Review on Trending Routing Protocols for Data Transmission
the route creation is done by inter-connecting the most adjacent nodes. Routing is the main issue in wireless network. The whole data delivery depends upon this because if the selected route is not appropriate or effici...
Fabrication of Engine Operated Weeder
As the situation of Indian farmer to increase the fertility and productivity of per unit area of that land it is essential to have vital agricultural implements which farmer can use and allow them to use for custom hiri...
Novel Switched-Capacitor Inverter Using Series/Parallel Conversion For 11-Level
In this paper Switched capacitor (SC) inverter is employed to amend high output voltage than the input voltage with minimal number of switches. A new series/parallel topology is proposed this require switches and capaci...