Duplicates Detection Within Incomplete Data Sets Using Blocking and Dynamic Sorting Key Methods
Journal Title: International Journal of Advanced Computer Science & Applications - Year 2018, Vol 9, Issue 9
Abstract
In database records duplicate detection, blocking method is commonly used to reduce the number of comparisons between the candidate record pairs. The main procedure in this method requires selecting attributes that will be used as sorting keys. Selection accuracy is essential in clustering candidates records that are likely matched in the same block. Nevertheless, the presence of missing values affects the creation of sorting keys and this is particularly undesirable if it involves the attributes that are used as the sorting keys. This is because, consequently, records that are supposed to be included in the duplicate detection procedure will be excluded from being examined. Thus, in this paper, we propose a method that can deal with the impact of missing values by using a dynamic sorting key. Dynamic sorting is an extension of blocking method that essentially works on two functions namely uniqueness calculation function (UF) (to choose unique attributes) and completeness function (CF) (to search for missing values). We experimented a particular blocking method called as sorted neighborhood with a dynamic sorting key on a restaurant data set (that consists of duplicate records) obtained from earlier research in order to evaluate the method’s accuracy and speed. Hypothetical missing values were applied to testing data set used in the experiment, where we compare the results of duplicate detection with (and without) dynamic sorting key. The result shows that, even though missing values are present, there is a promising improvement in the partitioning of duplicate records in the same block.
Authors and Affiliations
Abdulrazzak Ali, Nurul A. Emran, Siti A. Asmai, Awsan Thabet
Keywords
Related Articles
- EP ID EP394294
- DOI 10.14569/IJACSA.2018.090979
- Views 108
- Downloads 0
Design and Performance Analysis of Microstrip Array Antennas with Optimum Parameters for X-band Applications
Md. Tanvir Ishtaique-ul Huque, Md. Kamal Hosain, Md. Shihabul Islam, Md. Al-Amin Chowdhury Abstract - This paper demonstrates simple, low cost and high gain microstrip array antennas with suitable feeding techniqu...
Low-Power Hardware Design of Binary Arithmetic Encoder in H.264
Context-Based Adaptive Binary Arithmetic Coding (CABAC) is a well-known bottleneck in H.264/AVC, owing to the highly serialized calculation and high data dependency of the binary arithmetic encoder. This work presents a...
Quality of Service and Power Consumption Optimization on the IEEE 802.15.4 Pulse Sensor Node based on Internet of Things
The Purpose of this research is to determine the Quality of Service (QoS) Zigbee or IEEE 802.15.4 sensor Node use the indicators, i.e. the Receiver Signal Strength and PathLoss (attenuation (-dB)) at the time of communic...
Developing a Dengue Forecasting Model: A Case Study in Iligan City
Dengue is a viral mosquito-borne infection that is endemic and has become a major public health concern in the Philippines. Cases of dengue in the country have been recorded to be increasing, however, it is reported that...
A Brief Survey on 5G Wireless Mobile Network
The new upcoming technology of the fifth generation wireless mobile network is advertised as lightning speed internet, everywhere, for everything, for everyone in the nearest future. There are a lot of efforts and resear...