Classifying and Segmenting Classical and Modern Standard Arabic using Minimum Cross-Entropy
Journal Title: International Journal of Advanced Computer Science & Applications - Year 2017, Vol 8, Issue 4
Abstract
Text classification is the process of assigning a text or a document to various predefined classes or categories to reflect their contents. With the rapid growth of Arabic text on the Web, studies that address the problems of classification and segmentation of the Arabic language are limited compared to other languages, most of which implement word-based and feature extraction algorithms. This paper adopts a PPM character-based compression scheme to classify and segment Classical Arabic (CA) and Modern Standard Arabic (MSA) texts. An initial experiment using the PPM classification method on samples of text resulted in an accuracy of 95.5%, an average precision of 0.958, an average recall of 0.955 and an average F-measure of 0.954, using the concept of minimum cross-entropy. PPM-based classification experiments on standard Arabic corpora showed that they contained different types of text (CA or MSA), or a mixture of the both (CA and MSA). Further experiments with the same corpora showed that a more accurate picture of the contents of the corpora was possible using the PPM-based segmentation method. Tag-based compression experiments (using tags produced by parts-of-speech Arabic taggers) also showed that the quality of the tagging (as measured by compression quality) is significantly affected when tagging either CA and MSA text. The conclusion is that NLP applications (such as taggers) should treat these texts separately and use different training data for each or process them differently.
Authors and Affiliations
Ibrahim S Alkhazi, William J. Teahan
Keywords
Related Articles
- EP ID EP258381
- DOI 10.14569/IJACSA.2017.080456
- Views 116
- Downloads 0
Laguerre Kernels –Based SVM for Image Classification
Support vector machines (SVMs) have been promising methods for classification and regression analysis because of their solid mathematical foundations which convey several salient properties that other methods hardly prov...
A Comparative study of Arabic handwritten characters invariant feature
this paper is practically interested in the unchangeable feature of Arabic handwritten character. It presents results of comparative study achieved on certain features extraction techniques of handwritten cha...
MDSA: Modified Distributed Storage Algorithm for Wireless Sensor Networks
In this paper, we propose a Modified distributed storage algorithm for wireless sensor networks (MDSA). Wireless Sensor Networks, as it is well known, suffer of power limitation, small memory capacity,and limited process...
Measuring the Data Openness for the Open Data in Saudi Arabia e-Government – A Case Study
Conceptually, data can be found at the lowest level of abstraction from where information and knowledge are being extracted. Furthermore, data itself has no meaning, unless it’s being interpreted and transferred into inf...
Permutation of Web Search Query Types for User Intent Privacy
Privacy remains a major concern when using search engines to find for information on the web due to the fact that search engines own massive resources in preserving search logs of each user and organizations. However, ma...