Classifying and Segmenting Classical and Modern Standard Arabic using Minimum Cross-Entropy
Journal Title: International Journal of Advanced Computer Science & Applications - Year 2017, Vol 8, Issue 4
Abstract
Text classification is the process of assigning a text or a document to various predefined classes or categories to reflect their contents. With the rapid growth of Arabic text on the Web, studies that address the problems of classification and segmentation of the Arabic language are limited compared to other languages, most of which implement word-based and feature extraction algorithms. This paper adopts a PPM character-based compression scheme to classify and segment Classical Arabic (CA) and Modern Standard Arabic (MSA) texts. An initial experiment using the PPM classification method on samples of text resulted in an accuracy of 95.5%, an average precision of 0.958, an average recall of 0.955 and an average F-measure of 0.954, using the concept of minimum cross-entropy. PPM-based classification experiments on standard Arabic corpora showed that they contained different types of text (CA or MSA), or a mixture of the both (CA and MSA). Further experiments with the same corpora showed that a more accurate picture of the contents of the corpora was possible using the PPM-based segmentation method. Tag-based compression experiments (using tags produced by parts-of-speech Arabic taggers) also showed that the quality of the tagging (as measured by compression quality) is significantly affected when tagging either CA and MSA text. The conclusion is that NLP applications (such as taggers) should treat these texts separately and use different training data for each or process them differently.
Authors and Affiliations
Ibrahim S Alkhazi, William J. Teahan
Keywords
Related Articles
- EP ID EP258381
- DOI 10.14569/IJACSA.2017.080456
- Views 96
- Downloads 0
Towards Network-Aware Composition of Big Data Services in the Cloud
Several Big data services have been developed on the cloud to meet increasingly complex needs of users. Most times a single Big data service may not be capable in satisfying user requests. As a result, it has become nece...
Visualising Arabic Sentiments and Association Rules in Financial Text
Text mining methods involve various techniques, such as text categorization, summarisation, information retrieval, document clustering, topic detection, and concept extraction. In addition, because of the difficulties in...
Effects of Modulation Index on Harmonics of SP-PWM Inverter Supplying Universal Motor
This manuscript presents the effects of changing modulation indices on current and voltage harmonics of universal motor when it is supplied by single phase PWM (SP-PWM) inverter, the effect has been analyzed with simulat...
Ranking Attribution: A Novel Method for Stylometric Authorship Identification
Stylometric Authorship attribution is one of the essential approaches in the text mining. The present research endorses a Stylometric method called Stylometric Authorship Ranking Attribution (SARA) overcomes the usual pr...
Junction Point Detection and Identification of Broken Character in Touching Arabic Handwritten Text using Overlapping Set Theory
Touching characters are formed when two or more characters share the same space with each other. Therefore, segmentation of these touching character is very challenging research topic especially for handwritten Arabic de...