Development of the algorithm of keyword search in the Kazakh language text corpus
Journal Title: Восточно-Европейский журнал передовых технологий - Year 2019, Vol 5, Issue 2
Abstract
<p>The issue of semantic text analysis occupies a special place in computational linguistics. Researchers in this field have an increased interest in developing an algorithm that will improve the quality of text corpus processing and probabilistic determination of text content. The results of the study on the application of methods, approaches, algorithms for semantic text analysis in computational linguistics in international and Kazakhstan science led to the development of an algorithm of keyword search in a Kazakh text. The first step of the algorithm was to compile a reference dictionary of keywords for the Kazakh language text corpus. The solution to this problem was to apply the Porter (stemmer) algorithm for the Kazakh language text corpus. The implementation of the stemmer allowed highlighting unique word stems and getting a reference dictionary, which was subsequently indexed. The next step is to collect learning data from the text corpus. To calculate the degree of semantic proximity between words, each word is assigned a vector of the corresponding word forms of the reference dictionary, which results in a pair of a keyword and a vector. And the last step of the algorithm is neural network learning. During learning, the error backpropagation method is used, which allows a semantic analysis of the text corpus and obtaining a probabilistic number of words close to the expected number of keywords. This process automates the processing of text material by creating digital learning models of keywords. The algorithm is used to develop a neurocomputer system that will automatically check the text works of online learners. The uniqueness of the keyword search algorithm is the use of neural network learning for texts in the Kazakh language. In Kazakhstan, scientists in the field of computational linguistics conducted a number of studies based on morphological analysis, lemmatization and other approaches and implemented linguistic tools (mainly translation dictionaries). The scope of neural network learning for parsing of the Kazakh language remains an open issue in the Kazakhstan science.</p>The developed algorithm involves solving one of the problems of effective semantic analysis of the text in the Kazakh language
Authors and Affiliations
Akerke Akanova, Nazira Ospanova, Yevgeniya Kukharenko, Gulmira Abildinova
Keywords
Related Articles
- EP ID EP667099
- DOI 10.15587/1729-4061.2019.179036
- Views 42
- Downloads 0
Development of complex-structure ore deposits by means of chamber systems under conditions of the Kryvyi Rih iron ore field
<p>In order to keep their positions in the world markets, mining enterprises of the Kryvyi Rih iron ore field using the deep-mine method need to develop a resource-saving technology for the development of the fields repr...
Construction of mathematical models for the estimation of signal strength at the input to the 802.11 standard receiver in a 5 GHz band
<p>The paper proposes mathematical models for the spatial estimation of signal strength at the input of the receiver for the 802.11x family of standards in a 5 GHz range. The models were constructed based on the experime...
Modeling of the functioning of territorial systems with the purpose of identification of problem situations
<p>The purposeful territorial system (country, region), containing production, consumption, management, and environmental spheres is considered. In the process of system functioning, problem situations arise. The method...
Research into energy characteristics of single-phase active four-quadrant rectifiers with the improved hysteresis modulation
<p>The traction electric drives for electric rolling stock of alternating current employ the diode and thyristor rectifiers that predetermine a series of shortcomings. These include the significant emission of reactive p...
Technological features in obtaining highly effective hydrogel dressings for medical purposes
The new technology of formation of hydrogel dressings for medical purposes, containing particles of silver was developed. To obtain metal-containing composite hydrogels, we proposed a high-tech single-stage method, which...