A General Evaluation Framework for Text Based Conversational Agent
Journal Title: International Journal of Advanced Computer Science & Applications - Year 2016, Vol 7, Issue 3
Abstract
This paper details the development of a new evaluation framework for a text based Conversational Agent (CA). A CA is an intelligent system that handle spoken or/and text based conversations between machine and human. Generally, the lack of evaluation frameworks for CAs effects its development. The idea behind any system’s evaluation is to make sure about the system’s functionalities and to continue development on it. A specific CA has been chosen to test the proposed framework on it; namely ArabChat. The ArabChat is a rule based CA and uses pattern matching technique to handle user’s Arabic text based conversations. The proposed and developed evaluation framework in this paper is natural language independent. The proposed framework is based on the exchange of specific information between ArabChat and user called “Information Requirements”. This information are tagged for each rule in the applied domain and should be exist in a user’s utterance (conversation). A real experiment has been done in Applied Science University in Jordan as an information point advisor for their native Arabic students to evaluate the ArabChat and then evaluating the proposed evaluation framework.
Authors and Affiliations
Mohammad Hijjawi, Zuhair Bandar, Keeley Crockett
Keywords
Related Articles
- EP ID EP106887
- DOI 10.14569/IJACSA.2016.070304
- Views 114
- Downloads 0
A New Artificial Neural Networks Approach for Diagnosing Diabetes Disease Type II
Diabetes is one of the major health problems as it causes physical disability and even death in people. Therefore, to diagnose this dangerous disease better, methods with minimum error rate must be used. Different models...
Evaluation of Peer Robot Communications using CryptoROS
The demand of cloud robotics makes data encryp-tion essential for peer robot communications. Certain types of data such as odometry, action controller and perception data need to be secured to prevent attacks. However, t...
Optimal Network Design for Consensus Formation: Wisdom of Networked Agents
The wisdom of crowds refers to the phenomenon in which the collective knowledge of a community is greater than the knowledge of any individual. This paper proposes a network design for the fastest and slowest consensus f...
Comparison of Agile Method and Scrum Method with Software Quality Affecting Factors
The software industry used software development lifecycle (SDLC) to design, develop, produce high quality, reliable and cost-effective software products. To develop an application, project team used some methodology whic...
An Optimized Analogy-Based Project Effort Estimation
Despite the predictive performance of Analogy-Based Estimation (ABE) in generating better effort estimates, there is no consensus on: (1) how to predetermine the appropriate number of analogies, (2) which adjustment tech...