A General Evaluation Framework for Text Based Conversational Agent

Abstract

This paper details the development of a new evaluation framework for a text based Conversational Agent (CA). A CA is an intelligent system that handle spoken or/and text based conversations between machine and human. Generally, the lack of evaluation frameworks for CAs effects its development. The idea behind any system’s evaluation is to make sure about the system’s functionalities and to continue development on it. A specific CA has been chosen to test the proposed framework on it; namely ArabChat. The ArabChat is a rule based CA and uses pattern matching technique to handle user’s Arabic text based conversations. The proposed and developed evaluation framework in this paper is natural language independent. The proposed framework is based on the exchange of specific information between ArabChat and user called “Information Requirements”. This information are tagged for each rule in the applied domain and should be exist in a user’s utterance (conversation). A real experiment has been done in Applied Science University in Jordan as an information point advisor for their native Arabic students to evaluate the ArabChat and then evaluating the proposed evaluation framework.

Authors and Affiliations

Mohammad Hijjawi, Zuhair Bandar, Keeley Crockett

Keywords

Related Articles

Design of a Cloud Learning System Based on Multi-Agents Approach

Cloud Computing can provide many benefits for university. It is a new paradigm of IT, which provides all resources such as software (SaaS), platform (PaaS) and infrastructure (IaaS) as a service over the Internet. In clo...

Teaching Software Testing using Data Structures

Software testing is typically a rushed and neglected activity that is done at the final stages of software development. In particular, most students tend to test their programs manually and very seldom perform adequate t...

WordNet based Implicit Aspect Sentiment Analysis for Crime Identification from Twitter

Crime analysis has become an interesting field that deals with serious public safety issues recognized around the world. Today, investigating Twitter Sentiment Analysis (SA) is a continuing concern within this field. Asp...

 Defect Diagnosis in Rotors Systems by Vibrations Data Collectors Using Trending Software

 Vibration measurements have been used to reliably diagnose performance problems in machinery and related mechanical products. A vibration data collector can be used effectively to measure and analyze the machinery...

A Type-2 Fuzzy in Image Extraction for DICOM Image

Eradication of a desired portion of an image is a very important role in image processing and is also called feature extraction. This is mainly concern about reducing the number of possessions required to portray a large...

Download PDF file
  • EP ID EP106887
  • DOI 10.14569/IJACSA.2016.070304
  • Views 112
  • Downloads 0

How To Cite

Mohammad Hijjawi, Zuhair Bandar, Keeley Crockett (2016). A General Evaluation Framework for Text Based Conversational Agent. International Journal of Advanced Computer Science & Applications, 7(3), 23-33. https://europub.co.uk/articles/-A-106887