Artificial conversations for customer service chatter bots: Architecture, algorithms, and evaluation metrics

• We describe limitations of the current generation of computer-based chatter-bots.
• We demonstrate structures and algorithms producing computer-based conversations.
• Our conversations combine semantic and pragmatic knowledge for customer service.
• Our computer-based dialogs are based on conversation theory and Grice’s maxims.
• We demonstrate methods to evaluate conversations in the customer service domain.

Chatter bots are software programs that engage in artificial conversations through a text-based input medium. They are extensively deployed in customer service applications. Existing approaches to artificial conversation generation emphasize grammatical and linguistic modeling techniques. They focus on generation of discrete sentence-level utterances. These approaches perform poorly in conversational situations requiring contextual continuity over a series of utterances. We present an approach that combines pragmatics with content semantics to generate artificial conversations in the customer service domain. A conversation is a process that adheres to well-defined semantic conventions and is contextually grounded in domain-specific knowledge. We model this using stochastic finite state machines, where the parameters of the model are learned from a corpus of human conversations. We present a specific set of criteria which we then use to evaluate the quality of artificial conversations in the customer service domain. We also compare chatter bot generated artificial conversations with human generated natural conversations in this domain.