CHATBOT: An Overview of Conversational AI

10 min readDec 8, 2020

A comprehensive overview of Chatbot Technology

Sustaining a good customer service process is a vital ingredient to the success of any business. The way this process looks, however, is dramatically changing. It’s hard to imagine a world without Chatbots since it has become part of ordinary transactions today.

Chatbots have been on the rise for a couple of years and have been widely adopted. It’s bringing a new way for businesses to communicate with the world through the accelerated development of all kinds of sensors and wearables, and of course with the rise of emerging technologies and Artificial Intelligence (AI).

A Chatbot is a computer program that interacts with users using natural language, text, or audio in a way that the user thinks he is having a dialogue with a human. It is designed to convincingly simulate the way a human would behave as a conversational partner.

Most of the Chatbots utilize the algorithms of AI in order to generate the required response. Earlier Chatbots merely created an illusion of intelligence by employing much simpler pattern matching and string processing design techniques for their interaction with users using rule-based and generative-based models. However, with the emergence of new technologies more intelligent systems have emerged using complex knowledge-based models.

Chatbots are used in dialog systems for various purposes including customer service, request routing, or information gathering. While some chatbot applications use extensive word-classification processes, natural language processors, and sophisticated AI, others simply scan for general keywords and generate responses using common phrases obtained from an associated library or database.

The Rise of ChatBots

Chatbots was first contemplated by Alan Turing, a British computer scientist, through his famous article, “Computer Machinery and Intelligence”, published in 1950. In this article, he proposed the Imitation Game (“Can machines think?”), which is now called the Turing Test -a criterion test of intelligence, This criterion depends on the ability of a computer program to imitate a human in a real-time conversation with a human judge to the extent that the judge is unable to distinguish between the program and a real human, based on the conversational content only.

The Turing Test stimulated great interest in many computer scientists, among them was Joseph Weizenbaum, who developed the first known Chatbot, ELIZA, in 1966, whose purpose was to act as a psychotherapist returning the user utterances in a question form. It used simple pattern matching and a template-based response mechanism.

Weizenbaum himself did not claim that ELIZA was genuinely intelligent, but it was enough at a time to make people believe they were conversing with a real human. and give them the stimulus to start developing other Chatbots.

ELIZA set out the foundation for the structure of chatbots used today such as pre-programmed responses, keywords, and specific phrases.

In 1972, an improvement over ELIZA was a chatbot with attitude, named PARRY, developed by a Stanford scientist, Keneth Colby, which tried to model the behavior of a paranoid schizophrenic.

Another Chatbot ALICE, a language processing bot developed in 1995, gained popularity, it won the Loebner Prize, an annual Turing Test, in 2000, 2001, and 2004. It was the first computer to gain the rank of the “most human computer”. ALICE relies on a simple pattern-matching algorithm with the underlying intelligence based on the Artificial Intelligence Markup Language (AIML), which makes it possible for developers to define the building blocks of the chatbot knowledge.

Chatbots, like Smarterchild, were developed and became available through messenger applications and across SMS networks. Which are considered as precursors of Siri and S Voice.

The next step was the creation of personal digital assistants, like Apple Siri, Google Assistant, and IBM Watson, Microsoft Cortana, Samsung S Voice, Amazon Alexa, which are the forefront of technology of voice recognition and artificial intelligence(AI).

ChatBot Technology

Below are some essential concepts related to Chatbot technology.

Pattern Matching

Pattern Matching is predicated on representative pattern-template blocks. It requires a lot of pre-generated patterns. Based on these pre-generated patterns the chatbot can easily pick the pattern which best matches the user input and provide an output for it.

Rule-based, scripted, and structured Chatbots mostly fall under this category. Such chatbots use a knowledge base that contains documents and each document comprises a particular <pattern> and <template>.

Eliza and ALICE were developed using pattern recognition algorithms. When the bot receives an input that matches the <pattern>, it sends the message stored in the <template> as a response. The <pattern> can either be a phrase like “What’s your name?” or a pattern “My name is *”, where the ‘*’ is a regular expression. Typically, these <pattern> <template> pairs are manually inserted.

The disadvantage of this approach is that the responses are entirely predictable, repetitive, and lack the human touch. Also, there is no storage of past responses, which can lead to looping conversations.

Artificial Intelligence Markup Language (AIML)

A standard structure of Pattern Matching is “AIML”. It was created from 1995 to 2000, and it is based on the concepts of the Pattern Matching technique. The AIML language´s purpose is to make the task of dialog modeling easy, according to the pattern-template approach.

AIML is a just simple XML or similar to HTML, in that it consists of standard and extensible tags that you use to mark up text so that it can be understood by an AIML interpreter. Tags are identifiers that are responsible to make code snippets and insert commands in the chatbot.

For an AIML object/tag to be well defined, it must follow the XML standards. For example, object names cannot start with numbers, they are case-sensitive (there is a distinction between uppercase and lowercase letters) and blanks are not allowed.

Latent Semantic Analysis(LSA)

Latent Semantic Analysis is a classical tool for automatically extracting similarities between documents, it is used to discover likenesses between words as vector representation, through dimensionality reduction.

It is worth mentioning that AIML, and other pattern-matching languages, sometimes are used together with Latent Semantic Analysis (LSA) or other techniques. For example, AIML may answer questions based on specific templates, while questions that cannot be answered in this way can use LSA for the production of responses.

ChatScript

ChatScript is a chatbot engine that manages dialog or NL tools. It is rule-based, where rules are created by users in program scripts through a process called dialog flow scripting. It uses a scripting metalanguage as its source code.

ChatScript is an expert system, consisting of open-source technology, comes with unique technical merits, as well as major business advantages, that makes it apt for natural language processing.

It can support multiple chatbots at the same time, either all acting independently or all coordinating in something like a theatrical production where multiple bots chat with each other and with the user. If you asked one chatbot a question about his family you’d get one answer, and if you asked a different one the same question you’d get a different answer. And if you asked one a fact question about some topic the other knows, usually the other would answer it instead. ChatScript has won the Loebner’s prize 4 times.

Natural Language Processing(NLP)

NLP is a branch of informatics, mathematical linguistics, machine learning, and artificial intelligence that helps computers read and understand natural human language. It facilitates human-to-machine communication without humans needing to “speak” Java or any other programming language as it allows machines to obtain and process information from written or verbal user inputs.

NLP helps chatbot to analyze the human language and generate the text. It aims to extract context and meanings from natural language user inputs, which may be unstructured and respond appropriately according to user intention. It identifies user intent and extracts domain-specific entities. Its main goal is to improve human-machine communication.

Why is ChatBot a BigDeal?

Advancements in artificial intelligence, in addition to the proliferation of messaging apps, are fueling the development of chatbots — software programs that use messaging as the interface through which to carry out different tasks, from reporting weather to scheduling a meeting, helping users buy a pair of shoes.

Chatbots are capable of answering various user queries depending on the functionality. They can also guide users to attain the appropriate information anytime they require. AI-based bots can detect the user’s intents from the input query, and evaluate and respond accordingly.

Here are some benefits that Chatbot provides:

24/7 Automated Customer Support

Nobody likes to wait and so do customers. Remember those times where you dialed the customer service agent and had to listen to the boring music playing while you’re kept on hold for what seems forever? Fortunately, with Chatbots, live chats and other forms of contact such as emails and phone calls are being replaced. Gone are the days of waiting for the next available operative.

Customer support processes have been improved with the help of chatbots. Bots can be programmed to give automated answers to repetitive questions immediately and forward the request to a real person when a more complicated action is needed. This enables human customer service representatives to save time and assist more important cases rather than time-consuming simple tasks.

Monitoring Customer Data and Generating Insights

We live in a data-driven world. The best companies are gathering data from as many sources as possible, trying to uncover key insights.

Simple surveys regarding the business can be done on Chatbots to consolidate feedback. From this feedback, you’d be more informed on the improvements that can be made to your products/ services.

Chatbots can also be used to track purchasing patterns and consumer behaviors by monitoring user data. According to Forbes, this helps a company to decide “which products to market differently, which to market more and which to redevelop for relaunch”.

Better Lead Generation and Nurturing

Most of the businesses operate in an omnichannel model. They sell across the website, Whatsapp, and other channels. AI chatbots create an effortless way for customers to communicate with businesses via existing messaging platforms.

By deploying chatbots, personalized messaging can be engaged with customers by guiding and recommending them in making quick decisions.

Personalized messaging assists buyers along their consumer journey with the consumer information that chatbots receive. A chatbot can ask the necessary and related questions, persuade the user, and generate a lead for you. It ensures that the conversation flows in the right direction to convince consumers to make the purchase, thereby driving conversion rates.

Cost Saving

Chatbots are a one-time investment, which helps businesses to optimize customer service costs. Investing in chatbots saves the extra costs of adding more agents. It helps businesses reduce the number of staff required, thereby cutting the labor costs incurred.

Customer support chatbot can be designed to cater, and answer simple queries by consumers. Since chatbots are automated solutions, they allow organizations to handle many customers at once, and simultaneously. By “employing” chatbots that complement human agents, you will not only save on employee costs but you will also avoid the problems caused by human errors.

Reduction In Human Error

It’s a universal fact that human errors are inevitable. Manual work has a risk of errors that can eventually create big problems. Mistakes might happen while collecting customer information or sharing product pricing information.

With customer retention at the core of market dominance, businesses are seeking innovative ways to consistently deliver exceptional experiences. One way to achieve this is to reduce human error in handling and provide a near real-time service to customers.

Chatbots are the best way to deliver error-free service and avoid complications. As chatbots are programmed with all the information there is no chance of errors. Accuracy is guaranteed with chatbots.

Limitations

Yes, Chatbots are extremely amazing and can substitute humans in various fields, but there are some aspects where they fail to deliver what was expected! The creation and implementation of Chatbots is still a developing area, heavily related to AI and machine learning, so the provided solutions, while possessing obvious advantages, have some important limitations in terms of functionalities and use cases. However, this is changing over time.

Here are the most common ones.

As the database, used for output generation, is fixed and limited, chatbots can fail while dealing with an unsaved query.
Chatbots lack human context, they are unable to deal with multiple questions at the same time and so conversation opportunities are limited.
Poor conversational understanding, Chatbots have difficulty managing non-linear conversations that must go back and forth on a topic with a user.
Chatbots require a large amount of conversational data to train.
Users can easily detect the presence of robotic answers.

Conclusion

Chatbots are everywhere. More and more businesses are adopting digital transformation to modernize customer communication and improve internal processes, from online assistance to helper’s bot to home applications like Amazon.com’s Alexa. Chatbots have become one of the most visible consumer-facing applications of artificial intelligence and machine learning.

Connect with me on LinkedIn: https://www.linkedin.com/in/grace-kolawole/ and Twitter: https://twitter.com/Graceblarc_