Do you want to build a chatbot? Then, you are not alone! Facebook Messenger alone has more than 34,000 chatbots, which are increasingly helping clients answer questions and solve problems on their own, taking several manual work off the shoulders of agents in the contact centers.
Now, you might ask yourself: Where to get started? This article will guide you through the vital steps needed to build a chatbot from scratch, demonstrating how to find the right use case, design a bot’s ability to understand and respond to messages, pick the right testing tools and approaches, and deploy. It is, however, not a comprehensive guide, but will surely shed some light on how to get started with chatbot development easily and quickly.
1. Determine the role and goals of your bot
Identifying what you want the bot to do before you start anything is basic. You need to identify all the features required to address the questions the bot is supposed to answer to. Doing this upfront will result in a robust chatbot and ideally one that can handle most of the topics that your contact center is dealing with today, including strategies on how to handle unknown questions.
To start with, consider a modest goal that fits your philosophy of customer service. Is your initial plan to provide excellent responses to a limited set of questions with high confidence? Or to handle a broader range of topics to meet customer needs? Both goals can be achieved eventually, but there will be learning and refining that will happen once your customers have a chatbot available to use.
However, it is good to start with a simple bot whose purpose is exclusively to route questions to the appropriate live personnel. Identify the top 5 to 10 items customers are interested in. Limiting the topics provides a better understanding of the subject for fine-tuning of the bot, which in turn enables responses with greater confidence.
Build a set of Frequently Asked Questions (FAQs). They usually are 10-25 topics and answers with short text responses along with follow-up links. Since the chat needs to be conversational, make sure that responses are short and simple. Match the chatbot to the capabilities of your current voice self-service platform. Recreating topics will create consistency for your customers across all channels.
Identify all uses cases — both transactional as well as mere informational. This would create a fully functional bot that can handle all types of business tasks that your employees are currently handling, such as providing detailed order status or handling appointment reminders.
2. Evaluate and pick your channel
Whether it’s a traditional mobile channel (SMS, USSD), messaging app (Facebook Messenger, WeChat, Kik, Line, Viber), social media channels, or chat embedded on websites, text-based chatbots can be very effective on all communication channels. Whichever channel you choose, make sure that it offers an open API which the bots can embed programmatically. The channel you choose may be one on which you already engage with your customers, or it could be entirely new. The latter has many advantages as it won’t conflict or compete with any existing programs in your organization, and you can start fresh. In fact, the implementation on a new channel, as opposed to the one that you already use, is easier to launch.
3. Create a conversational architecture
Chatbots, particularly those which answer frequently asked customer questions, require a different type of user interface than a traditional graphical user interface (GUI) like a website or a mobile application. GUIs limit the possible interaction between users to a finite set of options. Conversational UIs (CUIs) are open-ended by design. The customer can send any text, and allow infinite ways of saying the same in different words. This is similar to a GUI Information Architecture, which puts the content of the website into a hierarchy of web pages. If the foundation for your bot is an existing set of FAQs and answers typically posted on your website, you also need to consider which follow-up questions a user might pose on a specific bot response.
4. Design the flows and storyboards dialog
While the objective of building a Conversational Architecture is to organize your content and decide the best ways to determine the bot’s responses, the storyboards and the flow of dialog go into all the details needed to implement the bot.
You need to create the representations of what the bot will say at each step in the dialog flow. However, the design of the messages should be outside the flow diagrams, as you want to design variations of the same message for frequently occurring steps of the dialog. This technique called random prompting allows the bot to use variations in wording to say the same thing, making the whole experience less robotic and more human-like.
5. Design your integrations
Integrations can be as diverse as those applications that you want to automate. Are there already existing integrations or data that can be used with your bot to support the customer experience? If you already integrate some of your systems with your existing self-service platforms (web or voice), the same integrations may be used again for chatbot purposes. For example, if your IVR application already checks your customer and can provide information about the order status, then the integration for your chatbot can most likely be reused.
However, the new concept of self-service bot may require new integrations. Usually, current software packages, technologies, and databases have robust connections that can provide a wealth of details needed for the bot to work.
It is vital to access customer data to answer customer questions, but not all chatbot functions need integration. For example, when FAQ responses are well-defined and documented, bot responses can be re-used to provide consistency of messages. To get started, this example does not require any back-end integration into other systems. Down the road, you may want to centrally manage FAQ responses better, which will need access to a Knowledge Base or similar systems.
6. Collect your chat data
Regardless of the platform you use to build your bot, the most important resource you will need is a collection of sentences that reflect different ways of expressing each of your bot’s intentions and slot values. Because of the incredible diversity of human language and the limitations of the developer’s imagination, this resource can be challenging to create. Real text from conversations with customers is the ideal resource. If you’re lucky enough to have agents in chat channels communicating with customers, you can mine those conversations for real examples. But for many businesses, this may be the first time they use a chat-based interface to engage in customer service.
If you need to create your initial data manually, make sure your collection includes not only intuitions from one developer but also input from a variety of people as large. Remember that the key is diversity. Therefore, working with QA and crowdsourcing companies that have access to large numbers of people from all over the world is a useful approach. It is helpful to find people with the same language background as your target users.
7. Choose a platform and an approach to development
Primarily, most chatbots consider their key tasks to be: (1) determining the intent of the sentence (what is the customer requesting or what use case does it wish to initiate?) and (2) extracting data from the sentence (what options did the customer request and what data does it provide to you?). Mostly, there are two different approaches to these tasks: one based on explicitly creating rules from the top down, and the other using machine learning algorithms to learn the task from a large corpus of transcribed interactions (a collection of written texts).
If you’ve been offering web chat for quite a while, you might have already gathered tens of thousands of customer inquiries with corresponding responses. Interestingly, most customer service dialogs are not simple question & answer pairs. The data set should be capable of applying machine learning algorithms to learn the most common answers to the most common questions. Note that with this approach, every new language you want the bot to speak will need to start from scratch. Also, tagging the data and analyzing the result to ensure quality is still a tremendous and mostly manual effort.
If you don’t already have such a corpus at your disposal, or you don’t have what you need to train an algorithm, you’ll have better luck with an approach based on writing distinct rules to extract meaning from messages. The easiest way to do that is to find and act upon specific keywords in the customer’s message. However, this carries a risk in that if keywords do not necessarily appear in the expected form, many messages could be misclassified.
Consider the difference between these questions: “Can I book a flight for tomorrow?” “Can I get a book to read on my flight tomorrow?” The questions are related to two separate things. But both contain the same keywords, such as “book,” “flight,” and “tomorrow.” A simplistic rule might end up believing that both questions are about booking a flight for tomorrow. A better platform will extend its reach of a rule-based approach with built-in linguistic tools that can leverage the relationships between many different words (synonyms, hypernyms/hyponyms, domains) and common syntactic patterns. This makes distinguishing the senses of a word like “book” easier. An advantage of the top-down approach to linguistics is that you have total control over the way a message is understood.
8. Implement the NLU flow and dialog
In this step, everything comes together: the conversational architecture, the flow, and storyboard dialogue, the platform you selected, and the data you collected. Your key task is to create a classifier that will map an incoming text to the response from the system.
If you’ve selected a machine learning platform, you’ll provide your example sentences to this platform for every possible purpose. The more examples and sentences you provide, the better the algorithm learns the variations of the expressions that can be used for each purpose, and the better it learns how to distinguish between intents. Note that some of your example sentences are to be reserved for the next step (testing). If you work with a platform based on linguistic rules, you will use the phrases differently. The rules you create will explicitly represent the characteristics determining which sentence belongs to intent A or intent B. In either situation; this is when having as close as possible to real user utterances as a diverse set of examples is of central importance. A small set of sources won’t start capturing the universe of differences that will display your real users when they express themselves to your system.
9. Test and revise your use case
Now, you are ready to go for automated testing. It is also better to have as many human testers for “real user” testing as possible. Test and revise your NLU component, and the bot flow until an acceptable level of accuracy is reached. Note that this step and the preceding step are iterative and approximate; the goal of 100 percent accuracy is unattainable because of the nature of the human language and the infinite possible expressions of every intention. However, you get closer each time you iterate through these steps.
In addition, as mentioned above, make sure that you are as close to your real end-users as possible. Any tester who is not a real user of your system with an honest need to chat with your bot will yield results that are of a slightly artificial nature. Consider a smaller rollout with a few target clients to vet some of the design decisions that you made along the way.
10. Deployment and revisions
Even if you’re ready to go live when the bot gets deployed, the work isn’t done. Even if your bot uses unsupervised or semi-supervised machine learning to adjust its behavior over time, monitoring its first real-time interactions will provide very useful information and may signal if necessary adjustments are needed.
You may also need to adjust the logic of your classification of intent, either by explicit manipulation of the rules or by providing more sentences of examples. Finally, if the designed use cases do not cover most user requests, you might need to add new use cases. If you start small as recommended, then this is the time you gather vital information about which use cases are the key to covering.
To ensure that your chatbot deployment is successful, view the creation as an iterative process: collect the data, review it, and apply it to your bot’s design. Log everything in for the future, above all. The lessons you will learn from this one could drive the success of other projects.
