Welcome to our blog series about chatbot methodology! Today, let’s talk about use cases for your chatbot, the second step of a successful chatbot project with the SAP Conversational AI bot-building platform.
In the bot design process, the Use Case design stage focuses on building the conversational workflows that will perform actions for the user. This helps in automating all the use cases and results in the bot responding relevantly to the users.
This phase comes right after building the Horizontal Coverage since the latter helps you identify the topics your users trigger most often. For more information about this concept, please read this great article.
Horizontal Coverage gives you data-driven insights into what should be automated first. As such, it’s a prerequisite to the Use Case design phase.
In the context of a chatbot, a Use Case (UC) is an action that can be modeled, handled, and accomplished through a conversational interaction between the bot and the user.
Creating a leave request in an HR system is a typical example of a Use Case.
When designing a Use Case, it is important to have all the actors involved so as to identify business aspects of the flow, exceptions, and technical limitations. In this phase, the whole project team is involved: the project manager, business/product representatives, and developers.
Designing the first Use Case might take up to one day but going forward the project team will gain expertise and it should take no more than half a day per use case.
Before designing our first use case, let’s define some key aspects: what’s our expected return on investment (also called ROI), what’s our target persona, and what’s the personality of our chatbot.
Why are we building this chatbot? Based on identified business needs in the context of a specific company or a market, we will specify metrics to track the return. For example:
* In the customer service industry, it can be: “The average conversation duration is reduced by 30% because customers have now access to the relevant information through the chatbot”.
* In the employee experience context, it can be: “Users are struggling to find relevant information and perform specific tasks in their HR software. Improving the software UX with a conversational interface will save 2 hours of work per month for a manager, compared to 20 minutes for an average employee. To have a solid business case, it, therefore, makes sense to start by automating the navigation of the manager with the higher ROI.”
You should be able to describe your ROI in one sentence. It is instrumental to have clear KPIs to constantly monitor the relevance and the performance of the chatbot from an economic perspective.
The ROI calculation will help us to define our target audience.
* Who are they?
* What do they do (i.e. what are their everyday tasks and responsibilities)?
* What vocabulary do they have?
* What are the requests that come most often from them?
* What are their pain points?
By having a clear view of who your users will be, you will be able to build a chatbot that addresses their needs effectively.
It is important to keep in mind that we are building a chatbot for real users trying to accomplish concrete tasks. Depending on the user group that is targeted, the chatbot would have a very specific persona. For example, building a chatbot to help sales representatives in their daily tasks (e.g. updating meeting details, creating new opportunities in the CRM…) is not the same user persona than building a chatbot to help finance teams managing their cost centers.
Once we have determined the ROI of our chatbot and the user persona, we need to decide how the bot will talk to them. In other words, its personality! Think about Siri or Alexa, they have their own personality.
* “Who” is the chatbot? Its name and character (neutral, formal/informal).
* What is its avatar?
* What vocabulary does it use? How does it say “Hello” or “Goodbye”, or how does it confirm that it has done something?
* What attitudes or behaviors would the user
* What is my goal? What am I trying to achieve?
Now we are ready to design use cases for our chatbot!
Here, we need to take all the use cases that you’ve collected from the Horizontal Coverage phase.
As we said before, the Horizontal Coverage exercise helped you identify the topics that your users trigger most often, giving you data-driven insights into the topics you should automate first. If 50% of user questions revolve around creating a leave request, automating this process to simplify the user experience would seem to be a valuable business decision.
So how do you choose to use cases that need to be automated?
Start from the most demanded use cases from your Horizontal Coverage. These are the most important ones according to your users. If there are only a few top ones, automate them all.
From this list, let’s filter out complex use cases. Keep in mind that a good conversation between a user and a chatbot is not more than 2-3 interactions. If the use case leads to too many interactions, take them out of scope.
Then think if use cases might need connections to backend systems that you can’t test right now. Take them out of the list as well.
The objective is to cover 30% of the use cases that can be automated while rolling in production with an initial version of the bot. Then you’ll gradually add more automated use cases for the future versions. In next to no time, most common scenarios will be automated – with no interruption to your business.
In this phase, we need to focus on the nominal flow. In other words, a straightforward scenario where everything goes according to plan. The user asks all the right questions, and the bot understands and provides them with exactly the info they asked for. There are some questions that you need to keep in mind:
* How to achieve the goal of the bot via a conversation? (laying out a basic start-to-finish interaction between the user and the bot)
* What are the questions that the bot needs answers from the user to accomplish the task?
* What information should the bot share with the user for the conversation to be successful?
* What is the mandatory information that the user must provide to move forward in the conversation?
* When will the bot get information from external sources (API, DB, etc.)?
The bot also needs to comply with the standard conversation rules: respond to a greeting with a greeting, respond to a question with an answer.
In this step, we also address technical requirements such as what APIs will be used, security, and channels to be used with the developers.
Once the nominal flow is drawn and validated by the project team, it is time to refine the dialogue and take any of the inevitable conversational exceptions into account. Here’s an example of an exception in the context of a use case helping a client understand their invoice:
* Customer: “I do not understand last month’s invoice, can you help?”
* Bot: “Yes, I can. Please provide me with your customer reference number”
* Customer: “Where can I find it?” > This question is outside of the nominal flow and needs to be dealt with in order for the bot to accomplish its task and provide a good user experience. This is an exception.
Once we have outlined a sample conversation between a user and a bot, we need to test it to make sure it flows naturally and gets the intended results. That is what UATs are for. The objective is to build a dozen (or more) conversations, most of which with exceptions to validate the developments.
These UATs will be enriched as we go taking into account new exceptions and new use cases. UATs are used to validate each delivery.
Once deliveries are completed, let’s go to the “Test & Train” stage. Here the goal is twofold:
* Test sessions are used to find loopholes and dysfunctions in the use cases defined.
Examples: refine the flow with new exceptions in the flow and fix bugs
* Train sessions are used to enrich the training dataset to make its horizontal coverage solid.
Examples: insert new expressions, get rid of confusions and merge or create intents
This is usually done under the form of test sessions that we run with the project team, business representatives, and even better, with future end-users.
At the end of this stage, we must have a bot with 85% accuracy and be confident enough to move to the next phase: Alpha and Beta testing with customers.
At this point of the project we have:
* At least 30% of the use cases automated and tested
* A bot with 85% accuracy and ready for customer testing
Your bot is ready for real-life testing, congrats! In our next blog, we’ll deep dive into the Alpha Testing, the third phase of a successful chatbot project with SAP Conversational AI.
If you have any questions about our chatbot methodology, please go to SAP Answers, our team will be available to answer.
For more information about SAP Conversational AI:
This Post was originally published on blogs.sap.com