For a long time now, Azure QnA Maker has been a staple of any Microsoft Bot Framework integration. At it’s simplest, QnA Maker is an extremely easy to use key/value pair knowledgebase. Where an incoming chat is best matched with a question inside QnA and that answer returned. Unfortunately, it’s rather basic and for a while has been relegated to only answering questions in a one question to one answer fashion. Essentially, QnA Maker lacked the ability to “follow up” questions to better drill down to an answer.

As an example, imagine the following question and answer.

Question : Where can I park?

Answer : If you are in Seattle, then you have to park around the back of the building using code 1234. If you are on the San Francisco campus, then unfortunately you will have to park on the street. Usually there are parks available on Smith Street. 

While we have answered the user’s question, we had to combine two different answers, one for parking in Seattle, the other for San Francisco. Maybe we add another campus, or we want to elaborate further on a particular location, things can get confusing for the user fast. It would be much better if a user asks where they can park, the first response is asking where they are located.

Thankfully, QnA Maker have recently released “Follow Up Prompts” which allows a bot to have a “Multi-Turn” conversation to better drill down an answer. There are a couple of gotcha’s with the interface at the moment, but for the most part it’s rather simple. Let’s take our example from above and see how it works.

Adding Follow Up Prompts To QnA Maker

The first thing we need to do is head to our KB Editor at https://www.qnamaker.ai/. This interface is generally fine as-is, but this time around we actually want to add one additional column. Select View options and select “Show Context”. This won’t immediately be evident what this does, but is super important as we add Follow Up Prompts.

Next, I’ll add the question “Where can I park?” like so :

Notice how our “answer” is actually the follow up question. Also notice that “Add follow-up prompt”. Clicking it, we need to fill out the the resulting popup like so :

The options are as follows :

Display Text is what our follow up button text will show. In our case, because our drill down question is asking the user which campus they are located at, we want to display a simple option of “Seattle”.

Link to QnA will actually be the initial answer. So we can fill this out as to how it will be answered if a user selects Seattle.

Importantly, we select “Context-Only” as this enforces that the only way someone can reach this, is by following the prompts from parking. Otherwise, a user can simply type “Seattle” even without first asking about parking.

After hitting save, because earlier we turned on the option to “Show context”, we will be shown a tree view of our conversation flow :

Let’s Save and Train, then Test.

Perfect! And if we ask “Seattle” out of the blue, we also see that it doesn’t return our parking answer out of the blue!

We can of course go back and add other options to the original question as often as we want.

Linking Existing QnA

One final thing I want to mention is that if you have QnA options that are somewhat close to each other, and you want to link between them, you can now also use Follow Up Prompts to do this. Most notably, I created a QnA answer to handle bad answers. I then can add it as a follow up question by typing the start of the question “Bad Answer”, and selecting the existing QnA question.

Obviously this is a great way to have a common method for handling bad answers, but you can also use this as a way to show “Related” QnA within the QnA Maker, and not have to handle conversation flow within your bot at all!