14.nlp-with-dispatch

NLP with Dispatch

Bot Framework v4 NLP with Dispatch bot sample

This sample shows how to create a bot that relies on multiple LUIS.ai and QnAMaker.ai models for natural language processing (NLP).

Prerequisites

• [Node.js][4] version 8.5 or higher

  # determine node version
  node --version

• Dispatch CLI

  # install the dispatch CLI tool
  npm install -g botdispatch

Dispatch is a tool to create and evaluate LUIS models used for NLP (Natural Language Processing). Dispatch works across multiple bot modules such as LUIS applications, QnA knowledge bases and other NLP sources (added to dispatch as a file type).

Use the Dispatch model in cases when:

• Your bot consists of multiple modules and you need assistance in routing user's utterances to these modules and evaluate the bot integration.
• Evaluate quality of intents classification of a single LUIS model.
• Create a text classification model from text files.

To try this sample

• Clone the repository

  git clone https://github.com/Microsoft/botbuilder-samples.git

• In a terminal, navigate to samples/javascript_nodejs/14.nlp-with-dispatch

  cd samples/javascript_nodejs/14.nlp-with-dispatch

• Install modules

  npm install

• Start the bot

  npm start

Testing the bot using Bot Framework Emulator v4

Microsoft Bot Framework Emulator is a desktop application that allows bot developers to test and debug their bots on localhost or running remotely through a tunnel.

• Install the Bot Framework emulator from here

Connect to bot using Bot Framework Emulator v4

• Launch Bot Framework Emulator
• File -> Open Bot Configuration and navigate to samples/javascript_nodejs/14.nlp-with-dispatch folder
• Select nlp-with-dispatch.bot file

Deploy this bot to Azure and configure services

You can use the MSBot Bot Builder CLI tool to clone and configure any services this sample depends on.

To install all Bot Builder tools -

Ensure you have Node.js version 8.5 or higher

npm i -g msbot chatdown ludown qnamaker luis-apis botdispatch luisgen

To clone this bot, perform the following:

• Collect your Luis Authoring Key from the the LUIS portal by selecting your name in the top right corner. Save this key for the next step.

• Run the following command from the project directory:

msbot clone services --name "<NAME>" --luisAuthoringKey "<YOUR AUTHORING KEY>" --folder "DeploymentScripts/MsbotClone" --location "ie, westus" --appId <YOUR APP ID> --appSecret <YOUR APP SECRET PASSWORD>

NOTE: You can obtain your appId and appSecret at the Microsoft's Application Registration Portal

NOTE: By default your Luis Applications will be deployed to your free starter endpoint. An Azure LUIS service will be deployed along with your bot but you must manually add and publish to it from the luis.ai portal and update your key in the .bot file.

• Note the generated secret generated by msbot.
• The secret key is used later in for the emulator and configuration.

The secret used to decrypt <NAME>.bot is:
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX=
NOTE: This secret is not recoverable and you should store this secret in a secure place according to best security practices.
Your project may be configured to rely on this secret and you should update it as appropriate.

• Inspect Bot configuration file.

• The msbot clone command above generates a bot configuration file.

• The name of the bot configuration file is .bot, where is the name of your bot used in the msbot clone step.

• The configuration file can be loaded by the Microsoft Bot Framework Emulator.

• Update bot.js Update the following line to add a prefix with the name of your bot (plus underscore '_').

const DISPATCH_CONFIG = '<NAME>_nlp-with-dispatchDispatch';

• Update homeAutomation.js Update the following line to add a prefix with the name of your bot (plus underscore '_').

const LUIS_CONFIGURATION = '<NAME>_Home Automation';

• Update weather.js Update the following line to add a prefix with the name of your bot (plus underscore '_').

const WEATHER_LUIS_CONFIGURATION = '<NAME>_Weather';

Alternately you can configure the required services by following the steps below.

Manually configure required services

Configure the LUIS service

To create required LUIS applications for this sample bot,

• Create an account with LUIS. If you already have an account, login to your account.
• Click on your name on top right corner of the screen -> settings and grab your authoring key.

To create the LUIS application this bot needs and update the .bot file configuration, in a terminal,

• Clone this repository
• Navigate to samples/javascript_nodejs/14.nlp-with-dispatch
• Run the following command

> ludown parse toluis --in resources/homeautomation.lu -o cognitiveModels --out homeAutomation.luis -n "Home Automation" -d "Home Automation LUIS application - Bot Builder Samples" --verbose

> ludown parse toluis --in resources/weather.lu -o cognitiveModels --out weather.luis -n Weather -d "Weather LUIS application - Bot Builder Samples" --verbose

> luis import application --in cognitiveModels/homeAutomation.luis --authoringKey <LUIS-AUTHORING-KEY> --region <LUIS-AUTHORING-REGION> --msbot | msbot connect luis --stdin

> luis import application --in cognitiveModels/weather.luis --authoringKey <LUIS-AUTHORING-KEY> --region <LUIS-AUTHORING-REGION> --msbot | msbot connect luis --stdin

Note: You can create the LUIS applications in one of the LUIS authoring regions. You can use a different region (westus or westeurope or australiaeast) by specifying them as --region value in the commands above.

Train and publish the LUIS models

You need to train and publish the LUIS models that were created for this sample to work. You can do so using the following CLI commands

> msbot get "Home Automation" | luis train version --wait --stdin
> msbot get "Weather" | luis train version --wait --stdin
> msbot get "Home Automation" | luis publish version --stdin
> msbot get "Weather" | luis publish version --stdin

Configure QnA Maker service

To create a new QnA Maker application for the bot,

• Follow instructions here to create a new QnA Maker Azure resource.
• Navigate to your QnA Maker resource -> keys and copy the subscription key

To create the QnA Maker application and update the .bot file with the QnA Maker configuration,

• Open a terminal
• Navigate to samples/javascript_nodejs/14.nlp-with-dispatch
• Run the following command

> ludown parse toqna --in resources/sample-qna.lu -o cognitiveModels --out dispatch.qna --verbose

> qnamaker create kb --in cognitiveModels/dispatch.qna --subscriptionKey <QNA-MAKER-SUBSCRIPTION-KEY> --msbot | msbot connect qna --stdin

Train and publish the QnA Maker KB

You need to train and publish the QnA Maker Knowledge Bases that were created for this sample to work. You can do so using the following CLI commands

> msbot get "sample-qna" | qnamaker publish kb --stdin

Configure the Dispatch application

Dispatch is a CLI tool that enables you to create a dispatch NLP model across the different LUIS applications and/ or QnA Maker Knowledge Bases you have for your bot. For this sample, you would have already created 2 LUIS applications (Home Automation and Weather) and one QnA Maker Knowledge base.

To create a new dispatch model for these services and update the .bot file configuration, in a terminal,

• Navigate to samples/javascript_nodejs/14.nlp-with-dispatch
• Run the following command

> dispatch create -b nlp-with-dispatch.bot | msbot connect dispatch --stdin

Then go into your bot file and for the "dispatch"-type service with the name "nlp-with-dispatchDispatch", add a "subscriptionKey" with your preferred LUIS Key. You can use the "subscriptionKey" that's found under the "luis"-type services.

Securing keys in your .bot file

Since your .bot file contains service IDs, subscription and authoring keys, its best to encrypt them. To encrypt the .bot file, you can run

msbot secret -n

This will generate a strong key, encrypt the bot file and print the key. Please keep this key securely.

Any time the bot file is encrypted, make sure to set the botFileSecret environment variable this sample relies on (either through the .env file or other means).

Testing the bot using Bot Framework Emulator

Microsoft Bot Framework Emulator is a desktop application that allows bot developers to test and debug their bots on localhost or running remotely through a tunnel.

• Install the Bot Framework emulator from here

Connect to bot using Bot Framework Emulator V4

• Launch Bot Framework Emulator
• File -> Open Bot Configuration and navigate to samples/javascript_nodejs/14.nlp-with-dispatch folder
• Select nlp-with-dispatch.bot file