Merge pull request #6 from microsoft/master

grabbing latest changes

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,32 +1,20 @@
-<!--- Provide a general summary of your changes in the Title above -->
-## Description
-<!--- Describe your changes in detail under any applicable category below and remove those that don't apply. This repository only accepts pull requests related to open issues, please link the open issue in description below. See https://help.github.com/articles/closing-issues-using-keywords/ for more information on automation. 
+<!--- 
+This repository only accepts pull requests related to open issues, please link the open issue in description below. 
+See https://help.github.com/articles/closing-issues-using-keywords/ to learn about automation. 
 For example...
-Close #123: This description for this goes here.-->
+Close #123: Description for this goes here.
+-->
 
-### Architecture
+## Purpose
+*What is the context of this pull request? Why is it being done?*
 
-### Bug Fixes
+## Changes
+*Are there any changes that need to be called out as significant or particularly difficult to grasp?*
 
-### Conversational Experience
+*Include illustrative screenshots for context if applicable.*
 
-### Maintenance
+## Tests
+*Is this covered by existing tests or new ones? If no, why not?*
 
-### Language Understanding
-
-## Testing Steps
-<!--- Include any instructions for testing your Pull Request. Include sample utterances, steps, etc. -->
-
-## Checklist
-<!--- You can remove any items below that don't apply to the pull request. -->
-- [ ] I have commented my code, particularly in hard-to-understand areas
-- [ ] I have added or updated the appropriate unit tests
-- [ ] I have tested any new/updated dialogs using Speech in the emulator to ensure the [Speak](https://docs.microsoft.com/en-us/azure/bot-service/dotnet/bot-builder-dotnet-text-to-speech?view=azure-bot-service-3.0) property is set to enable a high quality speech-first experience and the appropriate [InputHints](https://docs.microsoft.com/en-us/azure/bot-service/dotnet/bot-builder-dotnet-add-input-hints?view=azure-bot-service-3.0) are set correctly. 
-- [ ] I have updated related documentation
-
-<!--- If this contains changes that needs to be replicated between the Enterprise Template <-> Virtual Assistant-->
-- [ ] A duplicate issue is filed to track future  work.
-
-<!--- If you have updated responses or `.lu` files:-->
-- [ ] All languages have been updated
-- [ ] You have tested deployment with your new models
+## Feature Plan
+*Are there any remaining steps or dependencies before this issue can be fully resolved? If so, describe and link to any relevant pull requests or issues.*

.github/Tasks/Pull-Request-Commenter/Bots/bots.txt

@@ -0,0 +1,7 @@
+Thank you for raising a pull request. Please ensure you have have completed the following:
+
+- [ ] I have commented my code, particularly in hard-to-understand areas	
+- [ ] I have added or updated the appropriate tests	
+- [ ] I have updated related documentation
+- [ ] I have validated that new and updated responses use appropriate [Speak](https://docs.microsoft.com/en-us/azure/bot-service/dotnet/bot-builder-dotnet-text-to-speech?view=azure-bot-service-3.0) and [InputHint](https://docs.microsoft.com/en-us/azure/bot-service/dotnet/bot-builder-dotnet-add-input-hints?view=azure-bot-service-3.0) properties to ensure a high-quality speech-first experience
+- [ ] I have replicated language model changes across the English, French, Italian, German, Spanish, and Chinese `.lu` files and validated that deployment is successful

.github/Tasks/Pull-Request-Commenter/Default/default.txt

@@ -0,0 +1,5 @@
+Thank you for raising a pull request. Please ensure you have have completed the following:
+
+- [ ] I have commented my code, particularly in hard-to-understand areas	
+- [ ] I have added or updated the appropriate tests	
+- [ ] I have updated related documentation

.github/Tasks/Pull-Request-Commenter/Deployment-Scripts/deployment_scripts.txt

@@ -0,0 +1,6 @@
+Thank you for raising a pull request. Please ensure you have have completed the following:
+
+- [ ] I have commented my code, particularly in hard-to-understand areas	
+- [ ] I have updated related documentation
+- [ ] I have replicated my changes in the **Virtual Assistant Template** and **Sample** projects
+- [ ] I have replicated my changes in the **Skill Template** and **Sample** projects

.github/Tasks/Pull-Request-Commenter/bots.txt

@@ -0,0 +1,7 @@
+Thank you for raising a pull request. Please ensure you have have completed the following:
+
+- [ ] I have commented my code, particularly in hard-to-understand areas	
+- [ ] I have added or updated the appropriate tests	
+- [ ] I have updated related documentation
+- [ ] I have validated that new and updated responses use appropriate [Speak](https://docs.microsoft.com/en-us/azure/bot-service/dotnet/bot-builder-dotnet-text-to-speech?view=azure-bot-service-3.0) and [InputHint](https://docs.microsoft.com/en-us/azure/bot-service/dotnet/bot-builder-dotnet-add-input-hints?view=azure-bot-service-3.0) properties to ensure a high-quality speech-first experience
+- [ ] I have replicated language model changes across the English, French, Italian, German, Spanish, and Chinese `.lu` files and validated that deployment is successful

.github/Tasks/Pull-Request-Commenter/default.txt

@@ -0,0 +1,5 @@
+Thank you for raising a pull request. Please ensure you have have completed the following:
+
+- [ ] I have commented my code, particularly in hard-to-understand areas	
+- [ ] I have added or updated the appropriate tests	
+- [ ] I have updated related documentation

.github/Tasks/Pull-Request-Commenter/deployment_scripts.txt

@@ -0,0 +1,6 @@
+Thank you for raising a pull request. Please ensure you have have completed the following:
+
+- [ ] I have commented my code, particularly in hard-to-understand areas	
+- [ ] I have updated related documentation
+- [ ] I have replicated my changes in the **Virtual Assistant Template** and **Sample** projects
+- [ ] I have replicated my changes in the **Skill Template** and **Sample** projects

.github/stale.yml

@@ -0,0 +1,21 @@
+# Number of days of inactivity before an issue becomes stale
+daysUntilStale: 10
+# Number of days of inactivity before a stale issue is closed
+daysUntilClose: 3
+# Issues with these labels will never be considered stale
+exemptLabels:
+  - pinned
+  - security
+  - Support
+  - Status: Backlog
+  - Status: Committed
+  - Status: In Progress
+  - Status: In Review
+# Label to use when marking an issue as stale
+staleLabel: Status: Removed
+# Comment to post when marking an issue as stale. Set to `false` to disable
+markComment: >
+  This issue has been automatically marked as stale because it has not had
+  recent activity. It will be closed if no further activity occurs. Feel free to reopen as needed.
+# Comment to post when closing a stale issue. Set to `false` to disable
+closeComment: true

azure-pipelines.yml

@@ -0,0 +1,19 @@
+# Starter pipeline
+# Start with a minimal pipeline that you can customize to build and deploy your code.
+# Add steps that build, run tests, deploy, and more:
+# https://aka.ms/yaml
+
+trigger:
+- master
+
+pool:
+  vmImage: 'ubuntu-latest'
+
+steps:
+- script: echo Hello, world!
+  displayName: 'Run a one-line script'
+
+- script: |
+    echo Add other tasks to build, test, and deploy your project.
+    echo See https://aka.ms/yaml
+  displayName: 'Run a multi-line script'

docs/_config.yml

@@ -0,0 +1 @@
+theme: jekyll-theme-minimal

docs/howto/skills/addingskills.md

@@ -33,7 +33,7 @@ See the [Skills Overview](/docs/README.md#skills) section for details on the Ski
 
 ## Adding Skills to your Virtual Assistant
 
-Run the following command to add each Skill to your Virtual Assistant. This assumes you are running the CLI within the project directory and have created your Bot through the template and therefore have a `skills.json` file present in the working folder.
+Run the following command to add each Skill to your Virtual Assistant. This assumes you are running the CLI within the **assistant project directory** and have created your Bot through the template and therefore have a `skills.json` file present in the working folder.
 
 The `--luisFolder` parameter can be used to point the Skill CLI at the source LU files for trigger utterances. For Skills provided within this repo these can be found in the `Deployment\Resources\LU` folder of each Skill. The CLI will automatically traverse locale folder hierarchies.  This can be omitted for any of the skills we provide as the LU files are provided locally. Also, you have to specify the `--cs` (for C#) or `--ts` (for TypeScript) argument for determining the coding language of your assistant, since each language takes different folder structures that need to be taken into consideration.
 
@@ -56,14 +56,14 @@ Could not configure scopes automatically. You must configure the following scope
 In this situation for Microsoft Graph based skills follow the instructions below:
 
 1. Find the Azure AD Application for your Bot within the [Azure Portal](https://ms.portal.azure.com/#blade/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/RegisteredAppsPreview)
-2. In the Authentication section ensure the Redirect Uri is set to `https://token.botframework.com/.auth/web/redirect`
+2. In the Authentication section ensure the Redirect Uri is set to `https://token.botframework.com/.auth/web/redirect` and Supported Account Types is set to `Accounts in any organizational directory`.
 3. In the API permissions section click Add Permission, then Microsoft Graph and Delegated Permissions. Find each scope provided in the message shown during Skill registration and add.
 
 For Skills that require other Authentication connection configuration please follow the skill specific configuration information.
 
 ## Remove a Skill from your Virtual Assistant
 
-To disconnect a skill from your Virtual Assistant use the following command, passing the id of the Skill as per the manifest (e.g. calendarSkill). You can use the `botskills list` to view the registered skills.
+To disconnect a skill from your Virtual Assistant use the following command, passing the id of the Skill as per the manifest (e.g. calendarSkill). You can use the `botskills list` to view the registered skills. **Ensure your run this with the assistant project folder.**
 
 botskills:
 
@@ -86,7 +86,7 @@ botskills update --botName YOUR_BOT_NAME --remoteManifest "http://<YOUR_SKILL_MA
 ```
 
 ## Refresh Connected Skills
-To refresh the dispatch model with any changes made to connected skills use the following command, specifying the `--cs` (for C#) or `--ts` (for TypeScript) argument for determining the coding language of your assistant, since each language takes different folder structures that need to be taken into consideration. 
+To refresh the dispatch model with any changes made to connected skills use the following command, specifying the `--cs` (for C#) or `--ts` (for TypeScript) argument for determining the coding language of your assistant, since each language takes different folder structures that need to be taken into consideration. **Ensure your run this with the assistant project folder.**
 
 botskills:
 

docs/readme.md

@@ -49,7 +49,6 @@
 | [How-To: Customize Azure Resource Deployment](/docs/howto/assistant/customizedeployment.md) | How to customise the provided ARM template for different deployment scenarios. |
 | [How-To: Update your deployment scripts](/docs/howto/assistant/updatedeploymentscripts.md) | How to update the deployment scripts for Virtal Assistant and Skill projects that you have already created. |
 
-
 ## Skills
 
 | Name | Description |

docs/reference/knownissues.md

@@ -2,58 +2,76 @@
 
 ## The Teams channel doesn't render OAuth cards.
 
-There is a known issue in the Teams channel where the default OAuth Card is not shown when using the OAuthPrompt in the SDK. To workaround this in the short-term please follow the following instructions.
+Prior versions of the BF SDK and VA template experienced issues when using Teams whereby Authentication cards (OAuthPrompt generated) did not function as expected. This required manual changes to work around the issue which are now incorporated into the BF SDK and Virtual Assistant template. If you experience these problems please:
 
-Please be aware that you *must* use [App Studio](https://docs.microsoft.com/en-us/microsoftteams/platform/get-started/get-started-app-studio) to create an Application Manifest. Otherwise you won't be able to click any login buttons within Teams. It's key to ensure that under Domains and permissions in the Manifest Editor that you enter the domain token.botframework.com to enable clicking of the login button.  You cannot click the link in the Channel Page of the Bot Framework to start a conversation with your Bot.
+1. Update to Bot Framework SDK 4.4.5 or higher
+2. Update your `Microsoft.Bot.Builder.Solutions` and `Microsoft.Bot.Builder.Skills` nuget packages to 4.4.4.1 or higher.
 
-You then need to make the following code changes:
+Please be aware that you **must** use [App Studio](https://docs.microsoft.com/en-us/microsoftteams/platform/get-started/get-started-app-studio) to create an Application Manifest when using Teams. Otherwise you won't be able to click any login buttons within Teams. 
 
-### DialogBot.cs in your Bots folder
+It's key to ensure that under Domains and permissions in the Manifest Editor that you enter the domain `token.botframework.com` to enable clicking of the login button. 
 
-Add the following under the `if` handler for BotTimedOut. This responses to the Invoke messages sent by Teams and addresses the issue where the animated circle keeps spinning as Teams hasn't been responded to.
-```
-else if (turnContext?.Activity.Type == ActivityTypes.Invoke && turnContext?.Activity.Name == "signin/verifyState")
-{
-       await turnContext.SendActivityAsync(new Activity(ActivityTypesEx.InvokeResponse, value: null));
-}
-```
+>You cannot click the link in the Channel Page of the Bot Framework to start a conversation with your Bot through Teams.
+
+## WebChat doesn't work with your Virtual Assistant / Skills
 
-### MainDialog.cs in Dialogs folder
+Virtual Assistant's created using an earlier version of the template reference an older version of the nuget packages. Update your `Bot.Builder.Skills` and `Bot.Builder.Solutions` to the latest versions (4.4.41 or higher) along with the Bot Framework SDK to 4.4.5 or higher. Then apply the change in the item below.
 
-This change ensures the Invoke message is propogated into the waiting Dialog. The RouterDialog doesn't pass this along as it wasn't an expected ActivityType so we have to override this behaviour. We'll push a new Bot.Solutions package with a change to remove the need for this override shortly.
+## When invoking a Skill you may experience the initial message being sent twice
+
+We made a change to the behaviour when invoking Skills which removed the need for an additional `SkillBegin` event. This change enabled simplification of the SkillDialog logic which the latest Virtual Assistant template has incorporated. However existing assistants created using an earlier version of the template who have updated to the latest `Bot.Builder.Skills/Bot.Builder.Solutions` packages may experience a situation, whereby, when invoking a Skill the initial message is sent twice. 
+
+This is due to a line of code which can be safely removed from your existing Assistant project. Within your `MainDialog.cs` file and the `RouteAsync` method, remove the call to `ContinueDialogAsync` by changing the existing code:
 
 ```
-protected async override Task<DialogTurnResult> OnContinueDialogAsync(DialogContext innerDc, CancellationToken cancellationToken = default)
-{
-    if (innerDc.Context.Activity.Type == ActivityTypes.Invoke)
-    {   
-            var result = await innerDc.ContinueDialogAsync();
-
-            if (result.Status == DialogTurnStatus.Complete)
-            {
-                await CompleteAsync(innerDc);
-            }
-
-        return result;
-    }
-    else
-    {
-        return await base.OnContinueDialogAsync(innerDc, cancellationToken);
-    }
-}
+await dc.BeginDialogAsync(identifiedSkill.Id);
+
+// Pass the activity we have
+var result = await dc.ContinueDialogAsync();
 ```
+To
+```
+var result = await dc.BeginDialogAsync(identifiedSkill.Id);
+```
+Ensure you have also updated to the latest `Bot.Builder.Skills/Bot.Builder.Solutions` packages .
 
 ## QnAMaker can't be entirely deployed in a region other than westus.
 
 QnAMaker has a central Cognitive Service resource that must be deployed in `westus`, the dependent Web App, Azure Search and AppInsights resources can be installed in a broader range of regions. We have therefore fixed this QnAMaker resource to be `westus` in the ARM template (template.json) which can be found in the `deployment\resources` folder of your Virtual Assistant. When deploying in a region such as westeurope all dependent resources will be created in the requested region. This script will be updated as new regions are available.
 
-## Skills don't respond when using WebChat
+## Telemetry Gaps / FlowAggregate errors in the PowerBI Template
+
+If you try to use the PowerBI analytics dashboard with your Virtual Assistant / Skills and experience a `Errors in FlowAggregates` issue or experience some telemetry not being collected this likely relates to a bug experienced in the initial version of the Virtual Assistant template and Skills which has now been addressed.
+
+1. Change `appInsights` in appSettings.config to `ApplicationInsights`
+
+```
+"ApplicationInsights": {
+    "InstrumentationKey": ""
+  }
+```
+
+2. Update your `BotServices.cs` file with the changes [here](https://github.com/microsoft/botframework-solutions/blob/master/templates/Virtual-Assistant-Template/csharp/Sample/VirtualAssistantSample/Services/BotServices.cs).
+
+3. Update your `Startup.cs` file with the changes [here](https://github.com/microsoft/botframework-solutions/blob/master/templates/Virtual-Assistant-Template/csharp/Sample/VirtualAssistantSample/Startup.cs)
+
+4. Existing data in your Application Insights may cause the error to persist. You can either drop and re-create your Application insights resource updating the appSettings.config file with the new Instrumentation key or follow these [purge instructions](https://docs.microsoft.com/en-us/rest/api/application-insights/components/purge).
+
+
+## Deployment doesn't assign the newly created LUIS subscription key to the created LUIS models / LUIS Forbidden Error.
+
+Due to a limitation with the LUIS authoring APIs the deployment script isn't able to assign the newly created LUIS subscription-key to the deployed and published LUIS models. Instead, the current workaround is to rely on the Starter Key meaning the Virtual Assistant and Skills work with no manual steps. We are working on changes to the LUIS authoring APIs to enable this to be assigned automatically.
+
+This may cause you to also experience `Forbidden` LUIS errors when testing your Bot as you may have exhausted the quota for your starter LUIS key, changing from your starter LUIS subscription key will resolve this.
+
+In the meantime, you'll have to manually perform the following steps:
+
+1. As shown below go through **each LUIS model including Dispatch**, click Assign Resoucre and locate the appropriate subscription key and then re-publish. 
 
-There is a known issue with regard to responses from Skills not being returned to WebChat clients through the Virtual Assistant. They do however work across other channels. We have a fix in the latest daily build for [Bot.Builder.Skills](https://botbuilder.myget.org/feed/aitemplates/package/nuget/Microsoft.Bot.Builder.Skills) and [Bot.Builder.Solutions](https://botbuilder.myget.org/feed/aitemplates/package/nuget/Microsoft.Bot.Builder.Solutions) which resolves the problem. This will be published to nuget in our next release.
+![Assign Resource](/docs/media/luis-assignresource.png)
 
-There is a command line example on the Myget page of how to install from this repository but you can also add a package source in Visual Studio referencing https://botbuilder.myget.org/F/aitemplates/api/v3/index.json
+2. Update the `subscriptionKey` for each LUIS model (includign Dispatch) in `cognitiveModels.json` with your new subscription key. 
 
-Once you have updated the packages, redeploy your Virtual Assistant to Azure and try again.
 
 ## The introduction card isn't displayed when a locale is missing