Merge pull request #384 from aws-solutions/develop

Update to version v5.1.0

BotRoutingREADME.md

@@ -1,5 +1,5 @@
 # Bot Routing - Preview Mode
-(version 1.0 - December 2020)
+(version 1.1 - November 2021)
 
 Bots exist to perform a variety of automation tasks. Usually they take
 as input a message from a human and respond performing
@@ -54,13 +54,25 @@ BotRouter for the question being edited.**
 ![Configuration](./docs/botroutingconfig.png)
 
 * Bot Name or Lambda - You can configure and existing Lex Bot or configure
-a specialty BotRouter implemented via a Lambda function.
+a specialty BotRouter implemented via a Lambda function. QnABot route 
+requests to a Lex Version 1 or Version 2 bot. The syntax is different
+depending on the target version. For Lex version 2 bots use the syntax
+lexv2::BotId/BotAliasId/LocaleId. For Lex version 1 bots use just the 
+BotName. Note that QnABot can be installed as either Lex V1 or Lex V2
+bot. It is important to check your QnABot configuration and select the
+appropriate identifiers when you want to use QnABot as a specialty bot.
 
 * Simple name - A short string that we expect web User Interfaces to use as
 a breadcrumb to identify where in an enterprise the user is interacting.
 
-* Lex Alias - If a target bot is configured, the alias used to communicate
-with the target bot.
+* Lex Alias - If your specialty bot is a Lex version 1 bot, specify the Lex version 1
+alias used to communicate with the target bot.
+
+* Lex session attributes to forward to the specialty bot. A comma separated
+list of session attribute names can be specified. The session attributes
+will be passed on each request to the specialty bot. They will override
+any session attributes which might have been returned from the specialty bot
+on the prior request. 
 
 The example image shows an integration we've developed which communicates
 with the Nutritionix Bot.

CHANGELOG.md

@@ -5,6 +5,31 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [5.1.0] - 2021-12-09
+
+### Added
+- Tags to questions in Content Designer and ability to create reports in Kibana.
+- Integration with Genesys call center platform.
+- Client Filtering with Session Attributes (i.e., Support to allow the same set of questions to be answered differently based on a session attribute).
+- Intelligent redaction of Personally Identifiable Information in logs with Amazon Comprehend.
+- A QnABot client (e.g. a Connect contact flow) can now  optionally provide a value for session attribute, `qnabotUserId`. When this session attribute is set, QnABot tracks user activity based on the provided value. If not set, QnABot continues to track user activity based on the request userId (LexV1) or sessionId (LexV2). NOTE: `qnabotUserId` value is not used when user authentication using JWTs is enabled - in this case users are securely identified and verified from the JWT.
+- Support for pre and post processing AWS Lambda Hooks.
+- Setting that determines whether Amazon Kendra responses are abbreviated when sent via SSML (`ALT_SEARCH_KENDRA_ABBREVIATE_MESSAGE_FOR_SSML`)
+- Setting that determines the types of responses that Amazon Kendra returns (`ALT_SEARCH_KENDRA_RESPONSE_TYPES`: `ANSWER`, `DOCUMENT`, `QUESTION_ANSWER`)
+
+### Fixed
+- Test tab in Content Designer to show same results as the web client when using Kendra FAQ.
+- Broken link in documentation for downloading CloudFormation template.
+- Integration with Slack on Amazon LexV2 bots.
+- QnABot will set the sessionAttribute from CONNECT_NEXT_PROMPT_VARNAME to an empty string if QnABot is in a response bot in a voice channel. This will prevent QnABot from saying the next prompt in the middle of a response bot flow.
+- Kendra FAQ sync where export Lambda was missing the Layer containing qnabot log.
+- Bug with response bots with Alexa where QnABot was filling in a malformed reprompt.text in the response.
+- Excel import improvements and bug fixes.
+
+### Changed
+- Bot routing capability to have multiple-bot architecture (e.g., General bot routing questions to specialty bots).
+- The built-in Amazon Kendra “web page indexer” with the new Kendra Web Crawler Datasource.
+
 ## [5.0.1] - 2021-10-20
 
 ### Added

Makefile

@@ -1,13 +1,14 @@
-TEMPLATES=$(shell for l in $$(ls ./templates | grep -v util | grep -v README.md);do echo templates/$$l;done)
+TEMPLATES=$(shell for l in $$(ls ./templates | egrep -v "util|lib|README.md");do echo templates/$$l;done)
 
 All: assets templates lambda website make_directories
 
 build: All
 
 make_directories:
-	mkdir -p build; mkdir -p build/lambda; mkdir -p build/templates/test;mkdir -p build/templates;mkdir -p build/documents; mkdir -p build/templates/dev
+	mkdir -p build/lambda build/documents build/templates/test  build/templates/dev
 
 .PHONY: lambda templates upload website test bootstrap assets config.aws-solutions.json
+.PHONY: $(TEMPLATES)
 
 config.json:
 	node bin/config.js > config.json
@@ -21,10 +22,10 @@ lambda:  make_directories
 bootstrap: make_directories
 	$(MAKE) ../../build/templates/dev/bootstrap.json -C templates/dev
 
-templates: $(TEMPLATES) make_directories
-	for l in $(TEMPLATES); do	\
-		$(MAKE) -C $$l;			\
-	done;
+templates: $(TEMPLATES)
+
+$(TEMPLATES): make_directories
+	$(MAKE) -C $@
 
 website: make_directories
 	$(MAKE) -C ./website

NOTICE.txt

@@ -22,8 +22,6 @@ This software includes third party software subject to the following copyrights:
 @babel/preset-stage-2 under the Massachusetts Institute of Technology (MIT) license
 ajv under the Massachusetts Institute of Technology (MIT) license
 alexa-sdk under the Apache License Version 2.0
-amazon-cognito-identity-js under the Apache License Version 2.0
-ask-cli under the Apache License Version 2.0
 async-mutex under the Massachusetts Institute of Technology (MIT) license
 autosize under the Massachusetts Institute of Technology (MIT) license
 aws-lex-web-ui under the Amazon Software License

README.md

@@ -8,11 +8,19 @@ This repository contains code for the QnABot, described in the AWS AI blog post
 
 Refer to [Getting Started](#getting-started) to launch your own QnABot, or [Automated Deployment](https://docs.aws.amazon.com/solutions/latest/aws-qnabot/automated-deployment.html) for detailed implementation instructions.
 
-**See all the new features list for 4.7.3** [Fulfillment Lambda Provisioned Concurrency](CHANGELOG.md#473---2021-08-04)
+The 5.1.0 release of QnABot provides support for a new mechanism where a Question can set a Lex session attribute. 
 
-**New features in 4.6.0** [Improved Kendra integration and Kibana dashboards. Additional settings to filter Kendra responses based on confidence levels](CHANGELOG.md#460---2021-04-30).
+In prior versions, the use of handlebars in an answer could set a session attribute. For example, the following
 
-**New features in 4.5.0** [Kendra Web Crawler, Comprehend PII Detection, Translate Custom Terminology, Increased deployment regions](CHANGELOG.md#450---2021-03-07).
+"{{setSessionAttr ‘attributeName' ‘attributeValue’}}""
+
+will set the an attribute called “attributeName” to the value “attributeValue”. The drawback in this approach is that the attribute value could not itself contain nested handle bars. 
+
+In 5.1.0, a Question can now define a set of name/value pairs to set as session attributes when the answer is returned. You’ll see a new field to set a name/value pair plus an “Add” button to define additional name/value pairs and a button to delete a name/value pair. The Value field can contain handlebar statements as well.
+
+The attribute name can be a simple name like “myAttribute” or a complex name like “myAttribute.subAttribute”. Using the dot notation an attribute several levels deep can be set. Note that one can overwrite reserved session attribute names. Avoid using “appContext” or “qnabotcontext” as attribute names. Setting these may have adverse effects on the system.  
+
+Note, the handlebar syntax still works just as it did before. The question configuration just has a new option.
 
 ## Architecture Overview
 ![Architecture](docs/architecture.png)

bin/check.js

@@ -60,7 +60,7 @@ async function run(stack,options={}){
             var Bucket=exp.Bucket
             var prefix=exp.Prefix
             var Key=`${prefix}/templates/${stack}.json`
-            var TemplateURL=`http://${Bucket}.s3.${region}.amazonaws.com/${Key}`
+            var TemplateURL=`https://${Bucket}.s3.${region}.amazonaws.com/${Key}`
             console.log(TemplateURL)
             await s3.putObject({Bucket,Key,Body:template}).promise()
             return cf.validateTemplate({TemplateURL}).promise()

bin/json.js

@@ -261,7 +261,7 @@ function printHelp() {
     w('                    -4   =>  -o jsony-4');
     w('                    -j0  =>  -o json-0');
     w('');
-    w('See <http://trentm.com/json> for more docs and ');
+    w('See <https://trentm.com/json> for more docs and ');
     w('<https://github.com/trentm/json> for project details.');
     /* END JSSTYLED */
 }

bin/launch.js

@@ -123,7 +123,7 @@ async function up(stack,options){
                 var exp=await bootstrap()
                 var bucket=exp.Bucket
                 var prefix=exp.Prefix
-                var url=`http://${bucket}.s3.${region}.amazonaws.com/${prefix}/templates/${stack}.json`
+                var url=`https://${bucket}.s3.${region}.amazonaws.com/${prefix}/templates/${stack}.json`
                 await s3.putObject({
                     Bucket:bucket,
                     Key:`${prefix}/templates/${stack}.json`,
@@ -171,7 +171,7 @@ function update(stack,options){
                 var start=bootstrap().then(function(exp){
                     var bucket=exp.Bucket
                     var prefix=exp.Prefix
-                    var url=`http://${bucket}.s3.${region}.amazonaws.com/${prefix}/templates/${stack}.json`
+                    var url=`https://${bucket}.s3.${region}.amazonaws.com/${prefix}/templates/${stack}.json`
                     console.log(url)
                     return s3.putObject({
                         Bucket:bucket,

docs/PII_Detection/README.md

@@ -1,16 +1,18 @@
-# Personally Identifiable Information Rejection
+# Personally Identifiable Information Rejection and redaction
 
-QnABot can now detect Personally Identifiable Information (PII) using [Amazon Comprehend](https://docs.aws.amazon.com/comprehend/latest/dg/how-pii.html) and regular expressions
+QnABot can now detect Personally Identifiable Information (PII) using [Amazon Comprehend](https://docs.aws.amazon.com/comprehend/latest/dg/how-pii.html) and regular expressions.
 
+If ENABLE_REDACTION is set to "true", the Comprehend detected PII entities will also be redacted from Amazon CloudWatch logs and Amazon Opensearch logs.
 
-![](./settings.png)
-
-
-Setting | Description |
----------|----------|
- PII_REJECTION_ENABLED | set to "true" to enable pii rejection |
- PII_REJECTION_QUESTION | If PII is found, the user's request (question) will change to this phrase|
- PII_REJECTION_WITH_COMPREHEND | Enable's Personal Identifiable Information detection with Amazon Comprehend |
- PII_REJECTION_REGEX | Used to find PII based on a regular expression |
- PII_REJECTION_IGNORE_TYPES | Do not detect the specified list of [entity types](https://aws.amazon.com/blogs/machine-learning/detecting-and-redacting-pii-using-amazon-comprehend/) |
+![settings image](./settings.png)
 
+|Setting | Type of Value | Description |
+--------|---------------|-------------|
+| ENABLE_REDACTING | true or false | Enable the system to redact log output
+| REDACTING_REGEX | regex expression | Redacts expressions matching regex from logs
+| PII_REJECTION_ENABLED | true or false | Enables PII Rejection
+| PII_REJECTION_QUESTION | text  | If PII is found, the user's request (question) will change to this phrase
+| PII_REJECTION_WITH_COMPREHEND: | true or false | Enable's [Personal Identifiable Information](https://aws.amazon.com/blogs/machine-learning/detecting-and-redacting-pii-using-amazon-comprehend/) detection with Amazon Comprehend
+| PII_REJECTION_REGEX | regex expression | Used to find PII based on a regex
+| PII_REJECTION_ENTITY_TYPES | comma separated list of [PII Entity Categories](https://aws.amazon.com/blogs/machine-learning/detecting-and-redacting-pii-using-amazon-comprehend/) | Only recognize PII entity types in the list
+| DISABLE_CLOUDWATCH_LOGGING | true or false | Disable all logging in fulfillment es query handler lambda. does not disable logging from Lambda Hooks or Conditional Chaining Lambda functions

docs/excel_import/README.md

@@ -28,6 +28,29 @@ Excel Column | Content Designer Field| Description
  displaytext1...displaytext(n) | Display Text | The text of the button. Each button text should be in a separate column numbered button1, button2, ...
  buttonvalue...buttonvalue(n) | Button Value| The value of the button. Each button value should be in a separate column numbered button1, button2, ...
 
+## Support for other Content Designer fields
+
+While special affordances have been made to allow importing the fields above, any "string" field in the [Content Designer schema](../../lambda/schema/qna.js)
+can be imported by specifying the JSON path as the column.  
+
+### Example of a field on the root level
+
+![Schema snippet](schema2.png)
+
+Excel Column | Content Designer Field
+-------------|-----------------------
+clientFilterValues | Client Filter: Values
+
+
+### Examples of nested fields
+
+![Schema snippet](schema.png)
+
+Excel Column | Content Designer Field (title in the schema.json) 
+-------------|------------------------
+elicitResponse.response_hook | Elicit Response: Responsebot Hook
+elicitResponse.response_sessionattr_namespace |  Response Session Attribute Namespace
+
 ## Importing the Microsoft Excel file
 
 Choose *Import* from the Menu and then choose *From File*

docs/kendra_crawler_guide/README.md

@@ -1,29 +1,39 @@
-# Kendra Web Indexer Integration
+# Amazon Kendra Web Crawler Integration
 
-Please read the [Kendra Integration](../../workshops/kendra/README.md) for instructions about configuring Kendra
+Please read the [Amazon Kendra Integration](../../workshops/kendra/README.md) for instructions about configuring Kendra
 
-QnABot can now index your website with Kendra and answer questions based on what it has found.
+QnABot can now crawl your website with Kendra and answer questions based on what it has found.
 
 Go to the Settings option in the Tools menu
 
 ![Setting](./settings.png)
 
-1. Enable the indexer by setting _ENABLE_WEB_PAGE_INDEXER_ to true
-1. Tell QnABot which web pages to index by specifying a comma separated list of addresses in _KENDRA_INDEXER_URLS_
-1. The indexer can be optionally configured to run on a schedule by setting the _KENDRA_INDEXER_SCHEDULE_
-   It supports standard [CloudWatch Events rate syntax](https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html#RateExpressions)
-1. Tell QnABot which Kendra index to use by adding your Kendra IndexId to the setting _KENDRA_WEB_PAGE_INDEX_
+The built in "Web Page Indexer" has been replaced with Kendra's native web crawler data source.
+The settings names have been maintained for compatibility.
 
-After you save your settings, go back to the Tools Menu and then choose **Kendra Web Page Indexing**
+| Setting | Valid Values | Description |
+|---------|--------------|-------------|
+| ENABLE_KENDRA_WEB_INDEXER | true or false | Enables the web crawler
+| KENDRA_INDEXER_URLS | comma separated list | List of web addresses QnABot should crawl and [index with Kendra](./kendra_crawler_guide/README.md)
+| KENDRA_INDEXER_SCHEDULE | [CloudWatch Rate Syntax](https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html) | Interval Indexer should crawl
+| KENDRA_INDEXER_CRAWL_DEPTH | number | Sets the depth to the number of levels in a website from the seed level that you want to crawl
+| KENDRA_INDEXER_CRAWL_MODE | HOST_ONLY \| SUBDOMAINS \| EVERYTHING | Determines which addresses should be crawled
+| KENDRA_WEB_PAGE_INDEX | Kendra Index Id | The index to use for the web crawler, a [custom data source](https://docs.aws.amazon.com/kendra/latest/dg/data-source-custom.html) will automatically be added to the specified index.  
 
-![Tools](./tools.png)
+After you save your settings, go back to the Tools Menu and then choose **Kendra Web Page Crawler**
 
-![No Index Dialog](./NoIndexDialog.png)
+![Tools](./tools.png)
 
 If your settings are correct, the first time you choose the option, you should see the following dialog.
+![No Index Dialog](./NoIndexDialog.png)
 
-Just press **Start Indexing** to index your web pages to Kendra.  
+Just press **Start Crawling** to index your web pages to Amazon Kendra.  
 
 The Content Designer will show you a history of your indexing.
 
 ![Index Dialog](./IndexDialog.png)
+
+If you are logged into your AWS Account, you can click on "View Web Crawling Errors in CloudWatch" to get the details of
+any pages that Amazon Kendra couldn't crawl.
+
+![CloudWatch](cloudwatch.png)