Selectable audio input device (#3079)

* Bump versions * Select audo input device * Clean up * Detect Web Audio support * Add sample * Converge version of Carbon SDK * Add entry * Add and update entries * Skip warning if AudioConfig is presented * Add test * Add tests * Add authorization token from webchat-waterbottle * Enable IE11 * Add notes * Use daily build * Use daily build * Clean up * Apply suggestions from code review Co-Authored-By: Corina <14900841+corinagum@users.noreply.github.com> * Support Direct Line Speech * Add Direct Line Speech test * Add Direct Line Speech test * Fix Direct Line Speech tests * Improve test reliability * Save WAV files for failures * Add message * Use mockbot-streaming token * Improve reliability * Test reliability * Remove buffer bar * Remove commented out code * Add Direct Line Speech * Apply suggestions from code review Co-Authored-By: Corina <14900841+corinagum@users.noreply.github.com> * Apply PR suggestions * Update entry * Include @babel/plugin-proposal-class-properties * Fix tests Co-authored-by: Corina <14900841+corinagum@users.noreply.github.com>
microsoft · Apr 22, 2020 · 7425004 · 7425004
1 parent 23a0537
commit 7425004
Show file tree

Hide file tree

Showing 95 changed files with 2,770 additions and 782 deletions.
diff --git a/.gitignore b/.gitignore
@@ -1,4 +1,5 @@
 /__tests__/__image_snapshots__/**/__diff_output__
+/.env
 /coverage
 /debug.log
 /gh-pages

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -27,6 +27,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 -  Resolves [#2897](https://github.com/microsoft/BotFramework-WebChat/issues/2897). Moved from JUnit to VSTest reporter with file attachments, by [@compulim](https://github.com/compulim) in PR [#2990](https://github.com/microsoft/BotFramework-WebChat/pull/2990)
 -  Added `aria-label` attribute support for default Markdown engine, by [@patniko](https://github.com/patniko) in PR [#3022](https://github.com/microsoft/BotFramework-WebChat/pull/3022)
 -  Resolves [#2969](https://github.com/microsoft/BotFramework-WebChat/issues/2969). Support sovereign cloud for Cognitive Services Speech Services, by [@compulim](https://github.com/compulim) in PR [#3040](https://github.com/microsoft/BotFramework-WebChat/pull/3040)
+-  Resolves [#2481](https://github.com/microsoft/BotFramework-WebChat/issues/2481). Support selecting different audio input devices for Cognitive Services Speech Services, by [@compulim](https://github.com/compulim) in PR [#3079](https://github.com/microsoft/BotFramework-WebChat/pull/3079)
 
 ### Fixed
 
@@ -43,6 +44,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 -  Fixes [#3074](https://github.com/microsoft/BotFramework-WebChat/issues/3074). Keep `props.locale` when sending to the bot, by [@compulim](https://github.com/compulim) in PR [#3095](https://github.com/microsoft/BotFramework-WebChat/issue/3095)
 -  Fixes [#3096](https://github.com/microsoft/BotFramework-WebChat/issues/3096). Use `<ScreenReaderText>` instead of `aria-label` for message bubbles, by [@compulim](https://github.com/compulim) in PR [#3097](https://github.com/microsoft/BotFramework-WebChat/issue/3097)
 -  Fixes [#2876](https://github.com/microsoft/BotFramework-WebChat/issues/2876). `messageBack` and `postBack` should send even if both `text` and `value` is falsy or `undefined`, by [@compulim](https://github.com/compulim) in PR [#3120](https://github.com/microsoft/BotFramework-WebChat/issues/3120)
+-  Fixes [#2668](https://github.com/microsoft/BotFramework-WebChat/issues/2668). Disable Web Audio on insecure connections, by [@compulim](https://github.com/compulim) in PR [#3079](https://github.com/microsoft/BotFramework-WebChat/issue/3079)
 
 ### Changed
 
@@ -102,11 +104,13 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
          -  [`core-js@3.6.4`](https://npmjs.com/package/core-js)
 -  Bumped Chrome Docker image to `3.141.59-zirconium` (Chrome 80.0.3987.106), by [@compulim](https://github.com/compulim) in PR [#2992](https://github.com/microsoft/BotFramework-WebChat/pull/2992)
 -  Added `4.8.0` to `embed/servicingPlan.json`, by [@compulim](https://github.com/compulim) in PR [#2986](https://github.com/microsoft/BotFramework-WebChat/pull/2986)
--  Bumped `microsoft-cognitiveservices-speech-sdk@1.10.1` and `web-speech-cognitive-services@6.1.0`, by [@compulim](https://github.com/compulim) in PR [#3040](https://github.com/BotFramework-WebChat/pull/3040)
+-  Bumped `microsoft-cognitiveservices-speech-sdk@1.10.1` and `web-speech-cognitive-services@6.1.0`, by [@compulim](https://github.com/compulim) in PR [#3040](https://github.com/microsoft/BotFramework-WebChat/pull/3040)
+-  Resolved [#2886](https://github.com/microsoft/BotFramework-WebChat/issues/2886) and [#2987](https://github.com/microsoft/BotFramework-WebChat/issue/2987), converged all references of [`microsoft-cognitiveservices-speech-sdk`](https://npmjs.com/package/microsoft-cognitiveservices-speech-sdk) to reduce footprint, by [@compulim](https://github.com/compulim) in PR [#3079](https://github.com/microsoft/BotFramework-WebChat/pull/3079)
 
 ## Samples
 
 -  Resolves [#2806](https://github.com/microsoft/BotFramework-WebChat/issues/2806), added [Single sign-on with On Behalf Of Token Authentication](https://webchat-sample-obo.azurewebsites.net/) sample, by [@tdurnford](https://github.com/tdurnford) in [#2865](https://github.com/microsoft/BotFramework-WebChat/pull/2865)
+-  Resolves [#2481](https://github.com/microsoft/BotFramework-WebChat/issues/2481), added selectable audio input device sample, by [@compulim](https://github.com/compulim) in PR [#3079](https://github.com/microsoft/BotFramework-WebChat/pull/3079)
 
 ## [4.8.0] - 2020-03-05
 

diff --git a/__tests__/__image_snapshots__/chrome-docker/video-js-video-1-snap.png b/__tests__/__image_snapshots__/chrome-docker/video-js-video-1-snap.png
diff --git a/...ization-token-with-direct-line-protocol-should-recognize-hello-world-1-snap.png b/...ization-token-with-direct-line-protocol-should-recognize-hello-world-1-snap.png
diff --git a/...-token-with-direct-line-speech-protocol-should-recognize-hello-world-1-snap.png b/...-token-with-direct-line-speech-protocol-should-recognize-hello-world-1-snap.png
diff --git a/...scription-key-with-direct-line-protocol-should-recognize-hello-world-1-snap.png b/...scription-key-with-direct-line-protocol-should-recognize-hello-world-1-snap.png
diff --git a/...on-key-with-direct-line-speech-protocol-should-recognize-hello-world-1-snap.png b/...on-key-with-direct-line-speech-protocol-should-recognize-hello-world-1-snap.png
diff --git a/__tests__/html/__jest__/fetchSpeechServicesAuthorizationToken.js b/__tests__/html/__jest__/fetchSpeechServicesAuthorizationToken.js
@@ -0,0 +1,24 @@
+import fetch from 'node-fetch';
+
+export default async function fetchSpeechServicesAuthorizationToken({ region, subscriptionKey, tokenURL }) {
+  if (!region && !tokenURL) {
+    throw new Error('Either "region" or "tokenURL" must be specified.');
+  } else if (region && tokenURL) {
+    throw new Error('Only either "region" or "tokenURL" can be specified.');
+  } else if (!subscriptionKey) {
+    throw new Error('"subscriptionKey" must be specified.');
+  }
+
+  const res = await fetch(tokenURL || `https://${region}.api.cognitive.microsoft.com/sts/v1.0/issueToken`, {
+    headers: {
+      'Ocp-Apim-Subscription-Key': subscriptionKey
+    },
+    method: 'POST'
+  });
+
+  if (!res.ok) {
+    throw new Error(`Failed to fetch authorization token, server returned ${res.status}`);
+  }
+
+  return await res.text();
+}
diff --git a/__tests__/html/__jest__/runPageProcessor.js b/__tests__/html/__jest__/runPageProcessor.js
@@ -1,5 +1,8 @@
 import { join } from 'path';
+import { promisify } from 'util';
+import { tmpdir } from 'os';
 import createDeferred from 'p-defer';
+import fs from 'fs';
 
 import { imageSnapshotOptions } from '../../constants.json';
 import createJobObservable from './createJobObservable';
@@ -9,6 +12,8 @@ const customImageSnapshotOptions = {
   customSnapshotsDir: join(__dirname, '../../__image_snapshots__/html')
 };
 
+const writeFile = promisify(fs.writeFile);
+
 export default async function runPageProcessor(driver, { ignoreConsoleError = false, ignorePageError = false } = {}) {
   const webChatLoaded = await driver.executeScript(() => !!window.WebChat);
   const webChatTestLoaded = await driver.executeScript(() => !!window.WebChatTest);
@@ -51,18 +56,25 @@ export default async function runPageProcessor(driver, { ignoreConsoleError = fa
     },
     next: async ({ deferred, job }) => {
       try {
+        let result;
+
         if (job.type === 'snapshot') {
-          try {
-            expect(await driver.takeScreenshot()).toMatchImageSnapshot(customImageSnapshotOptions);
-            deferred.resolve();
-          } catch (err) {
-            pageResultDeferred.reject(err);
-            deferred.reject(err);
-          }
+          expect(await driver.takeScreenshot()).toMatchImageSnapshot(customImageSnapshotOptions);
+        } else if (job.type === 'save file') {
+          const filename = join(tmpdir(), `${Date.now()}-${job.payload.filename}`);
+
+          await writeFile(filename, Buffer.from(job.payload.base64, 'base64'));
+
+          console.log(`Saved to ${filename}`);
+
+          result = filename;
         } else {
           throw new Error(`Unknown job type "${job.type}".`);
         }
+
+        deferred.resolve(result);
       } catch (err) {
+        pageResultDeferred.reject(err);
         deferred.reject(err);
       }
     }

diff --git a/__tests__/html/__jest__/setupRunHTMLTest.js b/__tests__/html/__jest__/setupRunHTMLTest.js
@@ -34,24 +34,22 @@ global.runHTMLTest = async (
   const params = parseURLParams(new URL(url, 'http://webchat2/').hash);
 
   try {
-    // For unknown reason, if we use ?wd=1, it will be removed.
-    // But when we use #wd=1, it kept.
+    // We are only parsing the "hash" from "url", the "http://localhost/" is actually ignored.
+    let { hash } = new URL(url, 'http://localhost/');
 
-    if (global.docker) {
-      params.wd = 1;
+    if (hash) {
+      hash += '&wd=1';
+    } else {
+      hash = '#wd=1';
     }
 
-    const baseURL = global.docker
-      ? new URL(url, 'http://webchat2/')
-      : new URL(url, `http://localhost:${global.webServerPort}/`);
-
-    const hash =
-      '#' +
-      Object.entries(params)
-        .map(([name, value]) => `${encodeURIComponent(name)}=${encodeURIComponent(value)}`)
-        .join('&');
-
-    await driver.get(new URL(hash, baseURL));
+    // For unknown reason, if we use ?wd=1, it will be removed.
+    // But when we use #wd=1, it kept.
+    await driver.get(
+      global.docker
+        ? new URL(hash, new URL(url, 'http://webchat2/'))
+        : new URL(url, `http://localhost:${global.webServerPort}/`)
+    );
 
     await runPageProcessor(driver, { ignoreConsoleError, ignorePageError });
 

diff --git a/__tests__/html/offlineUI.fatalError.html b/__tests__/html/offlineUI.fatalError.html
@@ -15,7 +15,7 @@
   <body>
     <div id="webchat"></div>
     <script crossorigin="anonymous" src="/__dist__/webchat-es5.js"></script>
-    <script type="text/babel" data-presets="es2015,stage-3">
+    <script type="text/babel" data-presets="env,stage-3">
       const { conditions, createStore, host, pageObjects, shareObservable, timeouts, token } = window.WebChatTest;
 
       (async function() {

diff --git a/__tests__/html/offlineUI.firstConnect.html b/__tests__/html/offlineUI.firstConnect.html
@@ -16,7 +16,7 @@
     <div id="webchat"></div>
     <script crossorigin="anonymous" src="https://unpkg.com/lolex@4/lolex.js"></script>
     <script crossorigin="anonymous" src="/__dist__/webchat-es5.js"></script>
-    <script type="text/babel" data-presets="es2015,stage-3">
+    <script type="text/babel" data-presets="env,stage-3">
       const { conditions, createStore, host, pageObjects, shareObservable, timeouts, token } = window.WebChatTest;
 
       (async function() {

diff --git a/__tests__/html/offlineUI.invalidCredentials.html b/__tests__/html/offlineUI.invalidCredentials.html
@@ -15,7 +15,7 @@
   <body>
     <div id="webchat"></div>
     <script crossorigin="anonymous" src="/__dist__/webchat-es5.js"></script>
-    <script type="text/babel" data-presets="es2015,stage-3">
+    <script type="text/babel" data-presets="env,stage-3">
       const { conditions, createStore, host, pageObjects, timeouts } = window.WebChatTest;
 
       (async function() {

diff --git a/__tests__/html/offlineUI.networkInterrupt.html b/__tests__/html/offlineUI.networkInterrupt.html
@@ -16,7 +16,7 @@
     <div id="webchat"></div>
     <script crossorigin="anonymous" src="https://unpkg.com/lolex@4/lolex.js"></script>
     <script crossorigin="anonymous" src="/__dist__/webchat-es5.js"></script>
-    <script type="text/babel" data-presets="es2015,stage-3">
+    <script type="text/babel" data-presets="env,stage-3">
       const { conditions, createStore, host, pageObjects, shareObservable, timeouts, token } = window.WebChatTest;
 
       (async function() {

diff --git a/__tests__/html/offlineUI.sendFailed.noAck.html b/__tests__/html/offlineUI.sendFailed.noAck.html
@@ -16,7 +16,7 @@
     <div id="webchat"></div>
     <script crossorigin="anonymous" src="https://unpkg.com/lolex@4/lolex.js"></script>
     <script crossorigin="anonymous" src="/__dist__/webchat-es5.js"></script>
-    <script type="text/babel" data-presets="es2015,stage-3">
+    <script type="text/babel" data-presets="env,stage-3">
       const { conditions, createStore, host, pageObjects, shareObservable, timeouts, token } = window.WebChatTest;
 
       (async function() {

diff --git a/__tests__/html/offlineUI.sendFailed.notSend.html b/__tests__/html/offlineUI.sendFailed.notSend.html
@@ -16,7 +16,7 @@
     <div id="webchat"></div>
     <script crossorigin="anonymous" src="https://unpkg.com/lolex@4/lolex.js"></script>
     <script crossorigin="anonymous" src="/__dist__/webchat-es5.js"></script>
-    <script type="text/babel" data-presets="es2015,stage-3">
+    <script type="text/babel" data-presets="env,stage-3">
       const { conditions, createStore, host, pageObjects, timeouts, token } = window.WebChatTest;
 
       (async function() {

diff --git a/__tests__/html/offlineUI.slowNetwork.firstConnect.html b/__tests__/html/offlineUI.slowNetwork.firstConnect.html
@@ -16,7 +16,7 @@
     <div id="webchat"></div>
     <script crossorigin="anonymous" src="https://unpkg.com/lolex@4/lolex.js"></script>
     <script crossorigin="anonymous" src="/__dist__/webchat-es5.js"></script>
-    <script type="text/babel" data-presets="es2015,stage-3">
+    <script type="text/babel" data-presets="env,stage-3">
       const { conditions, createStore, host, pageObjects, updateIn, timeouts, token } = window.WebChatTest;
 
       (async function() {

diff --git a/__tests__/html/offlineUI.slowNetwork.reconnect.html b/__tests__/html/offlineUI.slowNetwork.reconnect.html
@@ -16,7 +16,7 @@
     <div id="webchat"></div>
     <script crossorigin="anonymous" src="https://unpkg.com/lolex@4/lolex.js"></script>
     <script crossorigin="anonymous" src="/__dist__/webchat-es5.js"></script>
-    <script type="text/babel" data-presets="es2015,stage-3">
+    <script type="text/babel" data-presets="env,stage-3">
       const { conditions, createStore, host, pageObjects, shareObservable, timeouts, token } = window.WebChatTest;
 
       (async function() {