Conflict on gRPC methods with multiple Google Cloud libraries causes UNIMPLEMENTED

[buzzb0x]

Hello,

I am encountering a conflict between Google Cloud libraries and their gRPC connections when using them together. I first queue a task that will call another endpoint on the same server. This endpoint uses another Google Cloud library, for instance, Cloud Speech. This results in this error:

Error: 12 UNIMPLEMENTED: The GRPC target is not implemented on the server, host: cloudtasks.googleapis.com, method: /google.cloud.speech.v1.Speech/Recognize.

Do you see a reason why this conflict exists? I've tried closing the connection to the tasks client with Client.close() to no avail.

Here is a sample of my code, from how I queue the task to how the other Google Cloud library is called:

modules-config.js

import { CloudTasksClient } from '@google-cloud/tasks';
import Speech from '@google-cloud/speech';

import { creds } from '@project/config';

const googleCredentials = {
  projectId: creds.googleCloud.projectId,
  keyFilename: creds.googleCloud.keyFilename,
};

export const speech = () => new Speech.SpeechClient(googleCredentials);

export const tasks = () => {
  const client = new CloudTasksClient(googleCredentials);
  client.queue = client.queuePath(creds.googleCloud.projectId, 'us-central1', creds.googleCloud.queueName);
  return client;
};

lib.js

import { tasks } from '@project/modules-config';
import config from '@project/config';

// [•••]

export const createHTTPTask = async (name, payload) => {
  const tasksClient = tasks();
  await tasksClient.createTask({
    parent: tasksClient.queue,
    task: {
      httpRequest: {
        url: `${config.rootUrl}/tasks/${name}`,
        body: Buffer.from(JSON.stringify(payload)).toString('base64'),
        'POST',
        headers: {
          'Content-Type': 'application/json',
        },
        oidcToken: {
          serviceAccountEmail: config.credentials.googleCloud.serviceAccountEmail,
        },
      },
    },
  });
  await tasksClient.close();
};

webhook.js

import Router from 'express-promise-router';

import { createHTTPTask } from '@project/lib;

const webhook = async (req, res) => {
  const { data } = res.locals.params;

  await createHTTPTask('transcribe', { ...data });

  return res.success();
};

const endpoints = Router({ mergeParams: true });
endpoints.post('/webhook', webhook);
export default endpoints;

tasks.js

import { googleAuthentifiedRouter } from '@project/lib';
import { speech } from '@project/modules-config';
import { stt } from '@project/config';

const transcribe = async (req, res) => {
  const { data } = res.local.params;

  const speechClient = speech();
  const response = await speechClient.recognize({ audio: { data.uri }, config: stt });
  // •••
}

const endpoints = Router({ mergeParams: true });
endpoints.post('/transcribe', transcribe);

Environment details

• OS: macOS 10.15
• Node.js version: v10.17.0
• yarn version: 1.22.4
• @google-cloud/tasks version: 2.0.1

Hoping my issue is complete and you'll be able to enlighten me. Don't hesitate if anything's missing.

Cheers!

[alexander-fenster]

Hi @buzzb0x,

Sorry for the late response, it took some time to route this issue.

This should not happen. The service path (e.g. cloudtasks.googleapis.com or speech.googleapis.com) is hardcoded inside each @google-cloud/ client library (e.g. here and here). We do have use cases where two or more client libraries are used from the same process, and I haven't seen such a conflict.

I can imagine a few things that could go wrong in your case:

1. If, by chance, an option (servicePath or apiEndpoint) is passed to speech client constructor (e.g. if it somehow got into googleCredentials object in your example), it will override the hardcoded value.

2. Some weird DNS setup might contribute.

If you're willing to debug it, may I ask you to add some logging right into the node_modules folder: in both node_modules/@google-cloud/speech/build/src/v1/speech_client.js and node_modules/@google-cloud/tasks/build/src/v2/cloud_tasks_client.js, please console.log the value of opts before new this._gaxModule.GrpcClient(opts):

/* add logging: */ console.log(opts);
this._gaxGrpc = new this._gaxModule.GrpcClient(opts);

It should print the endpoints that it will connect to, and should be speech.googleapis.com for speech and cloudtasks.googleapis.com for tasks.

[buzzb0x]

Thanks for the answer, even if pretty late. I got around the issue by directly creating my instances where I need them. I'm going to test this when I have some time and report the results – might not be until next week.

Cheers

[bcoe]

@buzzb0x did @alexander-fenster's tips help you debug the issue?

[buzzb0x]

Haven't gotten around to testing their tips but will do so this week for sure!

[buzzb0x]

Hello,

I've been caught up in a lot of stuff and haven't gotten to testing since @alexander-fenster's tip last time. I've been working again on this part of our API and the issue is still here. Cloud Speech's GRPC client really gets initialized with Cloud Tasks' endpoint:

11/10/2020, 5:51:29 PM info: /twilio/record
GOOGLE CLOUD TASKS OPTS { projectId: 'project-samantha',
  keyFilename: 'config/google_credentials.local.json',
  servicePath: 'cloudtasks.googleapis.com',
  port: 443,
  clientConfig: {},
  scopes: [ 'https://www.googleapis.com/auth/cloud-platform' ] }
11/10/2020, 5:51:35 PM info: /tasks/recording
GOOGLE CLOUD SPEECH OPTS { projectId: 'project-samantha',
  keyFilename: 'config/google_credentials.local.json',
  servicePath: 'cloudtasks.googleapis.com',
  port: 443,
  clientConfig: {},
  scopes: [ 'https://www.googleapis.com/auth/cloud-platform' ] }
11/10/2020, 5:51:35 PM error: [Bugsnag] 12 UNIMPLEMENTED: The GRPC target is not implemented on the server, host: cloudtasks.googleapis.com, method: /google.cloud.speech.v1.Speech/Recognize.

---

EDIT:

You were right, googleCredentials does get modified from Tasks to Speech. Here's a result of my logging:

GOOGLE CREDS BEFORE TASKS { projectId: 'project-samantha',
  keyFilename: 'config/google_credentials.local.json' }
GOOGLE CREDS BEFORE SPEECH { projectId: 'project-samantha',
  keyFilename: 'config/google_credentials.local.json',
  servicePath: 'cloudtasks.googleapis.com',
  port: 443,
  clientConfig: {},
  scopes: [ 'https://www.googleapis.com/auth/cloud-platform' ] }

Why is it, will you fix it, and what's the best way to prevent this until then?

In the meantime, here's how I now initialize my Google modules:

const speech = new Speech.SpeechClient({ ...googleCredentials });

[alexander-fenster]

Hi @buzzb0x, this is fixed in #484 (and similar PRs across all libraries) but not yet released. Apparently, we modified the options object without making a proper copy. Whenever this is released to npm, it will start working for you.