Firebase environment variable deployment error

[Neofield-Dylan]

[REQUIRED] Environment info

firebase-tools: 12.2.1

Platform: Ubuntu 20.04.6 LTS

Node: 20.11.1

firebase-admin: ^12.0.0
firebase-functions: ^4.7.0
typescript: ^5.3.3

cloud functions 2nd gen

[REQUIRED] Test case

import { onSchedule } from 'firebase-functions/v2/scheduler';
import { defineString } from 'firebase-functions/params';

exports.testeCronjob = onSchedule(
  {
    schedule: defineString(`MY_CRONTAB`),
    vpcConnector: defineString(`MY_VPC_CONNECTOR_NAME`),
    vpcConnectorEgressSettings: defineString(`MY_VPC_CONNECTOR_EGRESS`),
  },
  myCronjobFunction,
);

[REQUIRED] Steps to reproduce

configure vpc connector with Parameterized configuration to set environment variable to set vpc connector egress settings.
And try to deploy.

[REQUIRED] Expected behavior

• No Typescrit error when trying to pass StringParam as input to ScheduleOptions.
• Load environment variables from .env.projectId
• No deployment errors or warnings

[REQUIRED] Actual behavior

Error with several warning.

{"severity":"WARNING","message":"params.GLOBAL_ENVIRONMENT.value() invoked during function deployment, instead of during runtime."}
{"severity":"WARNING","message":"This is usually a mistake. In configs, use Params directly without calling .value()."}
{"severity":"WARNING","message":"example: { memory: memoryParam } not { memory: memoryParam.value() }"}
{"severity":"WARNING","message":"params.GLOBAL_ENVIRONMENT.value() invoked during function deployment, instead of during runtime."}
{"severity":"WARNING","message":"This is usually a mistake. In configs, use Params directly without calling .value()."}
{"severity":"WARNING","message":"example: { memory: memoryParam } not { memory: memoryParam.value() }"}
{"severity":"WARNING","message":"params.TEST_API_URL.value() invoked during function deployment, instead of during runtime."}
{"severity":"WARNING","message":"This is usually a mistake. In configs, use Params directly without calling .value()."}
{"severity":"WARNING","message":"example: { memory: memoryParam } not { memory: memoryParam.value() }"}
Error: Failed to parse build specification:
- FirebaseError string endpoints[testeCronjob].vpc.egressSettings failed validation

[aalej]

Hey @Neofield-Dylan, thanks for sharing code snippets and the logs you encountered. I’m trying to reproduce this locally, but I can’t seem to get the correct setup. A couple of TypeScript errors are being raised after copying the provided code. Also, looking through the logs, it seems like you’re using a couple of other parameterized configurations.

By any chance, could you please provide an MCVE where the issue is reproducible, or additional steps on how you set up your project so that we can replicate the issue on our end?

Also, it looks like you're using an older version of firebase-tools. I’d recommend trying to upgrade to the latest version to see if that might fix the issue.

[Neofield-Dylan]

Hi, aalej!
Thanks for your response.
I already tried to run with firecast-tools v13.4.0 but still got the same error.

The first problem is onScheduleOptions does not accept StringParam type on vpcConnector and vpcConnectorEgressSettings as you described there is typescript errors when it shouldn't. Because the firebase documentation advice us to pass the parameters as StringParam which implies that every function parameter should accept one of Param types.

The second problem is even if I use some typescript trick to workaround like putting as unknown as string or as unknown as VpcEgressSetting to cast the parameters to acceptable types. I get the mentioned Firebase error when deploying.

I can work in a MCVE, but it may take some time to do so, because I have other things to do.
But the main idea is to try to implement a cronjob with cloud functions v2 (onSchedule) and set a VPC access settings (specially vpcConnectorEgressSettings) with parameterized configurations.

I think you does not need to worry with the others parameterized configuration that I used (which you mentioned you saw on the logs), because after I set the vpcConnectorEgressSettings with a hardcoded string like vpcConnectorEgressSettings : 'PRIVATE_RANGES_ONLY', I could deploy without any problem which makes me think that the problem is there.

[inlined]

It looks like there are two kinds of errors you're encountering:

1. You're calling myParam.value at global scope. When we load your code locally to understand it, those parameters haven't yet been set, so they can't be read. We have an imminent feature coming out with an onInit callback where you can safely add global initialization that will only run in production where .value works.

2. vpcConnector supports string parameters, but vpcConnectorEgressSettings does not because it's an enum. I'll check with the team to see what appetite there is for making this parameterizeable even if it may require changes to firebase-tools to do additional enforcement.

[inlined]

firebase/firebase-functions#1531 for the init function, which should be released soon

[inlined]

Download the latest version of the Firebase tools with npm install --global firebase-tools@latest to support parameterizing vpc egress settings. Download the latest version of the firebase-functions SDK by running npm install --save firebase-functions@latest in your functions directory to support the onInit function. Inside the onInit callback you can safely call param.value. For example:

import { defineSecret } from 'firebase-functions/params';
import { onInit } from 'firebase-functions/v2/core';
import { SdkClass } from "someAPI";

const apiKey = defineSecret("API_KEY");

let sdk: SdkClass;
onInit(() => {
  sdk = new SdkClass(apiKey.value);
});

[florin-bc]

@inlined I've been trying to use the onInit hook locally, on the functions emulator, to initialize the sentry client and aws s3 client globally, but the onInit hooks seems to be called more than once and when I try to import and use the clients/variables in other files, they're undefined. What am I doing wrong?

// env-variables.js

const s3PrivateSecret = defineSecret('S3_PRIVATE_SECRET');
const s3PrivateId = defineSecret('S3_PRIVATE_ID');
const s3PrivateBucket = defineString('S3_PRIVATE_BUCKET');
const s3PrivateLinkLifetime = defineInt('S3_PRIVATE_LINK_LIFETIME');

const sentryEnvironment = defineString('SENTRY_ENVIRONMENT');
const sentryDsn = defineSecret('SENTRY_DSN');

module.exports = {
  sentryEnvironment,
  sentryDsn,
  s3PrivateSecret,
  s3PrivateId,
  s3PrivateBucket,
  s3PrivateLinkLifetime,
};



// global.js

const Sentry = require('@sentry/serverless');
const { sentryEnvironment, sentryDsn, s3PrivateId, s3PrivateSecret } = require('../env-variables');
const { onInit } = require('firebase-functions/v2/core');
const { S3Client } = require('@aws-sdk/client-s3');

const sentryBeforeSendEvent = (event, hint) => {
  if (event.level !== 'fatal' && event.level !== 'error') {
    return null;
  }
  return event;
};

const processSentryOptions = (options) => {
  const { dsn, environment } = options;
  return {
    dsn,
    environment,
    tracesSampleRate: 1.0,
    beforeSend: sentryBeforeSendEvent,
    includeLocalVariables: true,
    integrations: [Sentry.sessionTimingIntegration()],
    // ignoreErrors: [],
    // initialScope: null,
    // enabled: false,
  };
};

let sentryOptions;
let s3Client;

onInit(() => {
  console.log('>>> Global inside onInit'); // gets called
  sentryOptions = processSentryOptions({ dsn: sentryDsn.value(), environment: sentryEnvironment.value() });
  s3Client = new S3Client({
    credentials: {
      accessKeyId: s3PrivateId.value(),
      secretAccessKey: s3PrivateSecret.value(),
    },
    region: 'us-east-1',
  });
  console.log('>>> Global onInit sentryOptions', sentryOptions);  // ok here
  console.log('>>> Global onInit s3Client', s3Client); // ok here
});

module.exports = {
  sentryOptions,
  s3Client,
};

Usage example for aws s3:

// aws.js

const { s3PrivateLinkLifetime, s3PrivateBucket } = require('../env-variables');
const { s3Client } = require('./global');

exports.createContentUrl = async (path) => {
  // Grant access to the object for the next X hours.
  const signedUrlExpireSeconds = constants.hourInSeconds * s3PrivateLinkLifetime.value();

  const expiresAt = new Date().valueOf() + signedUrlExpireSeconds * 1000;

  const command = new GetObjectCommand({
    Bucket: s3PrivateBucket.value(),
    Key: path,
  });
  console.log('>>> [createContentUrl] s3Client', s3Client); // // undefined here
  const url = await getSignedUrl(s3Client, command, {
    expiresIn: signedUrlExpireSeconds,
  });

  return { url, expiresAt };
};




// createContentUrl is called within a onCall cloud function

exports.cloudFunction = onCall(async ({data}) => {
  try {
    if (!data?.id) {
      return errors.projectError('missingBodyArgument', 'The id property is missing.');
    }
    return {
      object: await utils.createContentUrl(data?.path),
    };
  } catch (err) {
    return errors.projectError(err);
  }
});

Usage example for sentry:

// sentry.js

const Sentry = require('@sentry/serverless');
const { sentryOptions } = require('./global');

console.log('>>> [sentry.js] sentryOptions', sentryOptions); // undefined here
Sentry.GCPFunction.init(sentryOptions);

module.exports = Sentry;

// Sentry client is used within cloud functions as follows
const Sentry = require('../services/sentry');

exports.cloudFunction = onCall(async ({data}) => {
  try {
    // something
  } catch (err) {
    Sentry.captureException(err, { level: 'error' });
    return errors.projectError(err);
  }
});

Dependencies:

node: 18.16.1
firebase-tools: 13.13.0

"firebase": "9.6.8",
"firebase-admin": "12.1.1",
"firebase-functions": "5.0.1",
"@aws-sdk/client-s3": "^3.609.0",
"@aws-sdk/s3-request-presigner": "^3.609.0",
"@sentry/node": "7.76.0",
"@sentry/serverless": "7.114.0",