add envFrom

[garrett361]

Issue link

What changes have been made

Adds an optional envFrom field, making it easier to read in environment variables from secret files.

Example:

1. Create a secrets.yaml:

apiVersion: v1
kind: Secret
metadata:
  name: my-secrets
  namespace: my-namespace
type: Opaque
stringData:
  SECRET_ENV_0: my_api_key
  SECRET_ENV_1: another_api_key

2. oc create -f secrets.yaml to create the open shift secrets
3. Add an envFrom section to your job yaml:

jobName: my-job

[...]

envFrom:
 - secretRef:
    name: my-secrets

4. Launch a job: both SECRET_ENV_{0,1} will be set in all pods.

Verification steps

Tested by running through the above workflow one one- and two-node jobs, verifying that the env vars are set in all containers.

Checks

• I've made sure the tests are passing.
• Testing Strategy
  • Unit tests
  • Manual tests
  • Testing is not required for this change

[garrett361]

I don't really understand why pre-commit run --all-files wants to entirely remove my description of envFrom from the README. I've iterated a few times, but it is always removed. Could use some help there.

[dgrove-oss]

Thanks for the addition. I agree this is a useful shortcut if you want to load all the entries in a Secret or ConfigMap into the Pod's environment (the current support via enviornmentVariables forces you to list each entry in the Secret/ConfigMap).

The README.md file gets generated from the comments in values.yaml. So you'll need to add something similar to how environmentVariables is documented.

We've had enough problems with users defining both topologyFileConfigMap and a NCCL_TOPO_FILE environment variable that we added an explicit check for that here. We can't do that check if we're bulk loading an entire ConfigMap or Secret. This is fine, but it might be nice to include in the documentation that the user has to be sure they aren't making this mistake if they use envFrom.

[garrett361]

Updated @dgrove-oss

[dgrove-oss]

lgtm; thanks!