Two keys in /.well-known/jwks.json ?

[mig5]

Hi,

I've just deployed ory/hydra:1.10.7 across 4 Docker environments (think of these as dev, stage, test, prod).

As far as I can tell, the environments are more or less identical. The test and prod environments are running 2 copies of the Docker image, for H/A reasons. dev/stage run just one each.

I have not issued any oAuth2 keys yet in these environments.

Every environment has got these Hydra environment variables set (obviously I'm just showing the names here):

            "name": "SECRETS_SYSTEM",
            "name": "DSN",
            "name": "URLS_SELF_ISSUER",
            "name": "URLS_CONSENT",
            "name": "URLS_LOGIN",
            "name": "URLS_LOGOUT",
            "name": "SERVE_ADMIN_TLS_ALLOW_TERMINATION_FROM",
            "name": "SERVE_TLS_ALLOW_TERMINATION_FROM",
            "name": "SERVE_PUBLIC_TLS_ALLOW_TERMINATION_FROM",
            "name": "OIDC_SUBJECT_IDENTIFIERS_SUPPORTED_TYPES",
            "name": "OIDC_SUBJECT_IDENTIFIERS_PAIRWISE_SALT",
            "name": "LOG_LEAK_SENSITIVE_VALUES",
            "name": "SQA_OPT_OUT",

But for some reason, just one environment (the test environment) has two JWKs at /.well-known/jwks.json, and I don't understand this difference. All the other three environments (dev, stage and prod) have just one key at this URL.

I didn't manually create any keys in any of the environments - whatever's there has been added automatically by Hydra during provisioning.

Can anyone explain what might cause this second key in just one environment? Is it a problem?

Does the key get generated on the first HTTP request to Hydra? My only theory was it was the loadbalancer's very first health checks to the two containers when they came up, which might've occurred at exactly the same time. But, I'm surprised that that didn't happen in the prod environment as well, unless it's pure chance..

[mig5]

I dropped the database, recreated and re-migrated (seeded) it, and then stopped my containers to let them respawn... and now I have just the one key, which is more like what I'd expect.

I saw this in the logs of one of the containers:

time=2022-01-18T02:17:29Z level=warning msg=JSON Web Key Set "hydra.openid.id-token" does not exist yet, generating new key pair... audience=application service_name=ORY Hydra service_version=v1.10.7
time=2022-01-18T02:17:31Z level=warning msg=JSON Web Key Set "hydra.jwt.access-token" does not exist yet, generating new key pair... audience=application service_name=ORY Hydra service_version=v1.10.7
[...]
time=2022-01-18T02:17:35Z level=warning msg=JSON Web Key Set "hydra.https-tls" does not exist yet, generating new key pair... audience=application service_name=ORY Hydra service_version=v1.10.7

The other container only had the https-tls one:

time=2022-01-18T02:17:38Z level=warning msg=JSON Web Key Set "hydra.https-tls" does not exist yet, generating new key pair... audience=application service_name=ORY Hydra service_version=v1.10.7

My theory is that both containers both generated keys at the same time, the first time they came online. A purely-by-chance timing thing.

Would it be a problem? And would it be a problem that there might be two https-tls keys? Or am I worrying about nothing? :)

[aeneasr]

This shouldn't be a problem! This can happen when two workers start on an empty database. So we recommend to use one instance for the initial set up of the database and then later on have multiple / HA set up :) This is only relevant for the first time when you install the database.

[mig5]

Thanks, that's a good idea! Maybe it can be a tip in the Preparing for Production page, I might send a PR if you agree :)

[aeneasr]

Please do, that would be awesome! :)

[devshrm]

@aeneasr isn't it expected to get two public keys in .well-known/jwks.json : one for jwt and another for openid