Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dep: bump traefik from v2.3.7 to v2.4.1 #2001

Merged

Conversation

consideRatio
Copy link
Member

@consideRatio consideRatio commented Jan 19, 2021

This seem to work, but as I've grown cautious about Traefik's ability to reliably work as an ACME client against Let's encrypt I'm restarting the tests several times to see if there is a obvious degradation in reliability.

@consideRatio
Copy link
Member Author

consideRatio commented Jan 20, 2021

k8s 1.19 run failure 1

Pebble logs (ACME server)

Pebble 2021/01/20 17:35:07 ACME directory available at: https://:8443/dir
Pebble 2021/01/20 17:36:09 GET /dir -> calling handler()
Pebble 2021/01/20 17:36:09 HEAD /nonce-plz -> calling handler()
Pebble 2021/01/20 17:36:09 POST /sign-me-up -> calling handler()
Pebble 2021/01/20 17:36:09 There are now 1 accounts in memory
Pebble 2021/01/20 17:36:09 POST /order-plz -> calling handler()
Pebble 2021/01/20 17:36:09 There are now 1 authorizations in the db
Pebble 2021/01/20 17:36:09 Added order "yVhDh-EfnTVAiqAlxjNee8H3rwKO4X5ayasZnpwI0wY" to the db
Pebble 2021/01/20 17:36:09 There are now 1 orders in the db
Pebble 2021/01/20 17:36:09 POST /authZ/ -> calling handler()
Pebble 2021/01/20 17:36:09 POST /chalZ/ -> calling handler()
Pebble 2021/01/20 17:36:09 Pulled a task from the Tasks queue: &va.vaTask{Identifier:acme.Identifier{Type:"dns", Value:"local.jovyan.org"}, Challenge:(*core.Challenge)(0xc00008abe0), Account:(*core.Account)(0xc0000e8900)}
Pebble 2021/01/20 17:36:09 Starting 3 validations.
Pebble 2021/01/20 17:36:09 Attempting to validate w/ HTTP: http://local.jovyan.org:80/.well-known/acme-challenge/vD3mbW1cdgc0O3j3rz4KbexOcDfqJi1Sy_xPvUvjpA8
Pebble 2021/01/20 17:36:09 Attempting to validate w/ HTTP: http://local.jovyan.org:80/.well-known/acme-challenge/vD3mbW1cdgc0O3j3rz4KbexOcDfqJi1Sy_xPvUvjpA8
Pebble 2021/01/20 17:36:09 Attempting to validate w/ HTTP: http://local.jovyan.org:80/.well-known/acme-challenge/vD3mbW1cdgc0O3j3rz4KbexOcDfqJi1Sy_xPvUvjpA8
Pebble 2021/01/20 17:36:09 POST /authZ/ -> calling handler()
Pebble 2021/01/20 17:36:09 authz TQFlBqUT-Da0iCDBQp-Ru-F8dvT2JFgjdSBZ19fHdCo set INVALID by completed challenge I5Gj4EZMmCdTb-N8cLfyvXiGp5wWXadV9_88Cb6BX-g
Pebble 2021/01/20 17:36:09 order yVhDh-EfnTVAiqAlxjNee8H3rwKO4X5ayasZnpwI0wY set INVALID by invalid authz TQFlBqUT-Da0iCDBQp-Ru-F8dvT2JFgjdSBZ19fHdCo
Pebble 2021/01/20 17:36:15 POST /authZ/ -> calling handler()
Pebble 2021/01/20 17:36:15 POST /authZ/ -> calling handler()
Pebble 2021/01/20 17:36:15 POST /authZ/ -> calling handler()

Traefik logs

time="2021-01-20T17:36:09Z" level=debug msg="Building ACME client..." providerName=default.acme
time="2021-01-20T17:36:09Z" level=debug msg="https://pebble/dir" providerName=default.acme
time="2021-01-20T17:36:09Z" level=info msg=Register... providerName=default.acme
time="2021-01-20T17:36:09Z" level=debug msg="legolog: [INFO] acme: Registering account for jovyan@jupyter.test"
time="2021-01-20T17:36:09Z" level=debug msg="Using HTTP Challenge provider." providerName=default.acme
time="2021-01-20T17:36:09Z" level=debug msg="legolog: [INFO] [local.jovyan.org] acme: Obtaining bundled SAN certificate"
time="2021-01-20T17:36:09Z" level=debug msg="legolog: [INFO] [local.jovyan.org] AuthURL: https://pebble/authZ/TQFlBqUT-Da0iCDBQp-Ru-F8dvT2JFgjdSBZ19fHdCo"
time="2021-01-20T17:36:09Z" level=debug msg="legolog: [INFO] [local.jovyan.org] acme: Could not find solver for: tls-alpn-01"
time="2021-01-20T17:36:09Z" level=debug msg="legolog: [INFO] [local.jovyan.org] acme: use http-01 solver"
time="2021-01-20T17:36:09Z" level=debug msg="legolog: [INFO] [local.jovyan.org] acme: Trying to solve HTTP-01"
time="2021-01-20T17:36:09Z" level=debug msg="No default certificate, generating one"
time="2021-01-20T17:36:09Z" level=warning msg="No domain found in rule PathPrefix(`/`), the TLS options applied for this router will depend on the hostSNI of each request" entryPointName=https routerName=default@file
time="2021-01-20T17:36:09Z" level=debug msg="Looking for provided certificate(s) to validate [\"local.jovyan.org\"]..." providerName=default.acme
time="2021-01-20T17:36:09Z" level=debug msg="No ACME certificate generation required for domains [\"local.jovyan.org\"]." providerName=default.acme
time="2021-01-20T17:36:15Z" level=debug msg="legolog: [INFO] Deactivating auth: https://pebble/authZ/TQFlBqUT-Da0iCDBQp-Ru-F8dvT2JFgjdSBZ19fHdCo"
time="2021-01-20T17:36:15Z" level=error msg="Unable to obtain ACME certificate for domains \"local.jovyan.org\" : unable to generate a certificate for the domains [local.jovyan.org]: error: one or more domains had a problem:\n[local.jovyan.org] acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: Non-200 status code from HTTP: http://local.jovyan.org:80/.well-known/acme-challenge/vD3mbW1cdgc0O3j3rz4KbexOcDfqJi1Sy_xPvUvjpA8 returned 404, url: \n" providerName=default.acme

k8s 1.19 run failure 2

Pebble


Traefik logs

time="2021-01-20T18:00:14Z" level=debug msg="Building ACME client..." providerName=default.acme
time="2021-01-20T18:00:14Z" level=debug msg="https://pebble/dir" providerName=default.acme
time="2021-01-20T18:00:14Z" level=info msg=Register... providerName=default.acme
time="2021-01-20T18:00:14Z" level=debug msg="legolog: [INFO] acme: Registering account for jovyan@jupyter.test"
time="2021-01-20T18:00:14Z" level=debug msg="Using HTTP Challenge provider." providerName=default.acme
time="2021-01-20T18:00:14Z" level=debug msg="legolog: [INFO] [local.jovyan.org] acme: Obtaining bundled SAN certificate"
time="2021-01-20T18:00:14Z" level=debug msg="legolog: [INFO] [local.jovyan.org] AuthURL: https://pebble/authZ/eNaFmN1OBnu3VJuD9LgCTBzyCWnuIqDaTj2V0Ae0jyE"
time="2021-01-20T18:00:14Z" level=debug msg="legolog: [INFO] [local.jovyan.org] acme: Could not find solver for: tls-alpn-01"
time="2021-01-20T18:00:14Z" level=debug msg="legolog: [INFO] [local.jovyan.org] acme: use http-01 solver"
time="2021-01-20T18:00:14Z" level=debug msg="legolog: [INFO] [local.jovyan.org] acme: Trying to solve HTTP-01"
time="2021-01-20T18:00:14Z" level=warning msg="No domain found in rule PathPrefix(`/`), the TLS options applied for this router will depend on the hostSNI of each request" entryPointName=https routerName=default@file
time="2021-01-20T18:00:14Z" level=debug msg="Looking for provided certificate(s) to validate [\"local.jovyan.org\"]..." providerName=default.acme
time="2021-01-20T18:00:14Z" level=debug msg="No ACME certificate generation required for domains [\"local.jovyan.org\"]." providerName=default.acme
time="2021-01-20T18:00:20Z" level=debug msg="legolog: [INFO] Deactivating auth: https://pebble/authZ/eNaFmN1OBnu3VJuD9LgCTBzyCWnuIqDaTj2V0Ae0jyE"
time="2021-01-20T18:00:20Z" level=error msg="Unable to obtain ACME certificate for domains \"local.jovyan.org\" : unable to generate a certificate for the domains [local.jovyan.org]: error: one or more domains had a problem:\n[local.jovyan.org] acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: Non-200 status code from HTTP: http://local.jovyan.org:80/.well-known/acme-challenge/xaGRJM1wykJlFiEKvFBBLiNJmrvdLmaA6rYf_eAIL48 returned 404, url: \n" providerName=default.acme

k8s 1.18 run failure 3

@consideRatio
Copy link
Member Author

I reported the experienced intermittency failures in traefik/traefik#7784.

@consideRatio consideRatio marked this pull request as draft January 20, 2021 19:00
@consideRatio consideRatio force-pushed the pr/bump-traefik-v2.4.0 branch 3 times, most recently from 895bce2 to 093f31a Compare January 28, 2021 04:00
With this, I think our traefik will reliably acquire certificates
finally!
@consideRatio consideRatio changed the title dep: bump traefik from v2.3.7 to v2.4.0 dep: bump traefik from v2.3.7 to v2.4.1 Feb 2, 2021
@consideRatio consideRatio marked this pull request as ready for review February 2, 2021 13:03
@consideRatio
Copy link
Member Author

The ACME issues are resolved in v2.4.1 ❤️

@consideRatio consideRatio merged commit 9cc9b31 into jupyterhub:master Feb 2, 2021
consideRatio pushed a commit to jupyterhub/helm-chart that referenced this pull request Feb 2, 2021
jupyterhub/zero-to-jupyterhub-k8s#2001 Merge pull request #2001 from consideRatio/pr/bump-traefik-v2.4.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant