GoDaddy - Stuck in prop #1519

rsliotta · 2021-11-01T12:16:08Z

rsliotta
Nov 1, 2021

Welcome

Yes, I'm using a binary release within 2 latest releases.
Yes, I've searched similar issues on GitHub and didn't find any.
Yes, I've included all information below (version, config, etc).

What did you expect to see?

Propagation complete.

What did you see instead?

Propogation fails on SERVFAIL.

Deletes are now failing on GoDaddy do to some shema body error.

How do you use lego?

Through Traefik

Reproduction steps

Setup as per instructions. See log..... for error message.

Version of lego

Traefik 2.5.3

Logs

time="2021-11-01T11:37:47Z" level=debug msg="legolog: [INFO] [traefik-ui.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:37:47Z" level=debug msg="legolog: [INFO] [velero.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:38:17Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:38:17Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:38:19Z" level=debug msg="Send instance info to pilot: 2021-11-01 11:38:19.406039313 +0000 UTC m=+600.050396923"
time="2021-11-01T11:38:19Z" level=info msg="Anonymous stats sent to https://collect.traefik.io/9vxmmkcdmalbdi635d4jgc5p5rx0h7h8: {\"global\":{\"checkNewVersion\":true,\"sendAnonymousUsage\":true},\"serversTransport\":{\"maxIdleConnsPerHost\":200},\"entryPoints\":{\"metrics\":{\"address\":\"xxxx\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{}},\"traefik\":{\"address\":\"xxxx\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{}},\"web\":{\"address\":\"xxxx\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{}},\"websecure\":{\"address\":\"xxxx\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{}}},\"providers\":{\"providersThrottleDuration\":\"2s\",\"kubernetesIngress\":{},\"kubernetesCRD\":{}},\"api\":{\"dashboard\":true},\"metrics\":{\"prometheus\":{\"buckets\":[0.1,0.3,1.2,5],\"addEntryPointsLabels\":true,\"addServicesLabels\":true,\"entryPoint\":\"metrics\"}},\"ping\":{\"entryPoint\":\"traefik\",\"terminatingStatusCode\":503},\"log\":{\"level\":\"DEBUG\",\"format\":\"common\"},\"certificatesResolvers\":{\"boblife\":{\"acme\":{\"email\":\"xxxx\",\"caServer\":\"xxxx\",\"storage\":\"/tmp/tls-acme.json\",\"keyType\":\"RSA4096\",\"httpChallenge\":{\"entryPoint\":\"web\"}}},\"godaddy\":{\"acme\":{\"email\":\"xxxx\",\"caServer\":\"xxxx\",\"storage\":\"/tmp/acme.json\",\"keyType\":\"RSA4096\",\"dnsChallenge\":{\"provider\":\"godaddy\",\"delayBeforeCheck\":\"5s\",\"resolvers\":[\"xxxx\",\"xxxx\"]}}}},\"pilot\":{\"token\":\"xxxx\"}}"
time="2021-11-01T11:38:19Z" level=debug msg="unknown kind to hash: func"
time="2021-11-01T11:38:22Z" level=debug msg="legolog: [INFO] [traefik-ui.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:38:22Z" level=debug msg="legolog: [INFO] [velero.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:38:52Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:38:52Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:38:57Z" level=debug msg="legolog: [INFO] [velero.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:38:57Z" level=debug msg="legolog: [INFO] [traefik-ui.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:39:27Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:39:27Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:39:32Z" level=debug msg="legolog: [INFO] [velero.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:39:32Z" level=debug msg="legolog: [INFO] [traefik-ui.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:40:02Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:40:02Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:40:07Z" level=debug msg="legolog: [INFO] [traefik-ui.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:40:07Z" level=debug msg="legolog: [INFO] [velero.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:40:37Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:40:37Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:40:43Z" level=debug msg="legolog: [INFO] [velero.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:40:43Z" level=debug msg="legolog: [INFO] [traefik-ui.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:41:13Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:41:13Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:41:18Z" level=debug msg="legolog: [INFO] [velero.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:41:18Z" level=debug msg="legolog: [INFO] [traefik-ui.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:41:48Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:41:48Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:41:53Z" level=debug msg="legolog: [INFO] [velero.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:41:53Z" level=debug msg="legolog: [INFO] [traefik-ui.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:42:23Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:42:23Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:42:28Z" level=debug msg="legolog: [INFO] [traefik-ui.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:42:28Z" level=debug msg="legolog: [INFO] [velero.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:42:58Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:42:58Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:43:03Z" level=debug msg="legolog: [INFO] [traefik-ui.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:43:03Z" level=debug msg="legolog: [INFO] [velero.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:43:19Z" level=debug msg="Send instance info to pilot: 2021-11-01 11:43:19.406640227 +0000 UTC m=+900.050997838"
time="2021-11-01T11:43:33Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:43:33Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:43:38Z" level=debug msg="legolog: [INFO] [velero.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:43:38Z" level=debug msg="legolog: [INFO] [traefik-ui.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:44:08Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:44:08Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:44:13Z" level=debug msg="legolog: [INFO] [traefik-ui.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:44:13Z" level=debug msg="legolog: [INFO] [velero.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:44:43Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:44:43Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:44:49Z" level=debug msg="legolog: [INFO] [velero.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:44:49Z" level=debug msg="legolog: [INFO] [traefik-ui.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:45:19Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:45:19Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:45:24Z" level=debug msg="legolog: [INFO] [traefik-ui.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:45:24Z" level=debug msg="legolog: [INFO] [velero.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:45:54Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:45:54Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:45:59Z" level=debug msg="legolog: [INFO] [velero.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:45:59Z" level=debug msg="legolog: [INFO] [traefik-ui.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:46:29Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:46:29Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:46:34Z" level=debug msg="legolog: [INFO] [traefik-ui.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:46:34Z" level=debug msg="legolog: [INFO] [velero.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:47:04Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:47:04Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:47:09Z" level=debug msg="legolog: [INFO] [velero.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:47:09Z" level=debug msg="legolog: [INFO] [traefik-ui.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:47:39Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:47:39Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:47:44Z" level=debug msg="legolog: [INFO] [velero.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:47:44Z" level=debug msg="legolog: [INFO] [traefik-ui.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:48:14Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:48:14Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:48:19Z" level=debug msg="Send instance info to pilot: 2021-11-01 11:48:19.406306803 +0000 UTC m=+1200.050664414"
time="2021-11-01T11:48:19Z" level=debug msg="legolog: [INFO] [velero.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:48:19Z" level=debug msg="legolog: [INFO] [traefik-ui.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:48:49Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:48:49Z" level=debug msg="Delaying 5000000000 rather than validating DNS propagation now." providerName=godaddy.acme
time="2021-11-01T11:48:54Z" level=debug msg="legolog: [INFO] [velero.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:48:55Z" level=debug msg="legolog: [INFO] [traefik-ui.rsliotta.com] acme: Waiting for DNS record propagation."
time="2021-11-01T11:49:24Z" level=debug msg="legolog: [INFO] [velero.rsliotta.com] acme: Cleaning DNS-01 challenge"
time="2021-11-01T11:49:25Z" level=debug msg="legolog: [INFO] [traefik-ui.rsliotta.com] acme: Cleaning DNS-01 challenge"
time="2021-11-01T11:49:25Z" level=debug msg="legolog: [WARN] [traefik-ui.rsliotta.com] acme: cleaning up failed: godaddy: failed to remove TXT record: could not create record [{\"name\":\"google._domainkey\",\"type\":\"TXT\",\"data\":\"v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwueE9n9gCJbkAgJEKeZbw7J1rPajwOLFFCpf93jl8Gzrs5sHFYrGzwLLIi4HTTm3lJTwVwae+UYQsOMvCs56sHwWt6IKCqgHs9TUF7shgqP5v9KKY63w6AkQ+ufC4rtTNyj70yIb5BQpbOYjEQxV/21u2T9xVK+pSYAJJfxnhUaLrADcp5trJT+UgXuCQ9lmVSSVB13jjY2Em+d1Q9w+Ikfcv1pCXJS/eDbh9kxPCIfpj2zaCr1DH6Q4K8gpDlpbGJKYKv+Jd5NbeicYj8scTr1RDKsaQ3fWBEevDYhifELUBKV8svI9mr5Wh0NRL3d21sUOmnkRBxRHHGmcuUoK+wIDAQAB\",\"ttl\":600},{\"name\":\"_acme-challenge\",\"type\":\"TXT\",\"data\":\" \",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\" \",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"6rrai7-jm7l3PxL4WGmGoS6VMeefSHx24r-qCvUIOxU\",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"8Axt-PXQvjOVD2oi2YXqyyn8U5CDcC8P-BphlCxk3Ek\",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"8dPViXSprtioF0JUFdqX0GGa1GOjmOEJA3ZOB6YIx2I\",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"arL3w-UAiKGfVIyLE4oA_BWjSBMZXOvBtYvjSGG1ji4\",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"da898ulKf-FpIYlw_k4QNxsenHqRNnavDASvTrnWN7E\",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"fPUOMzkctKM6DcNb6V5EdUOBdoAOeEbW1gNI3Xed7UI\",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"kEtGXL3BJO09iDxqbjdlm9ly-j8qNB7idNLJDIOtnso\",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"nMZrZAxNPzVTZ0IvQtn2h96FxP95S25e_OrcGgbCdN8\",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"sJdxuKJjo_II40x39zWVq69Q8eEFf3_sPLZOu0ySbYM\",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"SOcP_mC6326YRWJHV87EV8GI_gXVeb7vYI5fib1N8kw\",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"V5qu_8vZzgH_h73Z7XFhnqJ5ivK8-9gaUmBRGUDXUow\",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"VNggz6Xlv5LjUCT8HYi1E7H5QY3Ql6GqGO6VXeEQIvU\",\"ttl\":600},{\"name\":\"_acme-challenge.velero\",\"type\":\"TXT\",\"data\":\" \",\"ttl\":600},{\"name\":\"_acme-challenge.velero\",\"type\":\"TXT\",\"data\":\"0Ad60wO_yxxJPFPb1deir6lQ37FPLeA02YCobo7Qm8A\",\"ttl\":600},{\"name\":\"_acme-challenge.velero\",\"type\":\"TXT\",\"data\":\"acme\",\"ttl\":600},{\"name\":\"_acme-challenge.velero\",\"type\":\"TXT\",\"data\":\"jrF4Bi2MqzkCYe-OBdHs6i6xocGuLVmNLMfAY2C7JjE\",\"ttl\":600},{\"name\":\"_acme-challenge.velero\",\"type\":\"TXT\",\"data\":\"OIO2Kav1i034NqCWAf5Ks7FY5odQx-j2gb2zCf1w0xk\",\"ttl\":600}]; Status: 422; Body: {\"code\":\"INVALID_BODY\",\"message\":\"Request body doesn't fulfill schema, see details in `fields`\"}"
time="2021-11-01T11:49:25Z" level=debug msg=" "
time="2021-11-01T11:49:26Z" level=debug msg="legolog: [INFO] Deactivating auth: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/832944808"
time="2021-11-01T11:49:26Z" level=debug msg="legolog: [INFO] Deactivating auth: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/832944848"
time="2021-11-01T11:49:26Z" level=error msg="Unable to obtain ACME certificate for domains \"traefik-ui.rsliotta.com\" : unable to generate a certificate for the domains [traefik-ui.rsliotta.com]: error: one or more domains had a problem:\n[traefik-ui.rsliotta.com] time limit exceeded: last error: NS ns18.domaincontrol.com. returned SERVFAIL for _acme-challenge.traefik-ui.rsliotta.com.\n" providerName=godaddy.acme
time="2021-11-01T11:49:26Z" level=error msg="Unable to obtain ACME certificate for domains \"velero.rsliotta.com\" : unable to generate a certificate for the domains [velero.rsliotta.com]: error: one or more domains had a problem:\n[velero.rsliotta.com] time limit exceeded: last error: NS ns18.domaincontrol.com. returned SERVFAIL for _acme-challenge.velero.rsliotta.com.\n" providerName=godaddy.acme
time="2021-11-01T11:53:19Z" level=debug msg="Send instance info to pilot: 2021-11-01 11:53:19.406343911 +0000 UTC m=+1500.050701521"
time="2021-11-01T11:58:19Z" level=debug msg="Send instance info to pilot: 2021-11-01 11:58:19.406628768 +0000 UTC m=+1800.050986378"
time="2021-11-01T11:58:33Z" level=debug msg="http: TLS handshake error from 10.42.2.1:50327: tls: client used the legacy version field to negotiate TLS 1.3"
time="2021-11-01T11:58:34Z" level=debug msg="Serving default certificate for request: \"\""
time="2021-11-01T11:58:40Z" level=debug msg="Serving default certificate for request: \"\""
time="2021-11-01T12:03:19Z" level=debug msg="Send instance info to pilot: 2021-11-01 12:03:19.406297733 +0000 UTC m=+2100.050655344"
time="2021-11-01T12:08:19Z" level=debug msg="Send instance info to pilot: 2021-11-01 12:08:19.406812431 +0000 UTC m=+2400.051170041"
[B:cities:traefik] liottar:/Users/liottar/projects/262life/262helm3/traefik$

Go environment (if applicable)

$ go version && go env
# paste output here

ldez · 2021-11-01T12:39:14Z

ldez
Nov 1, 2021
Maintainer

Hello,

the SERVFAIL error means that lego has a problem with your DNS.

I recommend checking the resolver's configuration of your system (ex: /etc/resolv.conf).
Maybe you have a local DNS server or a firewall that intercepts the DNS calls.

You can define a dedicated resolver if needed https://doc.traefik.io/traefik/v2.5/https/acme/#resolvers

From the machine with Traefik, you can try to show what piece responds to your DNS calls:

dig +trace traefik-ui.rsliotta.com SOA
# OR
drill -T traefik-ui.rsliotta.com SOA

1 reply

rsliotta Nov 1, 2021
Author

Hello.

liottar@tony:~$ dig  +trace traefik-ui.rsliotta.com SOA

; <<>> DiG 9.16.1-Ubuntu <<>> +trace traefik-ui.rsliotta.com SOA
;; global options: +cmd
.			7173	IN	NS	m.root-servers.net.
.			7173	IN	NS	l.root-servers.net.
.			7173	IN	NS	k.root-servers.net.
.			7173	IN	NS	j.root-servers.net.
.			7173	IN	NS	i.root-servers.net.
.			7173	IN	NS	h.root-servers.net.
.			7173	IN	NS	g.root-servers.net.
.			7173	IN	NS	f.root-servers.net.
.			7173	IN	NS	e.root-servers.net.
.			7173	IN	NS	d.root-servers.net.
.			7173	IN	NS	c.root-servers.net.
.			7173	IN	NS	b.root-servers.net.
.			7173	IN	NS	a.root-servers.net.
;; Received 262 bytes from 127.0.0.53#53(127.0.0.53) in 0 ms

;; Received 41 bytes from 193.0.14.129#53(k.root-servers.net) in 0 ms

It looks OK to me. If you look down at the logs, I believe goDaddy changed their API and the DELETE api is not working. There is an error 422 referencing that. I am creating and CAN dig the records as well.

liottar@tony:~$ dig  +trace traefik-ui.rsliotta.com SOA
; <<>> DiG 9.16.1-Ubuntu <<>> +trace traefik-ui.rsliotta.com SOA
;; global options: +cmd
.			7173	IN	NS	m.root-servers.net.
.			7173	IN	NS	l.root-servers.net.
.			7173	IN	NS	k.root-servers.net.
.			7173	IN	NS	j.root-servers.net.
.			7173	IN	NS	i.root-servers.net.
.			7173	IN	NS	h.root-servers.net.
.			7173	IN	NS	g.root-servers.net.
.			7173	IN	NS	f.root-servers.net.
.			7173	IN	NS	e.root-servers.net.
.			7173	IN	NS	d.root-servers.net.
.			7173	IN	NS	c.root-servers.net.
.			7173	IN	NS	b.root-servers.net.
.			7173	IN	NS	a.root-servers.net.
;; Received 262 bytes from 127.0.0.53#53(127.0.0.53) in 0 ms

;; Received 41 bytes from 193.0.14.129#53(k.root-servers.net) in 0 ms

Hello. This looks OK to me. I can also dig the records and the challenge records, but new ones keep getting created. In my config I pasted, I even pointed right at godaddys name servers where the records are populated. Please note also that I tried 8.8.8.8 and 8.8.4.4 and 1.1.1.1 with the same results.

If you look at the log, it looks like godaddy tightened down their API recently as you can see a status 422 come back with a schema error.

ldez · 2021-11-01T14:08:19Z

ldez
Nov 1, 2021
Maintainer

The expected output is:

$ dig traefik-ui.rsliotta.com

; <<>> DiG 9.16.20 <<>> traefik-ui.rsliotta.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 13168
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;traefik-ui.rsliotta.com.       IN      A

;; ANSWER SECTION:
traefik-ui.rsliotta.com. 600    IN      A       192.168.1.181

/ # dig +trace traefik-ui.rsliotta.com SOA

; <<>> DiG 9.16.20 <<>> +trace traefik-ui.rsliotta.com SOA
;; global options: +cmd
.                       84223   IN      NS      e.root-servers.net.
.                       84223   IN      NS      h.root-servers.net.
.                       84223   IN      NS      i.root-servers.net.
.                       84223   IN      NS      g.root-servers.net.
.                       84223   IN      NS      m.root-servers.net.
.                       84223   IN      NS      a.root-servers.net.
.                       84223   IN      NS      c.root-servers.net.
.                       84223   IN      NS      j.root-servers.net.
.                       84223   IN      NS      b.root-servers.net.
.                       84223   IN      NS      f.root-servers.net.
.                       84223   IN      NS      d.root-servers.net.
.                       84223   IN      NS      l.root-servers.net.
.                       84223   IN      NS      k.root-servers.net.
.                       84223   IN      RRSIG   NS 8 0 518400 20211114050000 20211101040000 14748 . f8SyuOFh7w4aT8Yz9rGm+YYkxSlAhk9w7VcbHPo2q2l+tXC/DaGqcSEQ MvRLioWo1iBN+/TQiWnp085gi/+LRZ+8Yg+pSzcdD97p4suidyb/Mt+N M5ZA3SGFMcb6Z0eDoV+ORdtTcPsNXcNLFaJrp4ZySYcnM6lkB43Pv2kh Vq1CSHbVUfzCFf+1DUciCQCrQcLTEnb+V7MSW0Qxa3++0EXHFiQoRL9D 763TJS+QJLNm6PR9FujgJUzYRsykBmuc1ePoRp5q5dcyUFyLphGdDNLf orTih84Ya9503v6N2bLzGdfqxX6oK6J99N5C3RyqIvGeAdIGuaVV2KOS daeGOg==
;; Received 540 bytes from 192.168.1.1#53(192.168.1.1) in 9 ms

com.                    172800  IN      NS      l.gtld-servers.net.
com.                    172800  IN      NS      b.gtld-servers.net.
com.                    172800  IN      NS      c.gtld-servers.net.
com.                    172800  IN      NS      d.gtld-servers.net.
com.                    172800  IN      NS      e.gtld-servers.net.
com.                    172800  IN      NS      f.gtld-servers.net.
com.                    172800  IN      NS      g.gtld-servers.net.
com.                    172800  IN      NS      a.gtld-servers.net.
com.                    172800  IN      NS      h.gtld-servers.net.
com.                    172800  IN      NS      i.gtld-servers.net.
com.                    172800  IN      NS      j.gtld-servers.net.
com.                    172800  IN      NS      k.gtld-servers.net.
com.                    172800  IN      NS      m.gtld-servers.net.
com.                    86400   IN      DS      30909 8 2 E2D3C916F6DEEAC73294E8268FB5885044A833FC5459588F4A9184CF C41A5766
com.                    86400   IN      RRSIG   DS 8 1 86400 20211114050000 20211101040000 14748 . Ladr2ceBVf26YT+bCl/LErChdd7LsUWayQiRqQF2GepH8W1VfGdUfvNP L52r5sK5uvPPB6/CCIXiXqM7bAAvOpYneTAC69I1JGGEB5gAbll1VH5+ lOkah8TOdR8cAe0lvLdwHJsCpvwcf8kWSaBJ3xPRrbqKFlB+NO9M/TAM C+fzofC616IhICA+9s3kVH2f1dlVDNU7TLzXZW4yKY09VhQ6X/8Gyc3B enOVhK1Td27/+Q/3h7lX9RMrFGTWKy4UayEcqAH9ZHAoG4QDSpbbSXlx SobfYv30ZihFs2aKO3BTc7OLenXe7+zl09vU7O+gx2ZyUjUT1bBALJoj sHvyTg==
;; Received 1183 bytes from 192.5.5.241#53(f.root-servers.net) in 9 ms

rsliotta.com.           172800  IN      NS      ns17.domaincontrol.com.
rsliotta.com.           172800  IN      NS      ns18.domaincontrol.com.
CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN NSEC3 1 1 0 - CK0Q1GIN43N1ARRC9OSM6QPQR81H5M9A NS SOA RRSIG DNSKEY NSEC3PARAM
CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN RRSIG NSEC3 8 2 86400 20211108052332 20211101031332 15549 com. H3fcTFeW7anO0xeSXGfZHFKSIVJCTJ3KAklIwRaehCkFP99QojNw2cPc ed6Qny95R1sscx1gYNHbe10pS47kPJGle4pK+gbCX7ExJ7/KP7NTx9Wj jq8JPhL/oeM7yAvXEYhRynMO0ivSe28WZzuGtEiYeFxuE7PdNiB1BJLk e7jKgj/6biUiX/pUCBhMJeT/QjSXgC3u0FhmKdeBA7Pm5w==
OHVQSQ02I4J2A64CA6PQ8T6PMT2SM9Q7.com. 86400 IN NSEC3 1 1 0 - OHVRDIKV6DOSU4BB8JGHG2LDOVO8NMCL NS DS RRSIG
OHVQSQ02I4J2A64CA6PQ8T6PMT2SM9Q7.com. 86400 IN RRSIG NSEC3 8 2 86400 20211107053010 20211031042010 15549 com. r/sPB8KsylzWyJvhL5t1ob+KmQWfrRsckumxpE8Hw8xAhw1dCgYD2wIe 7wlNp+YBGSlAeJ/ZIMQhdVs5rGADZWnnngq9k1FubqtvueVCdhWl5OuZ WzjT5jykFLsrYrEah+kiQbCnRDtbJIfG6CZjpxy8DrHlbw/ZYmATGXdw 97JGPJvhkNJqk5lvLOMhH7ANF++bK8cnXNGVQb/xIyY0tw==
;; Received 741 bytes from 192.33.14.30#53(b.gtld-servers.net) in 16 ms

rsliotta.com.           3600    IN      SOA     ns17.domaincontrol.com. dns.jomax.net. 2021110102 28800 7200 604800 3600
;; Received 120 bytes from 173.201.76.9#53(ns18.domaincontrol.com) in 19 ms

I recommend removing manually all TXT records with the names _acme-challenge, _acme-challenge.traefik-ui, _acme-challenge.velero.

FYI the GoDaddy doesn't really have a DELETE endpoint, the endpoint used to create or "delete" a record is the same.
The request body follows the API documentation.

5 replies

rsliotta Nov 1, 2021
Author

Yup. Did all that too. Been at this for a couple days. The real issue is the 422 error. I am confident it needs to be fixed. I will try again. Also each time I restart traefik it makes it worse.

Here is the SPECIFIC request body failure...

time="2021-11-01T11:49:25Z" level=debug msg="legolog: [INFO] [traefik-ui.rsliotta.com] acme: Cleaning DNS-01 challenge"
time="2021-11-01T11:49:25Z" level=debug msg="legolog: [WARN] [traefik-ui.rsliotta.com] acme: cleaning up failed: godaddy: failed to remove TXT record: could not create record [{\"name\":\"google._domainkey\",\"type\":\"TXT\",\"data\":\"v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwueE9n9gCJbkAgJEKeZbw7J1rPajwOLFFCpf93jl8Gzrs5sHFYrGzwLLIi4HTTm3lJTwVwae+UYQsOMvCs56sHwWt6IKCqgHs9TUF7shgqP5v9KKY63w6AkQ+ufC4rtTNyj70yIb5BQpbOYjEQxV/21u2T9xVK+pSYAJJfxnhUaLrADcp5trJT+UgXuCQ9lmVSSVB13jjY2Em+d1Q9w+Ikfcv1pCXJS/eDbh9kxPCIfpj2zaCr1DH6Q4K8gpDlpbGJKYKv+Jd5NbeicYj8scTr1RDKsaQ3fWBEevDYhifELUBKV8svI9mr5Wh0NRL3d21sUOmnkRBxRHHGmcuUoK+wIDAQAB\",\"ttl\":600},{\"name\":\"_acme-challenge\",\"type\":\"TXT\",\"data\":\" \",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\" \",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"6rrai7-jm7l3PxL4WGmGoS6VMeefSHx24r-qCvUIOxU\",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"8Axt-PXQvjOVD2oi2YXqyyn8U5CDcC8P-BphlCxk3Ek\",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"8dPViXSprtioF0JUFdqX0GGa1GOjmOEJA3ZOB6YIx2I\",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"arL3w-UAiKGfVIyLE4oA_BWjSBMZXOvBtYvjSGG1ji4\",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"da898ulKf-FpIYlw_k4QNxsenHqRNnavDASvTrnWN7E\",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"fPUOMzkctKM6DcNb6V5EdUOBdoAOeEbW1gNI3Xed7UI\",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"kEtGXL3BJO09iDxqbjdlm9ly-j8qNB7idNLJDIOtnso\",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"nMZrZAxNPzVTZ0IvQtn2h96FxP95S25e_OrcGgbCdN8\",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"sJdxuKJjo_II40x39zWVq69Q8eEFf3_sPLZOu0ySbYM\",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"SOcP_mC6326YRWJHV87EV8GI_gXVeb7vYI5fib1N8kw\",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"V5qu_8vZzgH_h73Z7XFhnqJ5ivK8-9gaUmBRGUDXUow\",\"ttl\":600},{\"name\":\"_acme-challenge.traefik-ui\",\"type\":\"TXT\",\"data\":\"VNggz6Xlv5LjUCT8HYi1E7H5QY3Ql6GqGO6VXeEQIvU\",\"ttl\":600},{\"name\":\"_acme-challenge.velero\",\"type\":\"TXT\",\"data\":\" \",\"ttl\":600},{\"name\":\"_acme-challenge.velero\",\"type\":\"TXT\",\"data\":\"0Ad60wO_yxxJPFPb1deir6lQ37FPLeA02YCobo7Qm8A\",\"ttl\":600},{\"name\":\"_acme-challenge.velero\",\"type\":\"TXT\",\"data\":\"acme\",\"ttl\":600},{\"name\":\"_acme-challenge.velero\",\"type\":\"TXT\",\"data\":\"jrF4Bi2MqzkCYe-OBdHs6i6xocGuLVmNLMfAY2C7JjE\",\"ttl\":600},{\"name\":\"_acme-challenge.velero\",\"type\":\"TXT\",\"data\":\"OIO2Kav1i034NqCWAf5Ks7FY5odQx-j2gb2zCf1w0xk\",\"ttl\":600}]; Status: 422; Body: {\"code\":\"INVALID_BODY\",\"message\":\"Request body doesn't fulfill schema, see details in `fields`\"}"
time="2021-11-01T11:49:25Z" level=debug msg=" "
time="2021-11-01T11:49:26Z" level=debug msg="legolog: [INFO] Deactivating auth: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/832944808"
time="2021-11-01T11:49:26Z" level=debug msg="legolog: [INFO] Deactivating auth: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/832944848"

ldez Nov 1, 2021
Maintainer

I've already READ the line with the error message...

I analyzed the JSON from the log and this request body is valid based on the documentation API.
I was thinking about an invalid existing TXT record, this is why I asked you to clean your records.
I didn't find any information about API changes.

ldez Nov 2, 2021
Maintainer

I have no clue to reproduce the bug. I will try to find a GoDaddy account.

ldez Nov 2, 2021
Maintainer

A colleague lent me a GoDaddy API key.
I take the JSON from your logs and I tried to use it, and I don't reproduce the problem.

There is something in your DNS entries that I don't see in the logs that creates this problem, but I don't know what.

ldez Nov 3, 2021
Maintainer

The GoDaddy API seems to have some weird behaviors AnalogJ/lexicon#362 (comment)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GoDaddy - Stuck in prop #1519

{{title}}

Replies: 2 comments 6 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

Select a reply

GoDaddy - Stuck in prop #1519

rsliotta Nov 1, 2021

Welcome

What did you expect to see?

What did you see instead?

How do you use lego?

Reproduction steps

Version of lego

Logs

Go environment (if applicable)

Replies: 2 comments · 6 replies

ldez Nov 1, 2021 Maintainer

rsliotta Nov 1, 2021 Author

ldez Nov 1, 2021 Maintainer

rsliotta Nov 1, 2021 Author

ldez Nov 1, 2021 Maintainer

ldez Nov 2, 2021 Maintainer

ldez Nov 2, 2021 Maintainer

ldez Nov 3, 2021 Maintainer

rsliotta
Nov 1, 2021

Replies: 2 comments 6 replies

ldez
Nov 1, 2021
Maintainer

rsliotta Nov 1, 2021
Author

ldez
Nov 1, 2021
Maintainer

rsliotta Nov 1, 2021
Author

ldez Nov 1, 2021
Maintainer

ldez Nov 2, 2021
Maintainer

ldez Nov 2, 2021
Maintainer

ldez Nov 3, 2021
Maintainer