Add retryable HTTP client to image-awaiter

[bleggett]

Uses https://pkg.go.dev/mod/github.com/hashicorp/go-retryablehttp which is a drop-in compatible wrapper around the default Go http client.

Will fix issue where image-awaiter never becomes healthy when deployed in a cluster with a service mesh - since the service mesh uses an init container to configure pod networking, the network might not be ready to route connections when image-awaiter tries to make a connection on startup, so image-awaiter should be a good resilient HTTP client and at least retry a few times.

Example using the retry settings of 5 retries:

~/Source/zero-to-jupyterhub-k8s/images/image-awaiter(master*) » docker run -it --rm --net=host yolo:test4 /image-awaiter -debug -namespace default -daemonset hook-image-puller                                                                                          bleggett@bleggett
2020/10/09 19:49:31 [DEBUG] GET http://localhost:8080/apis/apps/v1/namespaces/default/daemonsets/hook-image-puller
2020/10/09 19:49:31 [ERR] GET http://localhost:8080/apis/apps/v1/namespaces/default/daemonsets/hook-image-puller request failed: Get "http://localhost:8080/apis/apps/v1/namespaces/default/daemonsets/hook-image-puller": dial tcp [::1]:8080: connect: connection refused
2020/10/09 19:49:31 [DEBUG] GET http://localhost:8080/apis/apps/v1/namespaces/default/daemonsets/hook-image-puller: retrying in 5s (5 left)
2020/10/09 19:49:36 [ERR] GET http://localhost:8080/apis/apps/v1/namespaces/default/daemonsets/hook-image-puller request failed: Get "http://localhost:8080/apis/apps/v1/namespaces/default/daemonsets/hook-image-puller": dial tcp [::1]:8080: connect: connection refused
2020/10/09 19:49:36 [DEBUG] GET http://localhost:8080/apis/apps/v1/namespaces/default/daemonsets/hook-image-puller: retrying in 10s (4 left)
2020/10/09 19:49:46 [ERR] GET http://localhost:8080/apis/apps/v1/namespaces/default/daemonsets/hook-image-puller request failed: Get "http://localhost:8080/apis/apps/v1/namespaces/default/daemonsets/hook-image-puller": dial tcp [::1]:8080: connect: connection refused
2020/10/09 19:49:46 [DEBUG] GET http://localhost:8080/apis/apps/v1/namespaces/default/daemonsets/hook-image-puller: retrying in 20s (3 left)
2020/10/09 19:50:06 [ERR] GET http://localhost:8080/apis/apps/v1/namespaces/default/daemonsets/hook-image-puller request failed: Get "http://localhost:8080/apis/apps/v1/namespaces/default/daemonsets/hook-image-puller": dial tcp [::1]:8080: connect: connection refused
2020/10/09 19:50:06 [DEBUG] GET http://localhost:8080/apis/apps/v1/namespaces/default/daemonsets/hook-image-puller: retrying in 30s (2 left)
2020/10/09 19:50:36 [ERR] GET http://localhost:8080/apis/apps/v1/namespaces/default/daemonsets/hook-image-puller request failed: Get "http://localhost:8080/apis/apps/v1/namespaces/default/daemonsets/hook-image-puller": dial tcp [::1]:8080: connect: connection refused
2020/10/09 19:50:36 [DEBUG] GET http://localhost:8080/apis/apps/v1/namespaces/default/daemonsets/hook-image-puller: retrying in 30s (1 left)
2020/10/09 19:51:06 [ERR] GET http://localhost:8080/apis/apps/v1/namespaces/default/daemonsets/hook-image-puller request failed: Get "http://localhost:8080/apis/apps/v1/namespaces/default/daemonsets/hook-image-puller": dial tcp [::1]:8080: connect: connection refused
2020/10/09 19:51:06 GET http://localhost:8080/apis/apps/v1/namespaces/default/daemonsets/hook-image-puller giving up after 6 attempt(s): Get "http://localhost:8080/apis/apps/v1/namespaces/default/daemonsets/hook-image-puller": dial tcp [::1]:8080: connect: connection refused

EDIT: Fixes #1793

[welcome]

Thanks for submitting your first pull request! You are awesome! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please make sure you followed the pull request template, as this will help us review your contribution more quickly.

You can meet the other Jovyans by joining our Discourse forum. There is also a intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

[consideRatio]

This LGTM @bleggett, I made a comment about the docker image tag, but if you agree I think this can be merged right away!

I appreciate how you described the PR clearly and provided example log output into to help it seem plausible it actually works as intended!

I updated the title to not include the "fix issue #number" and instead added that to be part of the PR description, which will make GitHub automatically close the issue if this PR is merged, and also make for more conformant changelogs within this repo.

[welcome]

Congrats on your first merged pull request in this project! 🎉

Thank you for contributing, we are very proud of you! ❤️

[consideRatio]

Woop woop! Thank you @bleggett for your work, this is something I would have find troublesome to do as I'm a Golang beginner!!! ❤️ 🎉

[bleggett]

@consideRatio thanks - I accidentally left in a redundant build step in the Dockerfile - corrected that in this followup: #1833