feat(deployment): ingest version checking

[theosanderson]

We encountered an issue during a rollout in which the config map was changed and triggered a reload of the pod [edit: actually we didn't have the reloader - so I'm not sure if that is it] before the pod itself was upgraded to the next version, causing double version bumps in staging. This avoids that by making the config map contain the loculus version that it targets and making an initContainer in the deployment that checks that this matches that expected by the deployment.

https://ingest-version-checking.loculus.org/

[anna-parker]

Looks great! I see how this catches the case where we want to update the deployment before the config changes have rolled out.
However I still wonder if it will catch the case where the config changes roll out before the deployment changes. This check only runs in the init step of creating a deployment. What happens when the config is updated using reloader.stakater.com/auto? The deployment will nto be affected, the config we mounted will still get updated, however we will not perform this check as the deployment will still be running. So we will be in the same situation as before... unless maybe I miss sth?

[theosanderson]

What happens when the config is updated using reloader.stakater.com/auto? The deployment will nto be affected, the config we mounted will still get updated, however we will not perform this check as the deployment will still be running. So we will be in the same situation as before... unless maybe I miss sth?

So two things:

(A) this would work - what the reloader basically does is to trigger a new pod to be created - so the initContainers would run again
(B) I realise (just now) ingest isn't actually tagged with the reloader! We should probably fix that!

[anna-parker]

ah awesome! Didn't think that reloader caused initContainers to run again but you are correct!