DevOps repository setup

[joernb]

I would like to pick up the discussion about hosting / cloud platforms / having a preproduction environment / etc, that came up in the last call. However, I take a step back here and focus on the underlying requirements for a hosting solution and how we might structure our repositories to integrate different/competing solutions.

Brainstorming:

• We don't know, who will host the backend in the future. It might be ito, it might be some public institution. At the moment, we cannot find the perfect fit for a hosting solution because we do not know the stakeholder requirements (e.g. no cloud hosting, regulations, etc). In this situation, it is probably a good idea to favor platform-agnostic solutions (like Docker) over platform-specific ones.
• There are however still good reasons to invest time and effort into hosting solutions right now:
  • We need something to pitch app+backend to interested adopters and journalists.
  • Hosting a staging / preproduction environment and using it with a staging build of the app is also a good idea in my opinion.
  • I can imagine, that stakeholders like governments/public institutions who want to officially adopt the "app" actually expect to get some kind of full-stack "turnkey solution" delivered. We would probably help them if we also provide a scalable hosting solution for their own infrastructure and/or some public cloud platform.
• However, when development will be done by ito but the hosting will be done by some other organization, we will have an organizational boundary between development and hosting. Maybe we should consider this in the design of our DevOps workflows, so that they also work with external hosting teams right from the beginning (e.g. how do we roll out a new backend version from ito to an external team).
• Looking at the already ongoing discussions and different solutions about where/how to host the backend (Scalability discussion #11), it might make sense to build a repository structure that allow coexisting, competing solutions to be tried out.

So I thought about a way, how we can structure our repositories and workflows and came up with the following idea:

A new infrastructure repository:

• The repository provides a place for all kinds of infrastructure related assets:
  • Documentation
  • CI pipeline configurations for different cloud platforms
  • GitHub issues for infrastructure specific problems
• The idea is to separate development repositories like api-backend from infrastructure management. A deployment workflow for api-backend would look like this:
  • The CI pipeline of api-backend deploys a tagged docker image to the GitHub organization's docker registry and automatically opens a pull request on the infrastructure repo to use that new version.
  • Accepting the pull request triggers one or more infrastructure deployment pipelines, which will take the image and deploy it to some cloud platforms / servers.
• Competing solutions can coexist within the infrastructure repo. Which pipeline is run for which environment can be determined with some kind of combination of files, branches and pipeline triggers.
• Setting permissions for accepting pull requests on infrastructure is a way for restricting access to the infrastructure. Maybe it is necessary, that those permissions need to vary from those of the development repositories (e.g. maintainers on api-backend should just have contributor access on infrastructure).
• We can advertise the infrastructure repo as ito's turnkey solution for self-hosting. Other organizations can fork the repo and set up their own pipelines and deployment processes. ito can open pull requests on their infrastructure repository to hand over new versions of deployable artifacts (e.g. the backend).
• Because changes are applied via pull request, the git log of the infrastructure repo will become a public audit trail for all operational changes to the hosting infrastructure, which promotes transparency.
• The idea is based on the concept of GitOps.

Feel free to comment on this, please. If this idea finds some support, I would volunteer to spend some time on setting this up.

[assert-not-singularity]

I support the idea of having a concept which is platform independent since I think we shouldn't force someone who wants to use the backend to use a specific service.

Having a staging environment for testing and as a showcase would also be great (see also my issue on the app CI).

[joernb]

I set up an infrastructure repository.