Support objects for user-provided services credentials

[georgethebeatle]

Is there a related GitHub Issue?

#2900

What is this change about?

Previously Korifi supported plain map[string]string for UPSI
credentials. This deviates from the CF API spec where credentials can be
arbitrary objects.

As credentials are stored in secrets which are glorified
map[string][]byte we had to change the way we store the credentials
object:

• We marshal the credentials object into a byte array and store it under
  the crdentials key in the secret. We no longer use a secret key per
  every credentials value. The CFServiceInstance.Spec.SecretName
  references that credentials secret.
• We introduce CFServiceInstance.Status.Credentials and
  CFServiceInstance.Status.CredentialsObservedVersion that aremanaged
  by the CFServiceInstance controller. Once the controller
  reconciles the service instance, it sets the secret name and its
  resource version into the new status fields.
• The CFServiceBindingController reconciles the CFServiceInstance
  credentials secret into the flat format that is then used for the
  servicebinding.io binding. In case of a json object value to a key,
  the value is represented as a json string.
• Existing bindings are not affected when upgrading Korifi. However,
  once the user updates the service instance credentials, the binding
  secret is rebuilt which causes statefulset restart. Further updates to
  the credentials will not cause additional restarts.
• We have created ADR 16 documenting how this works

Does this PR introduce a breaking change?

No

Acceptance Steps

See #2900

Tag your pair, your PM, and/or team

@cloudfoundry/wg-cf-on-k8s

[tcdowney]

I think this looks like it should work, but I'm wondering if we have any automated tests that push an app that actually uses servicebindings.io bindings? Like a Java Spring app that uses https://github.com/spring-cloud/spring-cloud-bindings.

There are some important keys in there like the name of the binding and especially the type that I think are still wired through correctly, but are now becoming fairly brittle.

https://github.com/cloudfoundry-samples/spring-music is one I've used to validate service bindings are working.

[georgethebeatle]

Hey @tcdowney, thanks for taking a look, we just validated that spring-music works with this change. We thought it might be useful to deploy spring-music on the acceptance cluster, but realised that we are already deploying postfacto which is consuming a database as well. This should be good enough, right? If you think that spring boot is an important use case we might drop in a chore to deploy spring-music next to postfacto.

[tcdowney]

Hey @tcdowney, thanks for taking a look, we just validated that spring-music works with this change. We thought it might be useful to deploy spring-music on the acceptance cluster, but realised that we are already deploying postfacto which is consuming a database as well. This should be good enough, right? If you think that spring boot is an important use case we might drop in a chore to deploy spring-music next to postfacto.

Cool thank you for manually validating Spring Music! I think a chore would be good because Spring apps are one of the few app types that automatically get Cloud Native Service Binding support through https://github.com/spring-cloud/spring-cloud-bindings so we can be sure we're still creating them in a way that apps can use.

I don't think Postfacto is the best example cause I don't think we're actually even using service bindings for it. I think we're just constructing DATABASE_URL manually in its manifest:
https://github.com/eirini-forks/postfacto-release/blob/9dbfdee6dcd63787158b173dcec2468fb025eea8/manifest.yml#L6

[georgethebeatle]

@tcdowney created the chore: #3161