feat: Add the org and space based certificates for syslog drains

[ZPascal]

Description

This RFC proposes to introduce support for organization- and space-scoped client certificates for Cloud Foundry Loggregator syslog drains using mutual TLS (mTLS), covering both HTTPS and syslog+TLS protocols. By issuing certificates at the org or space level instead of per application, this initiative will simplify certificate lifecycle management, enable centralized rotation, and facilitate integration with central certificate authorities. The change targets the reduction of operational overhead and the enhancement of tenant-level security.

Involved Working Groups:

@cloudfoundry/toc
@cloudfoundry/wg-app-runtime-platform-logging-and-metrics-approvers
@cloudfoundry/wg-app-runtime-platform-logging-and-metrics-reviewers

[beyhan]

Could you please add some words about the backwards compatibility of this flow with the current flow to create and consume a syslog drain.

[beyhan]

Regarding "By issuing certificates at the org or space level instead of per application". Do you mean for the syslog drain use case? The https://docs.cloudfoundry.org/devguide/deploy-apps/instance-identity.html feature is used in different use cases.

[chombium]

IMO, this is only for the certificates used for Syslog drains and has nothing to do with the instance identity certificates.

[rkoster]

How will exiting connections drained and reestablished after an updated certificate?

[stephanme]

Can you add sections with proposed changes per CF component (e.g. Cloud Controller, loggregator, CF CLI). This helps to better understand the consequences of this RFC and required efforts. It also makes it easier to invite the affected WG areas for a review.

[stephanme]

These example configurations show the creation of user provided service instances that happen to contain certificates. How does it relate to syslog drains as documented in https://docs.cloudfoundry.org/devguide/services/log-management.html ?

Can you provide a complete example that shows e.g. how multiple apps in a space use the same space-scope certificate for a syslog drain binding and maybe even how you would rotate it?

[chombium]

@stephanme The call above will create credentials which have to be bound to the app. The problem with this approach is that these credentials are available to the app via the VCAP_SERVICES and will not be available to the Syslog Agent when creating the Syslog Drain. If this is accepted an app will have to somehow share the credentials with the Syslog Agent which will be hard to do as the Syslog Agent is a special kind of user provided service implemented as part of CF and not via an external service broker. An external broker could have collected all needed credentials and open a connection, but this is not the case.

[chombium]

IMHO, we don't need this change. What we need is a better documentation how to achieve this with the current way how syslog drains are created and work and the other CF tools we have at hand.

[chombium]

I understand the problem and complexity of managing certificates which this RFC tres to address.

At the moment, with the current implementation, one can create a single user provided service defining a Syslog Drain and bind it to multiple apps.

flowchart LR

app1["app 1"]
ups["User Provided Service"] --> b1["service binding 1"] --> sl1["Syslog Drain 1"]
app1 --> b1
app2["app 2"]
ups["User Provided Service"] --> b2["service binding 2"] --> sl2["Syslog Drain 2"]
app2 --> b2
app3["app 3"]
ups["User Provided Service"] --> b3["service binding 3"] --> sl3["Syslog Drain 3"]
app3 --> b3
appn["app n"]
ups["User Provided Service"] --> bn["service binding n"] --> sln["Syslog Drain n"]
appn --> bn

Loading

This way one can share the same Syslog Drain certificates between apps.

We've also used to have a cf drain CLI plugin for managing single and space drains which we've deprecated few years ago because of low usage.

The Syslog Drains are created as user provided services and hence are scoped to a space. One can share a service between spaces in an org with cf share-service and with that if the service is shared among all of the spaces in an organization it will have an organization wide syslog drain.

The only possible downsides of this approach are:

• one user provided service Syslog Drain per Syslog Drain URL. From my personal experience, I have rarely seen that a single organization uses different urls for their syslog drains
• possible duplication if the same certificates (ca, cert and key) are used. IMO, we should not stress much about few more bytes in the Cloud Controller's database

This RFC:

• opens a possibility to reuse credentials with different Syslog drain URLs which practically means that different client connections can be identified with the same client certificates which is a bad security practice
• breaks the cf cups call as the command for creating Syslog Drain a specific -l parameter followed by the Syslog drain url should be used. cf cups mydrain -l https://my-drain.example.com -p {...}

IMHO, we don't need this change. What we need is a better documentation how to achieve this.

Here is an example of the whole flow:
Let's say we have an organization org1 with two spaces, space1 and space2 and an org2 with space3. There is an app1 in the space1, app2 in space2 and app3 in space3.

# target org1 and space1
cf target -o org1 -s space1

# create the syslog drain
cf cups mydrain -l https://my-drain.example.com -p {...}

# bind app1 to mydrain
cf bind-service app1 mydrain

# share the service with space2
cf share-service mydrain -s space2

# share the service with space3 in org2
cf share-service mydrain -s space3 -o org2

# target space2
cf target -s space2

# bind app2 to mydrain
cf bind-service app2 mydrain

# target org2 space3
cf target -o org2 -s space3

# bind app3 to mydrain
cf bind-service app3 mydrain