From 8a0db505d3b993b16d17b2b7a67a4b1c810222f2 Mon Sep 17 00:00:00 2001 From: Pascal Zimmermann Date: Mon, 30 Jun 2025 09:15:11 +0200 Subject: [PATCH] feat: Add the custom headers syslog http drain RFC --- ...om-http-headers-for-syslog-https-drains.md | 41 +++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100644 toc/rfc/rfc-draft-custom-http-headers-for-syslog-https-drains.md diff --git a/toc/rfc/rfc-draft-custom-http-headers-for-syslog-https-drains.md b/toc/rfc/rfc-draft-custom-http-headers-for-syslog-https-drains.md new file mode 100644 index 000000000..1f7317a83 --- /dev/null +++ b/toc/rfc/rfc-draft-custom-http-headers-for-syslog-https-drains.md @@ -0,0 +1,41 @@ +# Meta +[meta]: #meta +- Name: Support Custom HTTP Headers for Syslog HTTPS Drains +- Start Date: 2025-06-30 +- Author(s): ZPascal +- Status: Draft +- RFC Pull Request: [community#1228](https://github.com/cloudfoundry/community/pull/1228) + +## Summary + +This proposal introduces support for specifying **custom HTTP headers** in **HTTPS syslog drains** for Cloud Foundry's Loggregator system. It enables native, secure integration with third-party logging systems that require authentication via bearer tokens or custom headers, such as **Splunk HEC**, **Datadog**, and similar modern log routing solutions. + +## Problem + +Modern observability platforms increasingly rely on HTTP-based ingestion with authentication through bearer tokens or custom headers (Splunk, Datadog). Currently, Loggregator supports HTTPS drains but **does not allow adding custom headers**, making integration with key services (e.g., Splunk HEC, Datadog, New Relic, Elastic) difficult or impossible without proxies or custom sidecar containers. Using proxies adds complexity and operational burden, while putting authentication in drain URLs is insecure and often not compatible. + +## Proposal + +We propose the following enhancements: + +1. **Extend the syslog drain binding specification** to allow a `headers` field (key-value map). +2. Modify Metron Agent and Syslog Adapter components to **inject the custom headers** into outbound HTTPS syslog requests. +3. Implement **validation logic** to block prohibited/unsafe headers (such as `Host`, `Content-Length`). +4. Ensure the headers field is visible in drain metadata available to platform operators and log integration tools. + +### Example configuration + +#### Example CUPS call +```bash +cf create-user-provided-service DRAIN-NAME -l SYSLOG-URL -p '{"headers": {"Authorization": "Bearer abc123xyz", "X-Splunk-Request-Source": "cloudfoundry"}}' +``` + +#### Forwarded JSON details +```json +{ + "headers": { + "Authorization": "Bearer abc123xyz", + "X-Splunk-Request-Source": "cloudfoundry" + } +} +``` \ No newline at end of file