Add Google Cloud Monitoring input (Stackdriver)

[oryband]

Feature Request

Proposal:

Google has made Stackdriver generally available, and has a metrics API.

Please add support for fetching metrics from Google Cloud Monitoring, similar to AWS CloudWatch input.

Use case

Google Cloud Platform is a major cloud provider nowadays, and I'd prefer pulling all metrics into a single backend source (InfluxDB, Prometheus) and manage alerting from there, instead of having to manage various alerting sources (in this case, Google Stackdriver along with my main monitoring/alerting backend).

[oryband]

Hi! Any idea when you'll start work on this issue?

[sparrc]

@oryband we most likely won't work on this, it is up to contributors

[derrley]

We have a use for this -- has anyone started working on it or have a non-official version we can try out? If not, I think we'll put in the effort to build it.

[danielnelson]

@derrley No one is working on this that I am aware of, so that would be great. When you have an idea of how it will work you could post the config here for comments.

[derrley]

@danielnelson I've got a basic implementation working that is more or less based on the aws cloudwatch plugin, and works with configuration like this:

[global_tags]
  test_tag = "kyle"

[agent]
  interval = "1m"

[[outputs.influxdb]]
  database = "kyledevtest"
  urls = [
    "http://localhost:9001"
  ]

[[inputs.gcp_stackdriver]]
  project = "projects/erudite-bloom-151019"
  rateLimit = 14
  exclude = [
    "agent",
    "aws",
    "custom"
  ]

It integrates against Google's V3 monitoring API (grpc) and basically just scrapes out all metrics and writes each to an independent measurement with the field name "value" and tag names which are basically the union of the metric and resource tags.

I'll probably add some configuration for specifying GCP credentials, but the integrated-against client import "cloud.google.com/go/monitoring/apiv3" is pretty good about scraping all the various different normal ways for people to configure GCP command lines tools (e.g., gcloud auth login) so I don't want to be too heavy handed there.

Anything else anyone wants to know before I start going thru the process of offering it up? Anything I should know about adding this before doing something stupid? :)

[derrley]

I should add --

One thing I'd like some feedback on is how you guys think "distribution" metrics should be handled. There really is no great way of doing it. Right now I'm actually writing out all the bucket values as different fields. Another alternative is asking GCP to pre-aggregate them into percentile buckets and writing those out. Pre-aggregated buckets makes the values harder to aggregate. But raw bucket values are pretty much impossible to query in influxdb.

[danielnelson]

@derrley Perhaps distribution metrics can be encoded in the same way the histogram aggregator does it?

[derrley]

I'm not super up on the aggregator in telegraf -- I suppose I need to experiment with it. I'm working with my company on allowing me to sign the contributor agreement and then I'll make a pr for initial feedback. It's working in our test env right now.

…

On Jan 29, 2018 1:57 PM, "Daniel Nelson" ***@***.***> wrote: @derrley <https://github.com/derrley> Perhaps distribution metrics can be encoded in the same way the histogram aggregator does it? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2099 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABLW3HI2jFY5A_P9nA2wz15OWND3Niwtks5tPiKugaJpZM4K975_> .

[danielnelson]

Quick explanation is that the upper bound of the bucket is encoded as a tag:

cpu,cpu=cpu1,host=localhost,le=0.0 usage_idle_bucket=0i 1486998330000000000
cpu,cpu=cpu1,host=localhost,le=50.0 usage_idle_bucket=2i 1486998330000000000
cpu,cpu=cpu1,host=localhost,le=100.0 usage_idle_bucket=2i 1486998330000000000
cpu,cpu=cpu1,host=localhost,le=+Inf usage_idle_bucket=2i 1486998330000000000

[derrley]

Oh cool. My implementation encodes the lower bound in the name. I can easily change that!

…

On Jan 29, 2018 2:20 PM, "Daniel Nelson" ***@***.***> wrote: Quick explanation is that the upper bound of the bucket is encoded as a tag: cpu,cpu=cpu1,host=localhost,le=0.0 usage_idle_bucket=0i 1486998330000000000 cpu,cpu=cpu1,host=localhost,le=50.0 usage_idle_bucket=2i 1486998330000000000 cpu,cpu=cpu1,host=localhost,le=100.0 usage_idle_bucket=2i 1486998330000000000 cpu,cpu=cpu1,host=localhost,le=+Inf usage_idle_bucket=2i 1486998330000000000 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2099 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABLW3MdekDOZk59I1gh9GQ1atzbUZehkks5tPigpgaJpZM4K975_> .

[russorat]

connect #3774