feat(kubernetes): Add kubernetes input plugin

[jchauncey]

Required for all PRs:

• CHANGELOG.md updated (we recommend not updating this until the PR has been approved by a maintainer)
• Sign CLA (if not already signed)
• README.md updated (if adding a new plugin)

closes #1774

[dgnorton]

@jchauncey thanks for the PR!

A couple requests:

• example yaml file for configuring telegraf to run as daemon set
• example output / line protocol

[dgnorton]

Will this and the other uid tags have very high or unlimited cardinality?

[jchauncey]

yes. everytime a pod dies and comes back it would have a new uuid. Same thing with pod name

[jchauncey]

Done

[jchauncey]

yes. everytime a pod dies and comes back it would have a new uuid. Same thing with pod name

[sparrc]

@jchauncey I'm concerned this plugin will generate stats with very high cardinality, it appears that you are storing a UUID as a tag for every:

• container
• pod
• volume

In a typical kubernetes setup, how frequently would you expect new UUIDs of each of these to get generated? how frequently in a high-volume kubernetes setup?

[jchauncey]

It largely depends. We could remove the uuid but we will still have high cardinality with the pod name because it has a unique identifier which changes every time the pod dies.

Im guess over a month there may be a few hundred pod uuids created on a moderately sized cluster.

[sstarcher]

I can tell you we churn around 3-5k pods a day. I would like to see options to limit the amount of churn for specific pods.

[jchauncey]

What I could do is have a flag that allows you to exclude pod name from the
metrics and remove uuid altogether

On Oct 14, 2016 6:19 PM, "Shane Starcher" notifications@github.com wrote:

I can tell you we churn around 3-5k pods a day.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1847 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAaRGOcmpDO6-GgxlHO909-a753PBVJrks5qz__igaJpZM4KNvsU
.

[sparrc]

@jchauncey AFAICT simply excluding the pod name & uuid will only lead to duplicate metrics overwriting each other. If flags are added they will need to completely disable to per-pod and per-container metrics from being gathered.

Does kubernetes not provide any sort of aggregated metrics? From what I can see in this PR the metrics are all very high cardinality.

[jchauncey]

No that's the point of this plugin is to get detailed pod metrics.

On Oct 14, 2016 6:58 PM, "Cameron Sparr" notifications@github.com wrote:

@jchauncey https://github.com/jchauncey AFAICT simply excluding the pod
name & uuid will only lead to duplicate metrics overwriting each other. If
flags are added they will need to completely disable to per-pod and
per-container metrics from being gathered.

Does kubernetes not provide any sort of aggregated metrics? From what I
can see in this PR the metrics are all very high cardinality.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1847 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAaRGHZ5dEO2K14ZmrWbsOVW90T2GbXIks5q0AkxgaJpZM4KNvsU
.

[jchauncey]

@sparrc @dgnorton Updated pr with the changes we discussed in slack

[dgnorton]

👍 lgtm - @jchauncey thank you for contributing this plugin and all the quick code review updates!

[rikatz]

Yeay!! This is a great plugin!!! 👍