Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Kafka output plugin topic_suffix option #3196

Merged

Conversation

trueneu
Copy link
Contributor

@trueneu trueneu commented Sep 4, 2017

Add ability to the Kafka output plugin to send data to different topics, based on the metric name or tags

Required for all PRs:

  • Signed CLA.
  • Associated README.md updated.
  • Has appropriate unit tests.

This addresses #3177

@trueneu trueneu force-pushed the pgurkov/kafka-topics-from-metric-name branch 4 times, most recently from 37ad17e to 6bbcfd8 Compare September 4, 2017 17:05
@trueneu trueneu force-pushed the pgurkov/kafka-topics-from-metric-name branch from 6bbcfd8 to 96c6bdf Compare September 4, 2017 18:01
Copy link
Contributor

@danielnelson danielnelson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets combine the tag and tags method using the tags method?

Should we have a separator between the topic and topic_suffix? Maybe rename key_separator to just separator?

"": TOPIC_SUFFIX_METHOD_EMPTY,
"measurement": TOPIC_SUFFIX_METHOD_MEASUREMENT,
"tag": TOPIC_SUFFIX_METHOD_TAG,
"tags": TOPIC_SUFFIX_METHOD_TAGS,
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would remove the map/enums, they don't really buy us anything from a type safety point of view. You could just have a list of valid strings for validating the config file.

Copy link
Contributor Author

@trueneu trueneu Sep 6, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're probably right about having separator between topic and topic_suffix, I think that follows the principle of least astonishment, good catch.
Renamed key_separator to just separator also.
I'm not sure about maps/enums though, because they kinda make the code less error-prone, it's easier to make a typo in a string than in a constant name. We have testcases covering that though. (And I'm not familiar with commonly accepted practices in Go).
Pushed! Any other thoughts?

Add test for ValidateTopicSuffixMethod
Remove k.Connect() from topic suffixes test
Modify existing testcases accordingly
Add more testcases for topic names
Modify README.md accordingly
@danielnelson danielnelson merged commit 5d4eec6 into influxdata:master Sep 6, 2017
@danielnelson danielnelson added this to the 1.5.0 milestone Sep 6, 2017
@danielnelson danielnelson added area/kafka feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin labels Sep 6, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/kafka feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants