Expose Kafka Idempotency Config #45
Conversation
* Upgraded Sarama Version; Added Kafka Idempotency (only once) config * Bump alpine-glibc version to resolve gcc vulnerability found in our security scan * Bump Node Version to current Active LTS version * Fixed issue with acks not being set properly (prerequisite for Idemptoncy) * Added config for Kafka MaxOpenRequests (prerequisite for Idemptoncy)
|
@zhouzhuojie or @marceloboeira would one of you be able to review this? |
|
Is there a way to rerun the ci / unit_test suite? It seems like it timed out, but I can't see any reason why it would and it runs fine in my local. I'm wondering if it just got caught on a network issue or something. |
|
@fenriskiba I haven't had the time yet to review this, I'll take a look later this week. In any case, I just re-triggered the jobs to see if it works now |
|
yes, let's separate the docker image bump from Kafka config changes, easy to revert if needed. thank you! |
| @@ -5,7 +5,7 @@ go 1.16 | |||
| require ( | |||
| cloud.google.com/go v0.37.4 | |||
| github.com/DataDog/datadog-go v0.0.0-20180330214955-e67964b4021a | |||
| github.com/Shopify/sarama v1.19.0 | |||
| github.com/Shopify/sarama v1.29.1 | |||
There was a problem hiding this comment.
this is a big version bump, any noticeable breaking changes?
There was a problem hiding this comment.
The only update I'm aware of that could be breaking is added SASLVersion in config #1410, though I'll admit, I'm not familiar enough with the SASL Handshake versions work to know how significant that is.
If you think that's a problem/risk, I could change it to v1.20.1. I went with latest (at the time of our internal testing) since I figured staying up to date would be preferable, but the Idempotency configs are back in v1.20.
Codecov Report
@@ Coverage Diff @@
## main #45 +/- ##
==========================================
+ Coverage 81.55% 81.57% +0.01%
==========================================
Files 27 27
Lines 2174 2176 +2
==========================================
+ Hits 1773 1775 +2
Misses 314 314
Partials 87 87
Continue to review full report at Codecov.
|
|
pending on @marceloboeira |
Resolving Issue #21
Description
Motivation and Context
See Issue #21
How Has This Been Tested?
Manually regression and stress tested in our OpenShift environment, both with default Kafka configs and Idempotency enabled.
Types of changes
Checklist: