[stale] Throughput limit exemptions for users #11555
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dlex
force-pushed
the
11438/tp-exemptions-for-users
branch
from
9696a2e
to
51ef2ec
Compare
dlex
force-pushed
the
11438/tp-exemptions-for-users
branch
from
51ef2ec
to
b080c8f
Compare
7 tasks
BenPope
reviewed
Jun 28, 2023
Comment on lines
75
to
108
| // Equality | ||
| for (size_t k = 0; k != cfg.cgroups().size(); ++k) { | ||
| BOOST_TEST(cfg.cgroups()[k] == cfg.cgroups()[k], "k=" << k); | ||
| for (size_t l = 0; l != cfg.cgroups().size(); ++l) { | ||
| if (k != l) { | ||
| BOOST_TEST( | ||
| !(cfg.cgroups()[k] == cfg.cgroups()[l]), | ||
| "k=" << k << " l=" << l); | ||
| } | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
nitpick: This code is duplicated in throughput_control_group_by_principal_test further down.
There was a problem hiding this comment.
It's the same in both tests however the data (the configuration) is different, so effectively one of them checks how client_id_matcher_type::operator== is used and the other checks acl_principal::operator==
dlex
force-pushed
the
11438/tp-exemptions-for-users
branch
from
b080c8f
to
2bbfd99
Compare
|
Force push: minor changes f/up the review |
|
CI failures in https://buildkite.com/redpanda/redpanda/builds/32168#01890492-de9f-4c14-b664-b55e48e3e2f0: in https://buildkite.com/redpanda/redpanda/builds/32168#018904b5-dbc2-420b-bcd6-861b6f5b19db: in https://buildkite.com/redpanda/redpanda/builds/32168#018904b5-dbc4-4cae-b10e-c4873e22a847:
|
|
/ci-repeat 5 |
|
/ci-repeat 1 |
Add a function to RpkTool to create an allow topic ACL for a user. A reusable part of acl_create_allow_cluster refactored.
dlex
force-pushed
the
11438/tp-exemptions-for-users
branch
from
2bbfd99
to
9481abd
Compare
Support --username/--password authentication options Add a function to check if the producer is running Log service name with progress messages so that in case of several producers running at the same time, it is possible to understand who is doing what
typos, renames, comments
when throughput group is matched, log the client address
Becasue of the unique_ptr involved, the default equality operator does not work correctly here.
`throughput_control_group` gets additional matching criteria by ACL principals (users, etc.). In yaml/json, there can now be a list of principals the group must match besides client_id if provided. Only the principal type `user` is supported in this commit.
Add user names to the matching pattern of throughput_control groups
acl_principal is now another key to snc_quotas_context. It is passed from the connection_context to create the context by matching with a tput ctrl group, and to verify whether the context is still valid.
Measure time RpkProducer has spent sending messages and make it available via a property Prefix the "Finish sending" message with the service_id so that it's clear in the log what producer it is for in case there are many running simultaneously
A new test to verify that when a user matches a tput ctrl group, the connections authenticated by the user are not throttled while the rest still are.
to complement the logging of node configurations. Note that cluster_config.yaml records cluster config when nodes stop, whlie this log dumps the initial bootstrap config.
dlex
force-pushed
the
11438/tp-exemptions-for-users
branch
from
9481abd
to
ea717ef
Compare
|
Force push:
|
aanthony-rp
changed the title
|
7 months old. not merging. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Throughput control groups are used to define throughput limit exemptions at the moment. This PR adds a criteria to throughput control groups allowing them to match authenticated connections by the principal type
user.Re #11438
Backports Required
Release Notes
Improvements
kafka_throughput_controlis used to define throughput control groups for which Kafka traffic will not be limited by the values specified bykafka_throughput_limit_node_*_bps. The criteria for these groups now can have a list of authenticated usernames along with client_id regex.