Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rafthttp: probe all raft transports #10155

Merged
merged 4 commits into from
Oct 9, 2018
Merged

rafthttp: probe all raft transports #10155

merged 4 commits into from
Oct 9, 2018

Commits on Oct 7, 2018

  1. etcdserver/api/rafthttp: rename to "pipelineProber" 

    Preliminary work to add prober to "streamRt"

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
    @gyuho
    gyuho committed Oct 7, 2018
    Configuration menu
    Copy the full SHA
    47cff4d View commit details
    Browse the repository at this point in the history

  2. etcdserver/api/rafthttp: display roundtripper name in warnings 

    Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
    @gyuho
    gyuho committed Oct 7, 2018
    Configuration menu
    Copy the full SHA
    4a23907 View commit details
    Browse the repository at this point in the history

  3. etcdserver/api/rafthttp: probe all Raft messages' RTT 

    This PR adds another probing routine to monitor the connection
for Raft message transports. Previously, we only monitored
snapshot transports.

In our production cluster, we found one TCP connection had >8-sec
latencies to a remote peer, but "etcd_network_peer_round_trip_time_seconds"
metrics shows <1-sec latency distribution, which means etcd server
was not sampling enough while such latency spikes happen
outside of snapshot pipeline connection.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
    @gyuho
    gyuho committed Oct 7, 2018
    Configuration menu
    Copy the full SHA
    7b1ef37 View commit details
    Browse the repository at this point in the history

  4. etcdserver/api/rafthttp: configure "streamProber" in tests 

    Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
    @gyuho
    gyuho committed Oct 7, 2018
    Configuration menu
    Copy the full SHA
    884a8bd View commit details
    Browse the repository at this point in the history