Bug 1815637: bump etcd v3.3.22 #47

hexfusion · 2020-05-11T14:58:58Z

This PR bumps etcd to v3.3.22 which includes a few important bugfixes:

etcd server

Fix corruption bug in defrag. mvcc/backend: Fix corruption bug in defrag etcd-io/etcd#11613

client v3

Fix "hasleader" metadata embedding. clientv3: fix "hasleader" metadata key, embed client version etcd-io/etcd#11687

gRPC Proxy

Fix panic on error for metrics handler. proxy/grpcproxy: add return on error for metrics handler etcd-io/etcd#11694

3.3.19 changelog: https://github.com/etcd-io/etcd/blob/master/CHANGELOG-3.3.md#v3319-2020-03-18
3.3.20 changelog https://github.com/etcd-io/etcd/blob/master/CHANGELOG-3.3.md#v3320-2020-04-01
3.3.21 changelog https://github.com/etcd-io/etcd/blob/master/CHANGELOG-3.3.md#v3321-2020-05-18
3.3.22 changelog https://github.com/etcd-io/etcd/blob/master/CHANGELOG-3.3.md#v3321-2020-05-18

CARRY: 3c5d90b

Once chk(ai) fails with auth.ErrAuthOldRevision it will always do, regardless how many times you retry. So the error is better be returned to fail the pending request and make the client re-authenticate.

Remove auth validation loop in v3_server.raftRequest(). Re-validation when error ErrAuthOldRevision occurs should be handled on client side.

Disable TestV3AuthOldRevConcurrent for now. See etcd-io#10468 (comment)

…f-#10218-etcd-io#10468-upstream-release-3.3 Automated cherry pick of etcd-io#10218 etcd-io#10468 on release 3.3

…-#11613-origin-release-3.3 Automated cherry pick of etcd-io#11613 to release-3.3

Check if boltOpenOptions is nil before use it.

…f-#11630-upstream-release-3.3 Automated cherry pick of etcd-io#11630 to release-3.3

Use members information from member add response, which is guaranteed to be up to date.

…f-#11638-upstream-release-3.3 Automated cherry pick of etcd-io#11638 on release-3.3

Signed-off-by: Sam Batschelet <sbatsche@redhat.com>

ref. etcd-io#11704 Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

ref. etcd-io#11687 Signed-off-by: Gyuho Lee <leegyuho@amazon.com> clientv3: fix racy writes to context key === RUN TestWatchOverlapContextCancel ================== WARNING: DATA RACE Write at 0x00c42110dd40 by goroutine 99: runtime.mapassign() /usr/local/go/src/runtime/hashmap.go:485 +0x0 github.com/coreos/etcd/clientv3.metadataSet() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/ctx.go:61 +0x8c github.com/coreos/etcd/clientv3.withVersion() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/ctx.go:47 +0x137 github.com/coreos/etcd/clientv3.newStreamClientInterceptor.func1() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/client.go:309 +0x81 google.golang.org/grpc.NewClientStream() /go/src/github.com/coreos/etcd/gopath/src/google.golang.org/grpc/stream.go:101 +0x10e github.com/coreos/etcd/etcdserver/etcdserverpb.(*watchClient).Watch() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/etcdserver/etcdserverpb/rpc.pb.go:3193 +0xe9 github.com/coreos/etcd/clientv3.(*watchGrpcStream).openWatchClient() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:788 +0x143 github.com/coreos/etcd/clientv3.(*watchGrpcStream).newWatchClient() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:700 +0x5c3 github.com/coreos/etcd/clientv3.(*watchGrpcStream).run() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:431 +0x12b Previous read at 0x00c42110dd40 by goroutine 130: reflect.maplen() /usr/local/go/src/runtime/hashmap.go:1165 +0x0 reflect.Value.MapKeys() /usr/local/go/src/reflect/value.go:1090 +0x43b fmt.(*pp).printValue() /usr/local/go/src/fmt/print.go:741 +0x1885 fmt.(*pp).printArg() /usr/local/go/src/fmt/print.go:682 +0x1b1 fmt.(*pp).doPrintf() /usr/local/go/src/fmt/print.go:998 +0x1cad fmt.Sprintf() /usr/local/go/src/fmt/print.go:196 +0x77 github.com/coreos/etcd/clientv3.streamKeyFromCtx() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:825 +0xc8 github.com/coreos/etcd/clientv3.(*watcher).Watch() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:265 +0x426 github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel.func1() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:959 +0x23e Goroutine 99 (running) created at: github.com/coreos/etcd/clientv3.(*watcher).newWatcherGrpcStream() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:236 +0x59d github.com/coreos/etcd/clientv3.(*watcher).Watch() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:278 +0xbb6 github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel.func1() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:959 +0x23e Goroutine 130 (running) created at: github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:979 +0x76d github.com/coreos/etcd/clientv3/integration.TestWatchOverlapContextCancel() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:922 +0x44 testing.tRunner() /usr/local/go/src/testing/testing.go:657 +0x107 ================== Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

ref. etcd-io#11687 Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

This reverts commit 3f6b978.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

This reverts commit acb9746.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

…uild. Signed-off-by: Sam Batschelet <sbatsche@redhat.com>

Signed-off-by: Sam Batschelet <sbatsche@redhat.com>

make build performs a sanity test on the binary image which causes problems for unsupport arch. Because we run full CI tests against the image this check is not nessisary and will allow images to be build regardless of arch. Signed-off-by: Sam Batschelet <sbatsche@redhat.com>

Signed-off-by: Sam Batschelet <sbatsche@redhat.com>

…xists, go ahead and start.

CARRY: This bump resolves and issue where max frame size (16kb) was not defined which has security implications. Signed-off-by: Sam Batschelet <sbatsche@redhat.com>

hexfusion · 2020-07-10T20:11:58Z

flake
/test e2e-aws-serial

retroflexer · 2020-07-10T20:28:37Z

/lgtm

openshift-ci-robot · 2020-07-10T20:28:53Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hexfusion, retroflexer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [hexfusion]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

hexfusion · 2020-07-10T20:32:34Z

cc @sdodson

sdodson · 2020-07-10T20:47:43Z

I'll fix the bug after this merges.

openshift-ci-robot · 2020-07-10T22:12:36Z

@hexfusion: Some pull requests linked via external trackers have merged: . The following pull requests linked via external trackers have not merged:

openshift/etcd#40 is closed
Bugzilla bug 1815637 has been moved to the MODIFIED state.

In response to this:

Bug 1815637: bump etcd v3.3.22

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-cherrypick-robot · 2020-07-10T22:26:06Z

@hexfusion: cannot checkout release-4.3: error checking out release-4.3: exit status 1. output: error: pathspec 'release-4.3' did not match any file(s) known to git.

In response to this:

here is to trying
/cherry-pick release-4.3

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

hexfusion · 2020-07-10T22:26:34Z

here is to trying

/cherry-pick openshift-4.3

openshift-cherrypick-robot · 2020-07-10T22:26:36Z

@hexfusion: #47 failed to apply on top of branch "openshift-4.3":

Applying: etcdserver: remove auth validation loop
Applying: integration: disable TestV3AuthOldRevConcurrent
Applying: mvcc/backend: Delete orphaned db.tmp files before defrag
Applying: mvcc/backend: check for nil boltOpenOptions
Applying: etcdctl: fix member add command
Applying: proxy/grpcproxy: add return on error for metrics handler
Applying: etcdserver/api/etcdhttp: log server-side /health checks
Applying: clientv3: embed api version in metadata
Applying: etcdserver/api/v3rpc: handle api version metadata, add metrics
Applying: version: 3.3.19
error: Failed to merge in the changes.
Using index info to reconstruct a base tree...
M	version/version.go
Falling back to patching base and 3-way merge...
Auto-merging version/version.go
CONFLICT (content): Merge conflict in version/version.go
Patch failed at 0011 version: 3.3.19

In response to this:

here is to trying

/cherry-pick openshift-4.3

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

horkhe and others added 30 commits November 20, 2019 16:45

etcdserver: Remove infinite loop in doSerialize

95095f8

Once chk(ai) fails with auth.ErrAuthOldRevision it will always do, regardless how many times you retry. So the error is better be returned to fail the pending request and make the client re-authenticate.

etcdserver: remove auth validation loop

5a48217

Remove auth validation loop in v3_server.raftRequest(). Re-validation when error ErrAuthOldRevision occurs should be handled on client side.

integration: disable TestV3AuthOldRevConcurrent

e1508f9

Disable TestV3AuthOldRevConcurrent for now. See etcd-io#10468 (comment)

Merge pull request etcd-io#11378 from jingyih/automated-cherry-pick-o…

70853d6

…f-#10218-etcd-io#10468-upstream-release-3.3 Automated cherry pick of etcd-io#10218 etcd-io#10468 on release 3.3

mvcc/backend: Delete orphaned db.tmp files before defrag

b3d9e29

Merge pull request etcd-io#11623 from jpbetz/automated-cherry-pick-of…

b0a4038

…-#11613-origin-release-3.3 Automated cherry pick of etcd-io#11613 to release-3.3

mvcc/backend: check for nil boltOpenOptions

7b1a92c

Check if boltOpenOptions is nil before use it.

Merge pull request etcd-io#11632 from jingyih/automated-cherry-pick-o…

e21e355

…f-#11630-upstream-release-3.3 Automated cherry pick of etcd-io#11630 to release-3.3

etcdctl: fix member add command

c58133b

Use members information from member add response, which is guaranteed to be up to date.

Merge pull request etcd-io#11665 from jingyih/automated-cherry-pick-o…

eb1df6d

…f-#11638-upstream-release-3.3 Automated cherry pick of etcd-io#11638 on release-3.3

proxy/grpcproxy: add return on error for metrics handler

1228d6c

Signed-off-by: Sam Batschelet <sbatsche@redhat.com>

etcdserver/api/etcdhttp: log server-side /health checks

30aaceb

ref. etcd-io#11704 Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

etcdserver/api/v3rpc: handle api version metadata, add metrics

d9027ce

ref. etcd-io#11687 Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

version: 3.3.19

f9c8920

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

words: whitelist "hasleader"

10d50e0

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

Revert "version: 3.3.19"

07562e2

This reverts commit 3f6b978.

version: 3.3.19

acb9746

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

travis.yaml: use Go 1.12.12

5088080

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

Revert "version: 3.3.19"

cd200b4

This reverts commit acb9746.

words: whitelist "racey"

a463bd5

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

version: 3.3.19

67da93f

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

pkg/ioutil: add "FlushN"

89ecd19

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

wal: add "etcd_wal_writes_bytes_total"

1aa5da9

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

version: 3.3.20

9fd7e2b

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

*: fix auth revision corruption bug

140bf53

auth: add new metric 'etcd_debugging_auth_revision'

06a2f81

auth: print warning log when error is ErrAuthOldRevision

e7291a1

auth: cleanup saveConsistentIndex in NewAuthStore

acd9422

etcdserver: print warn log when failed to apply request

27dffc6

hexfusion and others added 15 commits July 10, 2020 16:02

Dockerfile: resolve issue where binary was not properly copied from b…

516a080

…uild. Signed-off-by: Sam Batschelet <sbatsche@redhat.com>

OWNERS: add

a30c9dc

Signed-off-by: Sam Batschelet <sbatsche@redhat.com>

Dockerfile: set coreos org as canonical for release-3.3

7e7fa0e

Signed-off-by: Sam Batschelet <sbatsche@redhat.com>

Dockerfile: bump golang to 1.11

7342605

Signed-off-by: Sam Batschelet <sbatsche@redhat.com>

Dockerfile: bump golang 1.12

c9ebfee

Signed-off-by: Sam Batschelet <sbatsche@redhat.com>

add stub discovery-etcd-initial-cluster command

e4fd890

build openshift tools with etcd

317ecea

codify the initial cluster check as golang code

e839175

Archive data-dir if target member is unstarted

c7df133

fix removed member name, unmask error

0815076

If we weren't able to get client or get target member but memberDir e…

1810482

…xists, go ahead and start.

list all peers in initial-cluster

d883e05

make evaluation of targetMember strict

56f86d1

vendor: bump gRPC-go to v1.23.1

9860350

CARRY: This bump resolves and issue where max frame size (16kb) was not defined which has security implications. Signed-off-by: Sam Batschelet <sbatsche@redhat.com>

hexfusion force-pushed the bmp-3.3.20 branch from 3c5d90b to 9860350 Compare July 10, 2020 20:03

openshift-ci-robot assigned retroflexer Jul 10, 2020

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jul 10, 2020

sdodson added the cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. label Jul 10, 2020

openshift-merge-robot merged commit 31ade19 into openshift:openshift-4.4 Jul 10, 2020

hexfusion deleted the bmp-3.3.20 branch July 10, 2020 22:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug 1815637: bump etcd v3.3.22 #47

Bug 1815637: bump etcd v3.3.22 #47

hexfusion commented May 11, 2020 •

edited

Loading

hexfusion commented Jul 10, 2020

retroflexer commented Jul 10, 2020

openshift-ci-robot commented Jul 10, 2020

hexfusion commented Jul 10, 2020

sdodson commented Jul 10, 2020

openshift-ci-robot commented Jul 10, 2020

openshift-cherrypick-robot commented Jul 10, 2020

hexfusion commented Jul 10, 2020

openshift-cherrypick-robot commented Jul 10, 2020

Bug 1815637: bump etcd v3.3.22 #47

Bug 1815637: bump etcd v3.3.22 #47

Conversation

hexfusion commented May 11, 2020 • edited Loading

hexfusion commented Jul 10, 2020

retroflexer commented Jul 10, 2020

openshift-ci-robot commented Jul 10, 2020

hexfusion commented Jul 10, 2020

sdodson commented Jul 10, 2020

openshift-ci-robot commented Jul 10, 2020

openshift-cherrypick-robot commented Jul 10, 2020

hexfusion commented Jul 10, 2020

openshift-cherrypick-robot commented Jul 10, 2020

hexfusion commented May 11, 2020 •

edited

Loading