Skip to content

Commit

Permalink
tests/integration: Update TestLeasingDeleteRangeContendTxn
Browse files Browse the repository at this point in the history
The TestLeasingDeleteRangeContendTxn is trying to test for RangeDelete when
the target resources are being updated.  When the `txnLeasing` wants a
server-side transaction, it needs to ensure all the keys mod revision should
be leass than what it saw. If the compare fails, it will repeat to apply the
server-side transaction until it is sucessful. I believe the test-case is
trying to verify how the `txnLeasing` handles the race issue.

Before the patch etcd-io#15401, the resource-updating goroutine keeps updating until
the RangeDelete finishes. The testcase is flaky because two goroutines are
sharing one `ctx` and grpc-go client won't wait for the response if `ctx`
has been canceled.

For example,

| DelLease Goroutine   | PutLease Goroutine         | ETCD Server                    | Key/0 Status |
| --                   | ---                        | --                             | --           |
| deleted              |                            |                                | version = 0  |
|                      | send update(key/0=123) req | received update(key/0=123) req | version = 0  |
| cancel               |                            |                                | version = 0  |
|                      | exit because of cancel     |                                | version = 0  |
| get key/0 by putkv   |                            |                                | version = 0  |
|                      |                            | applied update(key/0=123)      | version = 1  |
| get key/0 by raw-cli |                            |                                | version = 1  |

So `raw-cli` gets `[key/0=123]` while the `putkv` gets `[]`. If `putkv`
applies two update reqs to ETCD server and the last one is canceled
before apply, the error will be like:

```
expected [key:"key/0" version:2 value:"123" ], got [key:"key/0" version:1 value:"123" ]
```

The resource-updating goroutine should not share the ctx with RangeDelete here.
And I also revert current main branch because the resource-update goroutine
only updates 8 times and might exit before `RangeDelete`. In this case,
the `txnLeasing` is not handling the race issue.

Fixes: etcd-io#15352

Signed-off-by: Wei Fu <fuweid89@gmail.com>
  • Loading branch information
fuweid committed Mar 7, 2023
1 parent 76a5636 commit 3419230
Showing 1 changed file with 7 additions and 7 deletions.
14 changes: 7 additions & 7 deletions tests/integration/clientv3/lease/leasing_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ package lease_test

import (
"context"
"errors"
"fmt"
"math/rand"
"reflect"
Expand Down Expand Up @@ -1286,12 +1285,13 @@ func testLeasingDeleteRangeContend(t *testing.T, op clientv3.Op) {
donec := make(chan struct{})
go func(t *testing.T) {
defer close(donec)
for i := 0; (i < maxKey) && (ctx.Err() == nil); i++ {
key := fmt.Sprintf("key/%d", i%8)
if _, err = putkv.Put(ctx, key, "123"); err != nil {
if !errors.Is(err, context.Canceled) {
t.Errorf("fail putting key %s, err was: %v", key, err)
}
for i := 0; ctx.Err() == nil; i++ {
key := fmt.Sprintf("key/%d", i%maxKey)
if _, err := putkv.Put(context.TODO(), key, "123"); err != nil {
t.Errorf("fail putting key %s: %v", key, err)
}
if _, err = putkv.Get(context.TODO(), key); err != nil {
t.Errorf("fail getting key %s: %v", key, err)
}
}
}(t)
Expand Down

0 comments on commit 3419230

Please sign in to comment.