fix dial-timeout not affected for client watch command #18336

chengjoey · 2024-07-16T06:07:05Z

Lines 327 to 330 in e7f5729

    
           	var cancel context.CancelFunc 
        
           	dctx, cancel = context.WithTimeout(c.ctx, c.cfg.DialTimeout) 
        
           	defer cancel() // TODO: Is this right for cases where grpc.WithBlock() is not set on the dial options? 
        
           }

As noted before, it is wrong to not have grpc.WithBlock()

in the test, we will also addoption grpc.WithBlock(

etcd/client/v3/client_test.go

Lines 104 to 118 in e7f5729

    
           // grpc.WithBlock to block until connection up or timeout 
        
           testCfgs := []Config{ 
        
           	{ 
        
           		Endpoints:   []string{"http://254.0.0.1:12345"}, 
        
           		DialTimeout: 2 * time.Second, 
        
           		DialOptions: []grpc.DialOption{grpc.WithBlock()}, 
        
           	}, 
        
           	{ 
        
           		Endpoints:   []string{"http://254.0.0.1:12345"}, 
        
           		DialTimeout: time.Second, 
        
           		DialOptions: []grpc.DialOption{grpc.WithBlock()}, 
        
           		Username:    "abc", 
        
           		Password:    "def", 
        
           	}, 
        
           }

k8s-ci-robot · 2024-07-16T06:07:11Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: chengjoey
Once this PR has been reviewed and has the lgtm label, please assign wenjiaswe for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot · 2024-07-16T06:07:15Z

Hi @chengjoey. Thanks for your PR.

I'm waiting for a etcd-io member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

jmhbnz · 2024-07-16T19:17:52Z

/ok-to-test

codecov-commenter · 2024-07-16T19:37:57Z

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 0.00%. Comparing base (bc9801f) to head (439fe66).
Report is 120 commits behind head on main.

❗ Current head 439fe66 differs from pull request most recent head 8caf4d0

Please upload reports for the commit 8caf4d0 to get more accurate results.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

see 417 files with indirect coverage changes

@@             Coverage Diff             @@
##             main   #18336       +/-   ##
===========================================
- Coverage   68.96%        0   -68.97%     
===========================================
  Files         417        0      -417     
  Lines       35332        0    -35332     
===========================================
- Hits        24366        0    -24366     
+ Misses       9554        0     -9554     
+ Partials     1412        0     -1412

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bc9801f...8caf4d0. Read the comment docs.

chengjoey · 2024-07-17T02:01:07Z

/test pull-etcd-unit-test-386
/test pull-etcd-unit-test-amd64
/test pull-etcd-e2e-amd64

chengjoey · 2024-07-17T02:07:48Z

/retest-required

jberkus · 2024-07-18T18:26:57Z

@chengjoey are you able to troubleshoot these test failures on your own? Or do you need help?

chengjoey · 2024-07-19T01:19:34Z

@chengjoey are you able to troubleshoot these test failures on your own? Or do you need help?

thanks @jberkus , i will troubleshoot the test failure in the next two days.

ahrtr · 2024-07-19T12:01:19Z

Thanks for raising this PR.

It's a valid fix, but it might break some of the existing user experience. I guess it might be the reason why some test cases failed. So I suggest to only fix this in main.

Signed-off-by: joey <zchengjoey@gmail.com>

chengjoey · 2024-07-20T04:45:54Z

updates:
Modifying the original logic and adding Option grpc.WithBlock will indeed affect the old logic. For example, grpc_proxy must create a client before starting the server. At this time, setting Block will cause the server to never start.

etcd/server/etcdmain/grpc_proxy.go

Lines 239 to 269 in 010d462

    
           httpClient := mustNewHTTPClient() 
        
           srvhttp, httpl := mustHTTPListener(lg, m, tlsInfo, client, proxyClient) 
        
           if err := http2.ConfigureServer(srvhttp, &http2.Server{ 
        
           	MaxConcurrentStreams: maxConcurrentStreams, 
        
           }); err != nil { 
        
           	lg.Fatal("Failed to configure the http server", zap.Error(err)) 
        
           } 
        
           errc := make(chan error, 3) 
        
           go func() { errc <- newGRPCProxyServer(lg, client).Serve(grpcl) }() 
        
           go func() { errc <- srvhttp.Serve(httpl) }() 
        
           go func() { errc <- m.Serve() }() 
        
           if len(grpcProxyMetricsListenAddr) > 0 { 
        
           	mhttpl := mustMetricsListener(lg, tlsInfo) 
        
           	go func() { 
        
           		mux := http.NewServeMux() 
        
           		grpcproxy.HandleMetrics(mux, httpClient, client.Endpoints()) 
        
           		grpcproxy.HandleHealth(lg, mux, client) 
        
           		grpcproxy.HandleProxyMetrics(mux) 
        
           		grpcproxy.HandleProxyHealth(lg, mux, proxyClient) 
        
           		lg.Info("gRPC proxy server metrics URL serving") 
        
           		herr := http.Serve(mhttpl, mux) 
        
           		if herr != nil { 
        
           			lg.Fatal("gRPC proxy server metrics URL returned", zap.Error(herr)) 
        
           		} else { 
        
           			lg.Info("gRPC proxy server metrics URL returned") 
        
           		} 
        
           	}() 
        
           }

So I changed it to only watch_command to set the block when creating the client, and other commands remain the same.
This will only fix the watch dial-timeout, and will not affect other logic
@ahrtr @jberkus PTAL

chengjoey · 2024-07-20T05:35:43Z

Just like e2e test does，newClient will set grpc.WithBlock

etcd/tests/e2e/utils.go

Lines 50 to 54 in d6c0127

    
           ccfg := clientv3.Config{ 
        
           	Endpoints:   entpoints, 
        
           	DialTimeout: 5 * time.Second, 
        
           	DialOptions: []grpc.DialOption{grpc.WithBlock()}, 
        
           }

k8s-ci-robot added the area/clientv3 label Jul 16, 2024

k8s-ci-robot added the needs-ok-to-test label Jul 16, 2024

k8s-ci-robot added the size/XS label Jul 16, 2024

k8s-ci-robot added ok-to-test and removed needs-ok-to-test labels Jul 16, 2024

chengjoey force-pushed the fix/watch-dial-timeout branch from 8e1f4a9 to cd83269 Compare July 20, 2024 04:35

k8s-ci-robot added area/etcdctl size/S and removed size/XS labels Jul 20, 2024

fix dial-timeout not affected for client watch command

8caf4d0

Signed-off-by: joey <zchengjoey@gmail.com>

chengjoey force-pushed the fix/watch-dial-timeout branch from cd83269 to 8caf4d0 Compare July 20, 2024 04:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix dial-timeout not affected for client watch command #18336

fix dial-timeout not affected for client watch command #18336

chengjoey commented Jul 16, 2024

k8s-ci-robot commented Jul 16, 2024

k8s-ci-robot commented Jul 16, 2024

jmhbnz commented Jul 16, 2024

codecov-commenter commented Jul 16, 2024 •

edited

Loading

chengjoey commented Jul 17, 2024

chengjoey commented Jul 17, 2024

jberkus commented Jul 18, 2024

chengjoey commented Jul 19, 2024

ahrtr commented Jul 19, 2024

chengjoey commented Jul 20, 2024

chengjoey commented Jul 20, 2024

	var cancel context.CancelFunc
	dctx, cancel = context.WithTimeout(c.ctx, c.cfg.DialTimeout)
	defer cancel() // TODO: Is this right for cases where grpc.WithBlock() is not set on the dial options?
	}

	// grpc.WithBlock to block until connection up or timeout
	testCfgs := []Config{
	{
	Endpoints: []string{"http://254.0.0.1:12345"},
	DialTimeout: 2 * time.Second,
	DialOptions: []grpc.DialOption{grpc.WithBlock()},
	},
	{
	Endpoints: []string{"http://254.0.0.1:12345"},
	DialTimeout: time.Second,
	DialOptions: []grpc.DialOption{grpc.WithBlock()},
	Username: "abc",
	Password: "def",
	},
	}

fix dial-timeout not affected for client watch command #18336

Are you sure you want to change the base?

fix dial-timeout not affected for client watch command #18336

Conversation

chengjoey commented Jul 16, 2024

k8s-ci-robot commented Jul 16, 2024

k8s-ci-robot commented Jul 16, 2024

jmhbnz commented Jul 16, 2024

codecov-commenter commented Jul 16, 2024 • edited Loading

Codecov Report

chengjoey commented Jul 17, 2024

chengjoey commented Jul 17, 2024

jberkus commented Jul 18, 2024

chengjoey commented Jul 19, 2024

ahrtr commented Jul 19, 2024

chengjoey commented Jul 20, 2024

chengjoey commented Jul 20, 2024

codecov-commenter commented Jul 16, 2024 •

edited

Loading