-
Notifications
You must be signed in to change notification settings - Fork 917
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ftr: Use invoker with same ip as client first. #1023
Merged
AlexStocks
merged 22 commits into
apache:develop
from
LaurenceLiZhixin:Ftr/selfDiscovery
Feb 25, 2021
Merged
Changes from 11 commits
Commits
Show all changes
22 commits
Select commit
Hold shift + click to select a range
b6022bb
fix ctx linter error
AlexStocks d305b67
Merge remote-tracking branch 'upstream/develop' into develop
LaurenceLiZhixin 5919b11
Merge remote-tracking branch 'upstream/develop' into develop
LaurenceLiZhixin 3dc2f17
Merge remote-tracking branch 'upstream/develop' into develop
LaurenceLiZhixin 55737aa
Merge remote-tracking branch 'upstream/develop' into develop
LaurenceLiZhixin 17c4a9e
Merge pull request #1015 from apache/dependabot/go_modules/develop/gi…
AlexStocks 65cd856
Merge remote-tracking branch 'upstream/develop' into develop
LaurenceLiZhixin 8414d85
feat: add ip level self discovery
LaurenceLiZhixin 512b6c6
fix: change comment
LaurenceLiZhixin 6a6e805
fix: change model to self
LaurenceLiZhixin 1353c33
fix: delete unused gost version
LaurenceLiZhixin c698ef2
Merge remote-tracking branch 'upstream/develop' into develop
LaurenceLiZhixin e65c046
Merge branch 'develop' into Ftr/selfDiscovery
LaurenceLiZhixin b04e7c1
fix: fmt project
LaurenceLiZhixin 17bfc3a
fix: go mod tidy
LaurenceLiZhixin a5b3ed4
fix: change directory name
LaurenceLiZhixin 5399c8e
fix: delete item in go sun
LaurenceLiZhixin d826f34
fix
LaurenceLiZhixin 58a3fba
fix: change name to self priority router
LaurenceLiZhixin 81b5d08
fix: change name to self priority router
LaurenceLiZhixin 152d4dd
fix: change name to self priority router
LaurenceLiZhixin e1b52d0
fix: change name
LaurenceLiZhixin File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,43 @@ | ||
/* | ||
* Licensed to the Apache Software Foundation (ASF) under one or more | ||
* contributor license agreements. See the NOTICE file distributed with | ||
* this work for additional information regarding copyright ownership. | ||
* The ASF licenses this file to You under the Apache License, Version 2.0 | ||
* (the "License"); you may not use this file except in compliance with | ||
* the License. You may obtain a copy of the License at | ||
* | ||
* http://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, software | ||
* distributed under the License is distributed on an "AS IS" BASIS, | ||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
* See the License for the specific language governing permissions and | ||
* limitations under the License. | ||
*/ | ||
|
||
package self | ||
|
||
import ( | ||
"github.com/apache/dubbo-go/cluster/router" | ||
"github.com/apache/dubbo-go/common" | ||
"github.com/apache/dubbo-go/common/constant" | ||
"github.com/apache/dubbo-go/common/extension" | ||
) | ||
|
||
func init() { | ||
extension.SetRouterFactory(constant.SelfDiscoveryRouterName, newSelfDiscRouteFactory) | ||
} | ||
|
||
// SelfDiscRouteFactory | ||
type SelfDiscRouteFactory struct { | ||
} | ||
|
||
// newSelfDiscRouteFactory construct a new SelfDiscRouteFactory | ||
func newSelfDiscRouteFactory() router.PriorityRouterFactory { | ||
return &SelfDiscRouteFactory{} | ||
} | ||
|
||
// NewPriorityRouter construct a new NewSelfDiscRouter via url | ||
func (f *SelfDiscRouteFactory) NewPriorityRouter(url *common.URL) (router.PriorityRouter, error) { | ||
return NewSelfDiscRouter(url) | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,74 @@ | ||
/* | ||
* Licensed to the Apache Software Foundation (ASF) under one or more | ||
* contributor license agreements. See the NOTICE file distributed with | ||
* this work for additional information regarding copyright ownership. | ||
* The ASF licenses this file to You under the Apache License, Version 2.0 | ||
* (the "License"); you may not use this file except in compliance with | ||
* the License. You may obtain a copy of the License at | ||
* | ||
* http://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, software | ||
* distributed under the License is distributed on an "AS IS" BASIS, | ||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
* See the License for the specific language governing permissions and | ||
* limitations under the License. | ||
*/ | ||
|
||
package self | ||
|
||
import ( | ||
"context" | ||
"testing" | ||
) | ||
|
||
import ( | ||
"github.com/stretchr/testify/assert" | ||
) | ||
|
||
import ( | ||
"github.com/apache/dubbo-go/common" | ||
"github.com/apache/dubbo-go/protocol" | ||
) | ||
|
||
// nolint | ||
type MockInvoker struct { | ||
url *common.URL | ||
} | ||
|
||
// nolint | ||
func NewMockInvoker(url *common.URL) *MockInvoker { | ||
return &MockInvoker{ | ||
url: url, | ||
} | ||
} | ||
|
||
// nolint | ||
func (bi *MockInvoker) GetUrl() *common.URL { | ||
return bi.url | ||
} | ||
|
||
// nolint | ||
func (bi *MockInvoker) IsAvailable() bool { | ||
return true | ||
} | ||
|
||
// nolint | ||
func (bi *MockInvoker) IsDestroyed() bool { | ||
return true | ||
} | ||
|
||
// nolint | ||
func (bi *MockInvoker) Invoke(_ context.Context, _ protocol.Invocation) protocol.Result { | ||
return nil | ||
} | ||
|
||
// nolint | ||
func (bi *MockInvoker) Destroy() { | ||
} | ||
|
||
// nolint | ||
func TestSelfDiscRouteFactory(t *testing.T) { | ||
factory := newSelfDiscRouteFactory() | ||
assert.NotNil(t, factory) | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,106 @@ | ||
/* | ||
* Licensed to the Apache Software Foundation (ASF) under one or more | ||
* contributor license agreements. See the NOTICE file distributed with | ||
* this work for additional information regarding copyright ownership. | ||
* The ASF licenses this file to You under the Apache License, Version 2.0 | ||
* (the "License"); you may not use this file except in compliance with | ||
* the License. You may obtain a copy of the License at | ||
* | ||
* http://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, software | ||
* distributed under the License is distributed on an "AS IS" BASIS, | ||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
* See the License for the specific language governing permissions and | ||
* limitations under the License. | ||
*/ | ||
|
||
package self | ||
|
||
import ( | ||
"github.com/RoaringBitmap/roaring" | ||
) | ||
|
||
import ( | ||
"github.com/apache/dubbo-go/cluster/router" | ||
"github.com/apache/dubbo-go/cluster/router/utils" | ||
"github.com/apache/dubbo-go/common" | ||
"github.com/apache/dubbo-go/common/logger" | ||
"github.com/apache/dubbo-go/protocol" | ||
) | ||
|
||
const ( | ||
selfDesc = "self-desc" | ||
name = "self-desc-router" | ||
) | ||
|
||
// SelfDiscRouter provides a ip-same-first routing logic | ||
// if there is not provider with same ip as consumer, it would not filter any invoker | ||
// if exists same ip invoker, it would retains this invoker only | ||
type SelfDiscRouter struct { | ||
url *common.URL | ||
localIP string | ||
} | ||
|
||
// NewSelfDiscRouter construct an SelfDiscRouter via url | ||
func NewSelfDiscRouter(url *common.URL) (router.PriorityRouter, error) { | ||
r := &SelfDiscRouter{ | ||
url: url, | ||
localIP: url.Ip, | ||
} | ||
return r, nil | ||
} | ||
|
||
// Route gets a list of match-logic invoker | ||
func (r *SelfDiscRouter) Route(invokers *roaring.Bitmap, cache router.Cache, url *common.URL, invocation protocol.Invocation) *roaring.Bitmap { | ||
addrPool := cache.FindAddrPool(r) | ||
// Add selfDesc invoker to the list | ||
selectedInvokers := utils.JoinIfNotEqual(addrPool[selfDesc], invokers) | ||
// If all invokers are considered not match, downgrade to all invoker | ||
if selectedInvokers.IsEmpty() { | ||
logger.Warnf(" Now all invokers are not match, so downgraded to all! Service: [%s]", url.ServiceKey()) | ||
return invokers | ||
} | ||
return selectedInvokers | ||
} | ||
|
||
// Pool separates same ip invoker from others. | ||
func (r *SelfDiscRouter) Pool(invokers []protocol.Invoker) (router.AddrPool, router.AddrMetadata) { | ||
rb := make(router.AddrPool, 8) | ||
rb[selfDesc] = roaring.NewBitmap() | ||
selfDescFound := false | ||
for i, invoker := range invokers { | ||
if invoker.GetUrl().Ip == r.localIP { | ||
rb[selfDesc].Add(uint32(i)) | ||
selfDescFound = true | ||
} | ||
} | ||
if selfDescFound { | ||
// found self desc | ||
logger.Debug("found self desc ") | ||
return rb, nil | ||
} | ||
for i, _ := range invokers { | ||
rb[selfDesc].Add(uint32(i)) | ||
} | ||
return rb, nil | ||
} | ||
|
||
// ShouldPool will always return true to make sure self call logic constantly. | ||
func (r *SelfDiscRouter) ShouldPool() bool { | ||
return true | ||
} | ||
|
||
func (r *SelfDiscRouter) Name() string { | ||
return name | ||
} | ||
|
||
// Priority | ||
func (r *SelfDiscRouter) Priority() int64 { | ||
return 0 | ||
} | ||
|
||
// URL Return URL in router | ||
func (r *SelfDiscRouter) URL() *common.URL { | ||
return r.url | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,76 @@ | ||
/* | ||
* Licensed to the Apache Software Foundation (ASF) under one or more | ||
* contributor license agreements. See the NOTICE file distributed with | ||
* this work for additional information regarding copyright ownership. | ||
* The ASF licenses this file to You under the Apache License, Version 2.0 | ||
* (the "License"); you may not use this file except in compliance with | ||
* the License. You may obtain a copy of the License at | ||
* | ||
* http://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, software | ||
* distributed under the License is distributed on an "AS IS" BASIS, | ||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
* See the License for the specific language governing permissions and | ||
* limitations under the License. | ||
*/ | ||
|
||
package self | ||
|
||
import ( | ||
"fmt" | ||
"testing" | ||
) | ||
|
||
import ( | ||
"github.com/stretchr/testify/assert" | ||
) | ||
|
||
import ( | ||
"github.com/apache/dubbo-go/cluster/router" | ||
"github.com/apache/dubbo-go/cluster/router/chain" | ||
"github.com/apache/dubbo-go/cluster/router/utils" | ||
"github.com/apache/dubbo-go/common" | ||
"github.com/apache/dubbo-go/protocol" | ||
"github.com/apache/dubbo-go/protocol/invocation" | ||
) | ||
|
||
const ( | ||
selfDiscRoute1010IP = "192.168.10.10" | ||
selfDiscRoute1011IP = "192.168.10.11" | ||
selfDiscRoute1012IP = "192.168.10.12" | ||
selfDiscRouteMethodNameTest = "test" | ||
selfDiscRouteUrlFormat = "dubbo://%s:20000/com.ikurento.user.UserProvider" | ||
) | ||
|
||
func TestSelfDiscRouterRoute(t *testing.T) { | ||
defer protocol.CleanAllStatus() | ||
consumerURL, _ := common.NewURL(fmt.Sprintf(selfDiscRouteUrlFormat, selfDiscRoute1010IP)) | ||
url1, _ := common.NewURL(fmt.Sprintf(selfDiscRouteUrlFormat, selfDiscRoute1010IP)) | ||
url2, _ := common.NewURL(fmt.Sprintf(selfDiscRouteUrlFormat, selfDiscRoute1011IP)) | ||
url3, _ := common.NewURL(fmt.Sprintf(selfDiscRouteUrlFormat, selfDiscRoute1012IP)) | ||
hcr, _ := NewSelfDiscRouter(consumerURL) | ||
|
||
var invokers []protocol.Invoker | ||
invoker1 := NewMockInvoker(url1) | ||
invoker2 := NewMockInvoker(url2) | ||
invoker3 := NewMockInvoker(url3) | ||
invokers = append(invokers, invoker1, invoker2, invoker3) | ||
inv := invocation.NewRPCInvocation(selfDiscRouteMethodNameTest, nil, nil) | ||
res := hcr.Route(utils.ToBitmap(invokers), setUpAddrCache(hcr.(*SelfDiscRouter), invokers), consumerURL, inv) | ||
// now only same ip invoker is selected | ||
assert.True(t, len(res.ToArray()) == 1) | ||
|
||
// now all invoker with ip that not match client are selected | ||
invokers = invokers[1:] | ||
res = hcr.Route(utils.ToBitmap(invokers), setUpAddrCache(hcr.(*SelfDiscRouter), invokers), consumerURL, inv) | ||
assert.True(t, len(res.ToArray()) == 2) | ||
} | ||
|
||
func setUpAddrCache(r router.Poolable, addrs []protocol.Invoker) router.Cache { | ||
pool, info := r.Pool(addrs) | ||
cache := chain.BuildCache(addrs) | ||
cache.SetAddrMeta(r.Name(), info) | ||
cache.SetAddrPool(r.Name(), pool) | ||
return cache | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -176,6 +176,8 @@ github.com/dubbogo/go-zookeeper v1.0.2/go.mod h1:fn6n2CAEer3novYgk9ULLwAjuV8/g4D | |
github.com/dubbogo/gost v1.9.0/go.mod h1:pPTjVyoJan3aPxBPNUX0ADkXjPibLo+/Ib0/fADXSG8= | ||
github.com/dubbogo/gost v1.10.1 h1:39kF9Cd5JOiMpmwG6dX1/aLWNFqFv9gHp8HrhzMmjLY= | ||
github.com/dubbogo/gost v1.10.1/go.mod h1:+mQGS51XQEUWZP2JeGZTxJwipjRKtJO7Tr+FOg+72rI= | ||
github.com/dubbogo/gost v1.11.0 h1:9KtyWQz1gMlAfwzen5iyhMdoe08SPBBUVhco4rdgJ9I= | ||
github.com/dubbogo/gost v1.11.0/go.mod h1:w8Yw29eDWtRVo3tx9nPpHkNZnOi4SRx1fZf7eVlAAU4= | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. using There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. fixed |
||
github.com/dubbogo/jsonparser v1.0.1/go.mod h1:tYAtpctvSP/tWw4MeelsowSPgXQRVHHWbqL6ynps8jU= | ||
github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk= | ||
github.com/dustin/go-humanize v1.0.0 h1:VSnTsYCnlFHaM2/igO1h6X3HA71jcobQuxemgkq4zYo= | ||
|
@@ -474,7 +476,9 @@ github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7 | |
github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU= | ||
github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w= | ||
github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM= | ||
github.com/k0kubun/colorstring v0.0.0-20150214042306-9440f1994b88 h1:uC1QfSlInpQF+M0ao65imhwqKnz3Q2z/d8PWZRMQvDM= | ||
github.com/k0kubun/colorstring v0.0.0-20150214042306-9440f1994b88/go.mod h1:3w7q1U84EfirKl04SVQ/s7nPm1ZPhiXd34z40TNz36k= | ||
github.com/k0kubun/pp v3.0.1+incompatible h1:3tqvf7QgUnZ5tXO6pNAZlrvHgl6DvifjDrd9g2S9Z40= | ||
github.com/k0kubun/pp v3.0.1+incompatible/go.mod h1:GWse8YhT0p8pT4ir3ZgBbfZild3tgzSScAn6HmfYukg= | ||
github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q= | ||
github.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00= | ||
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you use
common.SelfDiscoveryRouterName
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
common.SelfDiscoveryRouterName
is to find out registered router.selfDesc
is to find the bitmap of target router in cache.In order to be consistent with the code design of other routers, I think your suggesting way can be confusing.