Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add negotiation process for driver scope #516

Merged
merged 1 commit into from
Sep 15, 2015
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions docs/remote.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,16 @@ When loaded, a remote driver process receives an HTTP POST on the URL `/Plugin.A

Other entries in the list value are allowed; `"NetworkDriver"` indicates that the plugin should be registered with LibNetwork as a driver.

### Set capability

After Handshake, the remote driver will receive another POST message to the URL `/NetworkDriver.GetCapabilities` with no payload. The driver's response should have the form:

{
"Scope": "local"
}

Value of "Scope" should be either "local" or "global" which indicates the capability of remote driver, values beyond these will fail driver's registration and return an error to the caller.

### Create network

When the proxy is asked to create a network, the remote process shall receive a POST to the URL `/NetworkDriver.CreateNetwork` of the form
Expand Down
6 changes: 6 additions & 0 deletions drivers/remote/api/api.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,12 @@ func (r *Response) GetError() string {
return r.Err
}

// GetCapabilityResponse is the response of GetCapability request
type GetCapabilityResponse struct {
Response
Scope string
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Scope is the only capability that we are interested in.
Just want to make sure that when we add more capabilities, libnetwork must not fail in the future if the users are using an older version of plugin that might not have implemented the newly added capabilities. We just have to make sure that the driver.call function doesn't fail.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, got it. :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it possible to add a test case for this specifically ?
I expect more changes to the capability structure and want to ensure backward compatibility for any such changes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just found that to make use of driver.call function, the GetCapabilityResponse need to contain Response, without this, I need to write some duplicate codes as driver.call.
So I think maybe we'd better keep this although the error is not what we are interested in.

}

// CreateNetworkRequest requests a new network.
type CreateNetworkRequest struct {
// A network ID that remote plugins are expected to store for future
Expand Down
30 changes: 27 additions & 3 deletions drivers/remote/driver.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,16 +28,40 @@ func newDriver(name string, client *plugins.Client) driverapi.Driver {
// plugin is activated.
func Init(dc driverapi.DriverCallback) error {
plugins.Handle(driverapi.NetworkPluginEndpointType, func(name string, client *plugins.Client) {
c := driverapi.Capability{
Scope: driverapi.GlobalScope,
// negotiate driver capability with client
d := newDriver(name, client)
c, err := d.(*driver).getCapabilities()
if err != nil {
log.Errorf("error getting capability for %s due to %v", name, err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you think it will be better to default the scope to be GlobalScope if the driver hasn't implemented GetCapability ?
That will atleast provide a compatibility with the existing drivers. WDYT ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I noticed this.
On one hand, we have changed the plugin's registration process, and I think the existing plugin should make some relevant modifications to adapt this; one the other hand, I'm totally agreed with you about the compatibility things.
So, if you think to default the scope to be GlobalScope is better, I'm totally OK with that. :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There have been other breaking changes, so I don't think changing the protocol is inherently a problem.

You could argue that the robustness principle indicates that treating a 404 from the plugin as the default is reasonable. I'm not sure all plugins will return a 404 though -- from memory I don't think it's part of the plugin protocol spec, so there's a chance non-conforming plugins will return an error code, and be left in an unknown state. With that in mind I think it's better to break backwards compatibility and treat all non-conforming responses as errors.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @squaremo. that make sense.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, @squaremo gives a really convincing reason 👍

return
}
if err := dc.RegisterDriver(name, newDriver(name, client), c); err != nil {
if err = dc.RegisterDriver(name, d, *c); err != nil {
log.Errorf("error registering driver for %s due to %v", name, err)
}
})
return nil
}

// Get capability from client
func (d *driver) getCapabilities() (*driverapi.Capability, error) {
var capResp api.GetCapabilityResponse
if err := d.call("GetCapabilities", nil, &capResp); err != nil {
return nil, err
}

c := &driverapi.Capability{}
switch capResp.Scope {
case "global":
c.Scope = driverapi.GlobalScope
case "local":
c.Scope = driverapi.LocalScope
default:
return nil, fmt.Errorf("invalid capability: expecting 'local' or 'global', got %s", capResp.Scope)
}

return c, nil
}

// Config is not implemented for remote drivers, since it is assumed
// to be supplied to the remote process out-of-band (e.g., as command
// line arguments).
Expand Down
115 changes: 106 additions & 9 deletions drivers/remote/driver_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,91 @@ func (test *testEndpoint) AddStaticRoute(destination *net.IPNet, routeType int,
return nil
}

func TestGetEmptyCapabilities(t *testing.T) {
var plugin = "test-net-driver-empty-cap"

mux := http.NewServeMux()
defer setupPlugin(t, plugin, mux)()

handle(t, mux, "GetCapabilities", func(msg map[string]interface{}) interface{} {
return map[string]interface{}{}
})

p, err := plugins.Get(plugin, driverapi.NetworkPluginEndpointType)
if err != nil {
t.Fatal(err)
}

d := newDriver(plugin, p.Client)
if d.Type() != plugin {
t.Fatal("Driver type does not match that given")
}

_, err = d.(*driver).getCapabilities()
if err == nil {
t.Fatal("There should be error reported when get empty capability")
}
}

func TestGetExtraCapabilities(t *testing.T) {
var plugin = "test-net-driver-extra-cap"

mux := http.NewServeMux()
defer setupPlugin(t, plugin, mux)()

handle(t, mux, "GetCapabilities", func(msg map[string]interface{}) interface{} {
return map[string]interface{}{
"Scope": "local",
"foo": "bar",
}
})

p, err := plugins.Get(plugin, driverapi.NetworkPluginEndpointType)
if err != nil {
t.Fatal(err)
}

d := newDriver(plugin, p.Client)
if d.Type() != plugin {
t.Fatal("Driver type does not match that given")
}

c, err := d.(*driver).getCapabilities()
if err != nil {
t.Fatal(err)
} else if c.Scope != driverapi.LocalScope {
t.Fatalf("get capability '%s', expecting 'local'", c.Scope)
}
}

func TestGetInvalidCapabilities(t *testing.T) {
var plugin = "test-net-driver-invalid-cap"

mux := http.NewServeMux()
defer setupPlugin(t, plugin, mux)()

handle(t, mux, "GetCapabilities", func(msg map[string]interface{}) interface{} {
return map[string]interface{}{
"Scope": "fake",
}
})

p, err := plugins.Get(plugin, driverapi.NetworkPluginEndpointType)
if err != nil {
t.Fatal(err)
}

d := newDriver(plugin, p.Client)
if d.Type() != plugin {
t.Fatal("Driver type does not match that given")
}

_, err = d.(*driver).getCapabilities()
if err == nil {
t.Fatal("There should be error reported when get invalid capability")
}
}

func TestRemoteDriver(t *testing.T) {
var plugin = "test-net-driver"

Expand All @@ -177,6 +262,11 @@ func TestRemoteDriver(t *testing.T) {

var networkID string

handle(t, mux, "GetCapabilities", func(msg map[string]interface{}) interface{} {
return map[string]interface{}{
"Scope": "global",
}
})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for adding this test. in addition to this positive case, can you please add a couple of negative cases which am interested in seeing for backward compatibility ?

  1. empty return : return map[string]interface{}{ }
  2. return with extra params : return map[string]interface{}{ "Scope": "global", "foo":"bar" }

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@WeiZhang555 My previous comment got overwritten due to the recent changes.

thanks for adding this test. in addition to this positive case, can you please add a couple of negative cases which am interested in seeing for backward compatibility ?

  1. empty return : return map[string]interface{}{ }
  2. return with extra params : return map[string]interface{}{ "Scope": "global", "foo":"bar" }

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All right, I can do that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @WeiZhang555 appreciate it.

handle(t, mux, "CreateNetwork", func(msg map[string]interface{}) interface{} {
nid := msg["NetworkID"]
var ok bool
Expand Down Expand Up @@ -245,38 +335,45 @@ func TestRemoteDriver(t *testing.T) {
t.Fatal(err)
}

driver := newDriver(plugin, p.Client)
if driver.Type() != plugin {
d := newDriver(plugin, p.Client)
if d.Type() != plugin {
t.Fatal("Driver type does not match that given")
}

c, err := d.(*driver).getCapabilities()
if err != nil {
t.Fatal(err)
} else if c.Scope != driverapi.GlobalScope {
t.Fatalf("get capability '%s', expecting 'global'", c.Scope)
}

netID := "dummy-network"
err = driver.CreateNetwork(netID, map[string]interface{}{})
err = d.CreateNetwork(netID, map[string]interface{}{})
if err != nil {
t.Fatal(err)
}

endID := "dummy-endpoint"
err = driver.CreateEndpoint(netID, endID, ep, map[string]interface{}{})
err = d.CreateEndpoint(netID, endID, ep, map[string]interface{}{})
if err != nil {
t.Fatal(err)
}

joinOpts := map[string]interface{}{"foo": "fooValue"}
err = driver.Join(netID, endID, "sandbox-key", ep, joinOpts)
err = d.Join(netID, endID, "sandbox-key", ep, joinOpts)
if err != nil {
t.Fatal(err)
}
if _, err = driver.EndpointOperInfo(netID, endID); err != nil {
if _, err = d.EndpointOperInfo(netID, endID); err != nil {
t.Fatal(err)
}
if err = driver.Leave(netID, endID); err != nil {
if err = d.Leave(netID, endID); err != nil {
t.Fatal(err)
}
if err = driver.DeleteEndpoint(netID, endID); err != nil {
if err = d.DeleteEndpoint(netID, endID); err != nil {
t.Fatal(err)
}
if err = driver.DeleteNetwork(netID); err != nil {
if err = d.DeleteNetwork(netID); err != nil {
t.Fatal(err)
}
}
Expand Down