Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gateway: clean up its surface, and remove BlockList #2874

Merged
merged 1 commit into from
Jun 20, 2016

Conversation

ghost
Copy link

@ghost ghost commented Jun 18, 2016

This patch is in preparation for the gateway's extraction.

It's interesting to trace technical debt back to its
origin, understanding the circumstances in which it
was introduced and built up, and then cutting it back
at exactly the right places.

  • Clean up the gateway's surface
    The option builder GatewayOption() now takes only
    arguments which are relevant for HTTP handler muxing,
    i.e. the paths where the gateway should be mounted.
    All other configuration happens through the
    GatewayConfig object.
  • Remove BlockList
    I know why this was introduced in the first place,
    but it never ended up fulfilling that purpose.
    Somehow it was only ever used by the API server,
    not the gateway, which really doesn't make sense.
    It was also never wired up with CLI nor fs-repo.
    Eventually @krl started punching holes into it
    to make the Web UI accessible.
  • Remove --unrestricted-api
    This was holes being punched into BlockList too,
    for accessing /ipfs and /ipn on the API server.
    With BlockList removed and /ipfs and /ipns freely
    accessible, putting this option out of action
    is safe. With the next major release,
    the option can be removed for good.

License: MIT
Signed-off-by: Lars Gierth larsg@systemli.org

This patch is in preparation for the gateway's extraction.

It's interesting to trace technical debt back to its
origin, understanding the circumstances in which it
was introduced and built up, and then cutting it back
at exactly the right places.

- Clean up the gateway's surface
  The option builder GatewayOption() now takes only
  arguments which are relevant for HTTP handler muxing,
  i.e. the paths where the gateway should be mounted.
  All other configuration happens through the
  GatewayConfig object.

- Remove BlockList
  I know why this was introduced in the first place,
  but it never ended up fulfilling that purpose.
  Somehow it was only ever used by the API server,
  not the gateway, which really doesn't make sense.
  It was also never wired up with CLI nor fs-repo.
  Eventually @krl started punching holes into it
  to make the Web UI accessible.

- Remove --unrestricted-api
  This was holes being punched into BlockList too,
  for accessing /ipfs and /ipn on the API server.
  With BlockList removed and /ipfs and /ipns freely
  accessible, putting this option out of action
  is safe. With the next major release,
  the option can be removed for good.

License: MIT
Signed-off-by: Lars Gierth <larsg@systemli.org>
@ghost ghost added topic/gateway Topic gateway topic/cleanup Topic cleanup labels Jun 18, 2016
if !writableOptionFound {
writable = cfg.Gateway.Writable
if writableOptionFound {
cfg.Gateway.Writable = writable
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we are using defaults, the if block can be removed and just assign to cfg.Gateway.Writable.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mh okay, we shouldn't be using Default(false) with this one. From what I can tell, --writable if present is supposed to override cfg.Gateway.Writable.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So... the second return value (writableOptionFound) is separate from defaults -- it tells specifically whether the option was passed on the CLI.

So this assumption is still true and everything's good as-is:

From what I can tell, --writable if present is supposed to override cfg.Gateway.Writable.

In other words, the default value for --writable is never used -- only if it has actually been passed.

@whyrusleeping
Copy link
Member

overall this LGTM, its cleaner.

@ghost
Copy link
Author

ghost commented Jun 19, 2016

Mh okay, we shouldn't be using Default(false) with this one. From what I can tell, --writable if present is supposed to override cfg.Gateway.Writable.

Gonna investigate this before merging, hold off

@ghost ghost mentioned this pull request Jun 19, 2016
@ghost
Copy link
Author

ghost commented Jun 19, 2016

Gonna investigate this before merging, hold off

All good - RFM!

@ghost ghost added RFM and removed RFM labels Jun 19, 2016
@whyrusleeping whyrusleeping merged commit c6443fd into master Jun 20, 2016
@whyrusleeping whyrusleeping deleted the extract-gateway branch June 20, 2016 19:34
test_expect_success "GET IPFS path on API forbidden" '
test_curl_resp_http_code "http://127.0.0.1:$apiport/ipfs/$HASH" "HTTP/1.1 403 Forbidden"
'

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought this entire PR was reverted. I guess it was not because this test is not in master.

cc @lgierth we need this back, i think.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test has been moved here https://github.com/ipfs/go-ipfs/blob/master/test/sharness/t0400-api-security.sh (because it's not a gateway test)

@jbenet
Copy link
Member

jbenet commented Aug 27, 2016

Were there other changes to gateway code in this release?

If not, I think i would feel safer if all gateway changes were rebased out and placed at the end together so we can analyze them very clearly. (i guess we can diff the whole release in these files, but making sure this is safe may be worth rewriting history over...)

@ghost
Copy link
Author

ghost commented Aug 27, 2016

It should have all been reverted by 0.4.3-rc1, as part of fixing the KeyThief vuln. Well not reverted, but reintroduced, in the case of restricted api and --unrestricted-api. #2949 and #2956

@jbenet
Copy link
Member

jbenet commented Aug 28, 2016

Right but my question is: how do we know the behaviors are all back to the
same? It may be worth reverting and doing the proper gateway extraction
separately, and all in a larger patch set? Not sure. I think this area
lacks tests to feel super comfortable with it.
On Sat, Aug 27, 2016 at 07:45 Lars Gierth notifications@github.com wrote:

It should have all been reverted by 0.4.3-rc1, as part of fixing the
KeyThief vuln. Well not reverted, but reintroduced, in the case of
restricted api and --unrestricted-api. #2949
#2949 and #2956
#2956


You are receiving this because you commented.

Reply to this email directly, view it on GitHub
#2874 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAIcoYcnKTb_L1CEqM6nTxqGFlxavD9Cks5qkCNdgaJpZM4I5DOI
.

@ghost
Copy link
Author

ghost commented Aug 30, 2016

Right but my question is: how do we know the behaviors are all back to the same? It may be worth reverting and doing the proper gateway extraction separately, and all in a larger patch set? Not sure. I think this area lacks tests to feel super comfortable with it.

But, but... I removed lots of code and the blocking logic is literally 7 lines of code now. [1] It wires --unrestricted-api to this logic directly, without any intermediary function calls. BlockList was actually a pretty hefty abstraction and made a very simple knob complicated . The tests are now in a place that makes sense, comments were added everywhere.

I think it's is a net win, making this thing much more obvious. Its obscurity was the reason why I was able to be pretty sure it was dead code, and introduce KeyThief.

[1] daemon.go
[2] t0400-api-security.sh

@ghost
Copy link
Author

ghost commented Aug 30, 2016

and introduce KeyThief

well I not actually introduced it, but at least opened the gates. Anyway that's a different topic :)

@jbenet jbenet mentioned this pull request Aug 30, 2016
58 tasks
@ghost ghost mentioned this pull request Sep 11, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic/cleanup Topic cleanup topic/gateway Topic gateway
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants