Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle early responses #473

Merged
merged 3 commits into from
May 25, 2018
Merged

Handle early responses #473

merged 3 commits into from
May 25, 2018

Conversation

janko
Copy link
Member

@janko janko commented May 23, 2018

HTTP servers typically wait until the whole request body has been received before sending the response. However, some servers might send out a response early, before they've read the whole request body. For example, they might detect the request headers were invalid, and they want to tell that to the client immediately.

For example, sending a large request to http://google.com will return an early 400 Bad Request response. In this case http.rb will raise an Errno::EPIPE exception (wrapped in a HTTP::ConnectionError), because it tried to write to the socket but it couldn't because the server closed its end. However, the server still returned a valid response, so it's not really erroneous behaviour (e.g. curl doesn't print any errors in the same situations).

# Before
HTTP.get("http://google.com", body: "a" * 1*1024*1024)
#~> Errno::EPIPE

# After
HTTP.get("http://google.com", body: "a" * 1*1024*1024)
#=> #<HTTP::Response 400 Bad Request ...>

This commit changes Request::Writer to stop writing when an Errno::EPIPE exception occurred.

.rubocop.yml Outdated
## Lint ########################################################################

Lint/HandleExceptions:
Enabled: false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we return nil or something in that rescue, Rubocop will be satisfied.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since #send_request is a "command" method, I feel a bit weird explicitly returning a value. I always considered adding a comment to be enough when swallowing errors.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with you, but I'm slightly concerned about disabling this particular cop project-wide. It could catch a real bug someday. Perhaps disabling it just the once with an inline comment is an agreeable compromise?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, updated.

@@ -100,6 +111,8 @@ def write(data)
break unless data.bytesize > length
data = data.byteslice(length..-1)
end
rescue Errno::EPIPE
raise # re-raise Errno::EPIPE
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO this code is pretty self-explanatory and doesn't need comment.

write(chunk)
end
rescue Errno::EPIPE # rubocop:disable Lint/HandleExceptions
# server doesn't need any more data
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO it worth to explicitly add nil at the end of rescue block and remove rubocop disable.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh. It was discussed previously... Mmm... I still think we should just return nil from the rescue. Also body can be shortened:

def send_request
  each_chunk { |chunk| write chunk }
rescue Errno::EPIPE
  # no more data
  nil
end

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On one hand I think that Lint/HandleExceptions is pretty useless and enforces developer to write more than needed. On the other hand I prefer to see:

begin
  # ...
rescue
  nil
end

than

begin
  # ...
rescue
end

IMO it's more readable that way. Easier to understand that nil is result in case of error.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know that one might argue that we don't require explicit nil in case of return if something, but it's a bit different - it's one-liner. And reading it is pretty straight-forward to me. And when we explicitly return from non-one-liner code we add nil to return:

if something
  # ...
  return nil # <-- explicit `nil` here looks better than bare `return`
end

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, updated.

Copy link
Member

@ixti ixti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Please rebase on top of master though.

janko added 3 commits May 25, 2018 23:41
HTTP servers typically wait until the whole request body has been
received before sending the response. However, some servers might send
out a response early, before they've read the whole request body. For
example, they might detect the request headers were invalid, and they
want to tell that to the client immediately.

For example, sending a large request to http://google.com will return an
early `400 Bad Request` response. In this case HTTP.rb will raise an
Errno::EPIPE exception, because it tried to write to the socket but it
couldn't because the server closed its end. However, the server still
returned a valid response, so it's not really erroneous behaviour (e.g.
curl doesn't print any errors in the same situations).

  # Before
  HTTP.get("http://google.com", body: "a" * 1*1024*1024)
  #~> Errno::EPIPE

  # After
  HTTP.get("http://google.com", body: "a" * 1*1024*1024)
  #=> #<HTTP::Response 400 Bad Request ...>

This commit ignores Errno::EPIPE exceptions when writing request data to
the socket.
We want to stop writing after server has closed its end of the pipe (as
no more data will be accepted) and proceed with reading the response.
@janko
Copy link
Member Author

janko commented May 25, 2018

@ixti Done.

@ixti ixti merged commit 55ca249 into httprb:master May 25, 2018
@ixti
Copy link
Member

ixti commented May 25, 2018

Thanks!

@janko janko deleted the handle-early-responses branch May 25, 2018 22:12
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Jun 7, 2020
Update ruby-http to 4.4.1.


## 4.4.1 (2020-03-29)

* Backport [#590](httprb/http#590)
  Fix parser failing on some edge cases.
  ([@ixti])

## 4.4.0 (2020-03-25)

* Backport [#587](httprb/http#587)
  Fix redirections when server responds with multiple Location headers.
  ([@ixti])

* Backport [#599](httprb/http#599)
  Allow passing HTTP::FormData::{Multipart,UrlEncoded} object directly.
  ([@ixti])

## 4.3.0 (2020-01-09)

* Backport [#581](httprb/http#581)
  Add Ruby-2.7 compatibility.
  ([@ixti], [@janko])


## 4.2.0 (2019-10-22)

* Backport [#489](httprb/http#489)
  Fix HTTP parser.
  ([@ixti], [@fxposter])


## 4.1.1 (2019-03-12)

* Add `HTTP::Headers::ACCEPT_ENCODING` constant.
  ([@ixti])


## 4.1.0 (2019-03-11)

* [#533](httprb/http#533)
  Add URI normalizer feature that allows to swap default URI normalizer.
  ([@mamoonraja])


## 4.0.5 (2019-02-15)

* Backport [#532](httprb/http#532) from master.
  Fix pipes support in request bodies.
  ([@ixti])


## 4.0.4 (2019-02-12)

* Backport [#506](httprb/http#506) from master.
  Skip auto-deflate when there is no body.
  ([@Bonias])


## 4.0.3 (2019-01-18)

* Fix missing URL in response wrapped by auto inflate.
  ([@ixti])

* Provide `HTTP::Request#inspect` method for debugging purposes.
  ([@ixti])


## 4.0.2 (2019-01-15)

* [#506](httprb/http#506)
  Fix instrumentation feature.
  ([@paul])


## 4.0.1 (2019-01-14)

* [#515](httprb/http#515)
  Fix `#build_request` and `#request` to respect default options.
  ([@RickCSong])


## 4.0.0 (2018-10-15)

* [#482](httprb/http#482)
  [#499](httprb/http#499)
  Introduce new features injection API with 2 new feaures: instrumentation
  (compatible with ActiveSupport::Notification) and logging.
  ([@paul])

* [#473](httprb/http#473)
  Handle early responses.
  ([@janko-m])

* [#468](httprb/http#468)
  Rewind `HTTP::Request::Body#source` once `#each` is complete.
  ([@ixti])

* [#467](httprb/http#467)
  Drop Ruby 2.2 support.
  ([@ixti])

* [#436](httprb/http#436)
  Raise ConnectionError when writing to socket fails.
  ([@janko-m])

* [#438](httprb/http#438)
  Expose `HTTP::Request::Body#source`.
  ([@janko-m])

* [#446](httprb/http#446)
  Simplify setting a timeout.
  ([@mikegee])

* [#451](httprb/http#451)
  Reduce memory usage when reading response body.
  ([@janko-m])

* [#458](httprb/http#458)
  Extract HTTP::Client#build_request method.
  ([@tycoon])

* [#462](httprb/http#462)
  Fix HTTP::Request#headline to allow two leading slashes in path.
  ([@scarfacedeb])

* [#454](httprb/http#454)
  [#464](httprb/http#464)
  [#384](httprb/http#384)
  Fix #readpartial not respecting max length argument.
  ([@janko-m], [@marshall-lee])
meili-bors bot added a commit to meilisearch/meilisearch-ruby that referenced this pull request Apr 7, 2022
307: Improve error handling, rescue EPIPE and Net::* errors r=brunoocasali a=brunoocasali

# Pull Request
This PR fixes a bad DX when comes to errors, if a user tries to upload a big dataset, they will face some errors, one of them was related to the `Errno::EPIPE` error, which is really hard to figure out.

The idea of this PR is to handle the errors and improve the user experience.

## What does this PR do?
Fixes #305 

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


- [ ] Try something with other lib, which address the pipes error better like: httprb/http#473

Co-authored-by: Bruno Casali <brunoocasali@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants