Fix fetch parameters not being applied correctly #1870

xconverge · 2023-01-19T23:45:45Z

lib/fetch/index.js

xconverge · 2023-01-20T00:11:08Z

Not sure how this directly undoes #1386 and the test still passes, will look into it and see if I am missing something obvious?

xconverge · 2023-01-20T05:15:27Z

lib/agent.js

@@ -81,6 +81,10 @@ class Agent extends DispatcherBase {
    }
  }

+  options () {


this seems a little hacky, any thoughts?

This would need a bit more support from all the available agents.
Otherwise, I'm good with it.

mcollina · 2023-01-20T08:31:54Z

@ronag @KhafraDev why is fetch() completely overriding the agent options? I think it'd be best to stick to what is configured in the agent.

(I think I missed #1386, otherwise I would have objected then. It was a busy period of my life.).

codecov-commenter · 2023-01-20T08:36:05Z

Codecov Report

Base: 90.35% // Head: 90.33% // Decreases project coverage by -0.02% ⚠️

Coverage data is based on head (2c0caef) compared to base (2d2512c).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1870      +/-   ##
==========================================
- Coverage   90.35%   90.33%   -0.02%     
==========================================
  Files          70       70              
  Lines        6042     6043       +1     
==========================================
  Hits         5459     5459              
- Misses        583      584       +1

Impacted Files	Coverage Δ
lib/fetch/index.js	`84.83% <ø> (ø)`
lib/agent.js	`100.00% <100.00%> (ø)`
lib/fetch/file.js	`89.65% <0.00%> (-1.15%)`	⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

ronag

I think the fix here is simply that fetch doesn't pass it's own timeout values. Why do we override them in fetch?

KhafraDev · 2023-01-20T14:56:25Z

They are overridden in fetch to match chrome's behavior. I agree that the fix is to remove them.

edit: #1373

xconverge · 2023-01-20T16:01:12Z

Do I need to adjust any defaults anywhere to preserve behavior or simply delete the 2 lines in lib/fetch/index.js? And remove my 300 sec test that was aiming to enforce the previous feature request?

KhafraDev · 2023-01-20T16:06:30Z

removing the 2 lines is fine, the other changes should also be removed

xconverge · 2023-01-20T16:29:37Z

Ok this is ready for a re-review, it was my original changeset as well until I realized it was undo-ing #1373 and #1386

If you want me to add the 300 second tests back in from my previous commit and change the defaults elsewhere in the agent, please let me know, but for now at least the ability to override in fetch exists/works and a test that makes sure of that is in (unless the default gets changed to a much larger number in the future)

xconverge · 2023-01-20T16:57:42Z

Oops sorry I didn't know clicking re-request review on a change request removes the other reviewers

ronag

LGTM once test pass

mcollina

lgtm

mcollina · 2023-01-20T08:29:24Z

lib/agent.js

@@ -81,6 +81,10 @@ class Agent extends DispatcherBase {
    }
  }

+  options () {


This would need a bit more support from all the available agents.
Otherwise, I'm good with it.

xconverge · 2023-01-25T20:14:53Z

@mcollina @ronag what happens if you set bodyTimeout and headersTimeout to 20 minutes, but do NOT set keepAliveTimeout (default is 10 minutes), and a request takes say 15 minutes.

Looking at this code, it kind of looks to me like the request will just die and eventually timeout at the 20 minutes and return a bodyTimeout or headersTimeout. This is what I am experiencing (I know this is wack to have such long requests... but I am starting by just porting this request from axios which works)

    if (this.shouldKeepAlive && client[kPipelining]) {
      const keepAliveTimeout = this.keepAlive ? util.parseKeepAliveTimeout(this.keepAlive) : null

      if (keepAliveTimeout != null) {
        const timeout = Math.min(
          keepAliveTimeout - client[kKeepAliveTimeoutThreshold],
          client[kKeepAliveMaxTimeout]
        )
        if (timeout <= 0) {
          socket[kReset] = true
        } else {
          client[kKeepAliveTimeoutValue] = timeout
        }
      } else {
        client[kKeepAliveTimeoutValue] = client[kKeepAliveDefaultTimeout]
      }
    } else {
      // Stop more requests from being dispatched.
      socket[kReset] = true
    }

I will increase setting keepAliveTimeout but am wondering if there is a more elegant way to handle this so that the errors I get make sense

mcollina · 2023-01-25T21:08:02Z

I'm failing to understand how that relates to this PR. If not, please open a new issue.

xconverge · 2023-01-25T22:25:32Z

I'm failing to understand how that relates to this PR. If not, please open a new issue.

Ok fair enough, I have little to no information to put in a new issue yet so was hoping to just get some guidance. I was hopeful that this PR fixed my long request issue and it does not, so to me there is something about undici that I continue to not understand for a long request. I am glad I was able to write a tangible test for this particular issue, so I will see if I can figure out how to write a test case for what I am still experiencing.

xconverge · 2023-01-26T17:33:23Z

This is why I thought it was related, because it allowed me to expose the next issue basically #1887

* Fix default fetch parameters * Preserve existing behavior with 300 second timeout if not defined * Add test for 300 second timeout as default * Cleanup old unused tests * Simplify how fetch utilizes timeouts from agent

Fix default fetch parameters

8f16b85

xconverge commented Jan 20, 2023

View reviewed changes

lib/fetch/index.js Show resolved Hide resolved

xconverge added 3 commits January 19, 2023 16:49

Preserve existing behavior with 300 second timeout if not defined

0a666a2

Add test for 300 second timeout as default

db67229

Cleanup old unused tests

2c0caef

xconverge commented Jan 20, 2023

View reviewed changes

mcollina requested a review from KhafraDev January 20, 2023 08:27

ronag requested changes Jan 20, 2023

View reviewed changes

Simplify how fetch utilizes timeouts from agent

1e3d129

xconverge requested review from ronag and removed request for KhafraDev January 20, 2023 16:56

KhafraDev approved these changes Jan 20, 2023

View reviewed changes

ronag approved these changes Jan 20, 2023

View reviewed changes

mcollina approved these changes Jan 20, 2023

View reviewed changes

mcollina merged commit d6ba9e5 into nodejs:main Jan 20, 2023

xconverge deleted the testLongRequestTimeout branch January 20, 2023 17:45

ronag mentioned this pull request Feb 10, 2023

Default fetch timeout too short #1373

Open

mccaig mentioned this pull request Feb 18, 2023

18.14.1 release altered fetch timeout defaults from 300s to 30s nodejs/node#46706

Closed

metcoder95 mentioned this pull request Apr 12, 2023

feat: Add H2 support metcoder95/undici#33

Closed

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix fetch parameters not being applied correctly #1870

Fix fetch parameters not being applied correctly #1870

xconverge commented Jan 19, 2023

xconverge commented Jan 20, 2023

xconverge Jan 20, 2023

mcollina Jan 20, 2023

mcollina commented Jan 20, 2023

codecov-commenter commented Jan 20, 2023 •

edited

Loading

ronag left a comment

KhafraDev commented Jan 20, 2023 •

edited

Loading

xconverge commented Jan 20, 2023

KhafraDev commented Jan 20, 2023 •

edited

Loading

xconverge commented Jan 20, 2023

xconverge commented Jan 20, 2023

ronag left a comment

mcollina left a comment

mcollina Jan 20, 2023

xconverge commented Jan 25, 2023

mcollina commented Jan 25, 2023

xconverge commented Jan 25, 2023

xconverge commented Jan 26, 2023

Fix fetch parameters not being applied correctly #1870

Fix fetch parameters not being applied correctly #1870

Conversation

xconverge commented Jan 19, 2023

xconverge commented Jan 20, 2023

xconverge Jan 20, 2023

Choose a reason for hiding this comment

mcollina Jan 20, 2023

Choose a reason for hiding this comment

mcollina commented Jan 20, 2023

codecov-commenter commented Jan 20, 2023 • edited Loading

Codecov Report

ronag left a comment

Choose a reason for hiding this comment

KhafraDev commented Jan 20, 2023 • edited Loading

xconverge commented Jan 20, 2023

KhafraDev commented Jan 20, 2023 • edited Loading

xconverge commented Jan 20, 2023

xconverge commented Jan 20, 2023

ronag left a comment

Choose a reason for hiding this comment

mcollina left a comment

Choose a reason for hiding this comment

mcollina Jan 20, 2023

Choose a reason for hiding this comment

xconverge commented Jan 25, 2023

mcollina commented Jan 25, 2023

xconverge commented Jan 25, 2023

xconverge commented Jan 26, 2023

codecov-commenter commented Jan 20, 2023 •

edited

Loading

KhafraDev commented Jan 20, 2023 •

edited

Loading

KhafraDev commented Jan 20, 2023 •

edited

Loading