fix: Handle 406 http errors

[haleemur]

We have been noticing 406 errors when running discovery, specifically on the .../describe endpoint.

requests.exceptions.HTTPError: 406 Client Error: CustomNotAcceptable for url: https://XXX.salesforce.com/services/data/v53.0/sobjects/XXX/describe

This MR adds error handling for this. The error code 406 is new to me, so I wanted to look up what that meant. Salesforce's error codes page does not even document it. https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/errorcodes.htm

From the various pieces of literature I've read, it seems that the way to correct this issue is to set one of the accept, accept-encoding & accept-language appropriately, however we have not been providing alternate values for these headers and the error is encountered intermittently.

My best guess is that a separate issue occurs, but salesforce does not report the actual error, and instead we are returned a 406, which is not very meaningful. To recover from these failures, we simply rerun the taps. However, adding a retry specific to this error code seems like a better approach and it will benefit the wider meltano / singer community.

MDN defines 406 as

The HyperText Transfer Protocol (HTTP) 406 Not Acceptable client error response code indicates that the server cannot produce a response matching the list of acceptable values defined in the request's proactive content negotiation headers, and that the server is unwilling to supply a default representation.

In practice, this error is very rarely used. Instead of responding using this error code, which would be cryptic for the end user and difficult to fix, servers ignore the relevant header and serve an actual page to the user. It is assumed that even if the user won't be completely happy, they will prefer this to an error code.

If a server returns such an error status, the body of the message should contain the list of the available representations of the resources, allowing the user to choose among them.

source: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/406

Found the following through more google searches:

https://zendenwebdesign.com/causes-and-fixes-for-406-error-not-acceptable/

and also, in the salesforce community forums, users reported seeing this type of error:
https://trailhead.salesforce.com/trailblazer-community/feed/0D54S00000A93GsSAJ

Related Issues

• closes: HTTPError 406 CustomNotAcceptable encountered intermittently #58

[edgarrmondragon]

LGTM! Thanks @haleemur.